Penn Computing
Computing Menu Computing A-Z
Computing Home Information Systems & Computing Penn

Learning from Others: Comparisons and Good Ideas from Industry and Higher Education

A report from the Benchmarking Workgroup of the
Restructuring Task Force

February 1996

Penn has made a concerted effort to learn from other organizations as it restructures its own computing services campus-wide. The Benchmarking Workgroup was convened to investigate what information service organizations in higher education and industry are doing to support information technologies (IT). Our goal was to look at a variety of organizational structures for providing IT services and determine how Penn compared in terms of services offered, delivery mechanisms, and productivity of resources. We identified examples of best practices, based on recommendations from the restructuring task force and other outside organizations. We also looked at institutions that had recently restructured their computing services (or were in the process of restructuring) to learn from the choices they had made.


The time frame for benchmarking was short. We focused our efforts on developing an overview of what other organizations are doing and how they are structured. Most of the information we collected was descriptive, not quantitative.

The workgroup conducted personal interviews with representatives from 15 top tier higher education institutions. We looked at institutions that are similar to Penn in mission, size, and scope of services. Our project time frame allowed for only one interview at each site (usually with a central IT senior director or CIO). To ensure a broader understanding of the entire computing environment, we augmented the personal interviews with information from each school's Web site and from other published documents.

On the industry side, we gathered information through literature searches and reviews of Gartner Group reports; no personal interviews were conducted with corporations. Because more published comparative data are available for industry than for higher education, we were nevertheless able to compile useful ideas and make comparisons.

The team considers the initial "discovery phase" of benchmarking to be complete. The following summary provides insights into what other organizations are doing and offers data to assess Penn's productivity comparatively. The team was also successful in establishing relationships with key benchmarking partners, opening the lines of communication for ongoing information exchange. Our peers are as interested in what Penn is doing as we are in what they are doing.


The team's most important observations are summarized below:

Overall, Penn has a strong record of accomplishment on which to build. We rank high among our peers in providing network services, administrative computing, and support for instructional technologies. Penn can be considered leading edge with Project Cornerstone architectures, client/server systems, and the data warehouse. At the same time, we appear to be spending a smaller proportion of our institutional budget on information technology (IT) than our peers.

In both industry and higher education, computing is seen as an area of investment, not cost containment. Industry expects to see its investment in computing almost double over the next five years. Many universities have recently made, or anticipate making, significant investments in information technology.

Organizations in both industry and higher education are facing many of the same challenges. An increasing demand for network access and helpdesk services are two examples. The largest and fastest growing information technology budget item in industry is personnel costs – though head count remains relatively flat. Industry IT directors rate training and retraining as their most pressing concern. The relationships we established during the initial phase of benchmarking will allow us to continue tracking how others are meeting these challenges.

Distribution of computing support appears to be the trend in industry, though most IT funding remains centralized. Higher education employs various mixes of centralized and decentralized services.

Penn's new model integrates strategies that a number of forward-thinking institutions are already using (e.g., cross-functional process teams, service bureaus, and enterprise services). Our peers were very interested in Penn's Network Architecture Task Force and its cross-unit approach to recommending standards, which allow us to take advantage of new and emerging technologies. Penn's use of user groups, listservs, and newsgroups is highly regarded among our peers for increasing communication and understanding across units.

Other universities use carrots, not sticks, to encourage standards. They offer many of the incentives Penn uses: favorable pricing, technical support, and links to the emerging architecture. Industry is much more stringent than higher education in setting standards ("The choice is vanilla and everybody gets it.").

Almost 75 percent of companies report they outsource some IT functions, most commonly disaster recovery, network operations, and application development. If they could do it over, many companies would seek tighter controls on deliverables and schedules, a dedicated individual to manage outsourcing, and more flexibility in the contract.

The use of service level agreements is growing: 62 percent of Gartner industry respondents say they use such agreements. Our higher education peers who use service level agreements recommend them as a mechanism for grounding expectations.

No single support model is favored. A wide range of support models are in use in highly successful companies. We found a wealth of examples of "best practice" in select areas, but no single organizational structure that works for everyone. From conversations with leaders in higher education IT, we determined that there is no "right" model.

Asked about lessons learned from restructuring experiences, our interviewees stressed that organizational transitions take time, honest communications, and a personal investment on the part of those involved. What constitutes "success"? Even the experts don't always agree. For example, only five companies appeared on both lists of "top 100 companies" developed by two major information technology publications in 1995.

Ongoing benchmarking activities, both internal and external, can help us understand whether we've been successful, but ultimately it's our end users who will make that determination.

Data Analysis

In order to analyze the enormous amount of data collected during interviews and literature searches, the workgroup focused on a set of key questions about Penn's evolving support model. That approach is used below to summarize our findings and make comparisons with peer institutions and industry. To support our findings, we have included as Appendix A a list of institutions interviewed and a bibliography of articles and literature resources.

In addition, case summaries, interview transcripts, articles, reports, and supporting documentation are available at:

ISC Benchmarking Library
230A, 3401 Walnut Street
University of Pennsylvania
Philadelphia, PA 19104-6228

Benchmarking Summary

  1. Are Penn's resources being used productively?


    Productivity has two aspects, resources used and services received. As anticipated, the wide variations in definitions on both sides of the equation make precise comparisons difficult. However, the available information indicates that Penn's costs are well within the range of those of universities in general and comparable institutions in particular.

    Among universities interviewed, Penn remains competitive in the delivery of IT services. In some areas (e.g., network architecture task force, data warehouse, and information and systems architecture), Penn is considered the institution against which to benchmark. In other areas (e.g., distance learning), Penn has not made an institutional commitment and is not comparable to those who have made this a priority.

    Many universities have recently made, or anticipate making, significant investments in IT; virtually all those against whom we benchmarked view IT as an area for investment rather than cost containment.

    In general, it is difficult to make industry comparisons of productivity because of the differences in the task of the enterprise. Gartner Group data indicate that education as an industry uses a higher proportion of revenue for IT than any other industry except computer related businesses. Total IT investment and IT investment as a percent of revenue across industries is expected to almost double over the next five years.

    A more detailed presentation of the cost data follows.

    Costs as a Percent of Revenue:

    Gartner Group estimates of IT spending as a percent of revenue and hidden IT spending as a percent of central IT budget are shown in Table 1 below.

    Table 1
    Gartner Group Central IT Budget as Percent of Revenue Other IT Spending as Percent of Central IT Budget Total IT Spending as Percent of Revenue
    Survey Average 2.6% 28% 3.3%
    Education 4.2% 56% 6.6%
    Pharmaceuticals 2.1% 13% 2.4%
    Hospitals & Health Care 2.3% 21% 2.8%
    Government 1.9% 38% 2.6%
    Banking 4.3% 29% 5.6

    Penn's estimate of $50 million spent enterprise-wide on IT approximates Gartner's estimated average of 5 percent of revenue spent by universities.

    Interviews with comparable universities indicate that they have not systematically tracked the cost of IT across their institutions. As is true at Penn, much of the cost of IT is outside the central IT organization. The cost of central IT groups (as a percent of revenue) ranges from 1.6 percent to 6.1 percent. Penn falls well within this range, with ISC spending (approximately $20 million, including allocated or billed cost) accounting for 2.01 percent of revenue.


    1. Gartner Group estimates that IT costs are underestimated by virtually all industries for many of the same reasons that make it difficult for Penn and other universities to accurately assess cost. The rate of spending outside the central IT group differs across industries, but is believed to average roughly 28 percent of central IT, and 56 percent of central IT for education. To the extent that Penn's $50 million cost estimate (enterprise-wide) is more accurate than the estimates of comparable institutions, Penn's actual costs may be lower than might otherwise be expected.

    2. The purview of central IT groups within universities varies widely, with some including telecommunications, others including computer resale, and still others including support for the medical center. In addition, some comparable universities are more centralized than Penn and may include a higher proportion of total costs in the central estimate. Given the wide range of estimates provided by these schools for central IT costs, we believe Penn to be well within that range.

    Allocation of Costs by Category of Expense:

    Personnel costs are the largest IT budget item and the area of greatest growth. Central IT head count represents approximately 5 percent of the total head count in large enterprises. Penn's central IT head count is less than 1 percent (.83 percent), and about half of the average for education. The following table (Gartner Group) presents central IT head count as a percentage of total employee head count:

    • Survey Average 5.2%
    • Education 1.6%
    • Pharmaceuticals 4.1%
    • Hospitals & Health Care 2.8%
    • Government 3.3%

    IT personnel expense is growing and expected to continue to grow, even though head count has been rising slowly or not at all; higher cost, skilled service providers are being substituted for lower cost operations personnel. Consistent with the shift in the types of individuals needed within the IT organizations is the increasing importance of training and retraining of personnel. The biggest issue facing IT organizations in their efforts to maintain top levels of performance is providing staff training in new technologies with no growth in training budgets.

    The Future:

    Industry estimates are virtually unanimous in forecasting costs increasing as a percent of revenue. Estimates of the rate of increase vary widely. Gartner Group analysts note that average IT spending is expected to grow to 9 percent by the year 2000.

    In Table 2 Computerworld data show IT spending as a percent of revenue and percent of annual growth.

    Table 2
    Computerworld IT Spending as
    Percent of Revenue
    IT Spending as
    Percent of Annual
    Survey Average 2.6% 12.8%
    Manufacturing 8.9% ---
    Financials 6.4% ---
    Health Services & Pharm. --- 18%
    Building Contractors 0.2% 38%
    Food Services --- 21%

    Many universities have recently made, or anticipate making, significant investments in information technology. The University of Michigan, for example, expects its investment in IT to increase because it is buying higher-end products to remain competitive and because sheer volume is driving up support costs.

  2. How does Penn compare with other organizations we've studied in the following dimensions: networking, instructional technologies, standards, and administrative computing? Do we appear to be going with the trend of change or against it?


    Penn's networking services are in the high end among institutions we studied. This is based on our campus and dorm wiring strategies, networking technologies, and access support. In one area Penn seems to be ahead of other schools: Penn's Network Architecture Task Force looks ahead over the next 3-5 years and sets standards that can accommodate emerging technologies.

    Most institutions in higher education and industry have a centralized network infrastructure. Additionally, the trend in industry is for local area networks to be maintained and managed centrally – companies find it too expensive and too much trouble to distribute LAN experts. Standardization makes this possible. Higher education generally appears to be ahead of industry in the deployment of networks, particularly enterprise-wide networks and use of the Internet. Many companies, for example, are not using email yet, and others are only using it at the workgroup level rather than at the enterprise level. However, industry is catching up quickly.

    Of the universities studied, over half specifically stated that they had merged telecommunications and network services. This is an important area of followup for the benchmarking group. We will probe structures and economies and query the remaining institutions.

    Instructional Technologies:

    Among the institutions interviewed, Penn is on the high end in providing support for instructional technologies. This is based on our use of networking and instructional technologies in the classroom and provisions for faculty and student access to tools, technologies, and support. We consider several schools "best practice" in certain areas: Northwestern, for its Technology in Learning and Teaching (TILT) program which pairs faculty and staff for training on network tools; MIT, for its distributed outreach focus across units and commitment to center/school collaboration; and Michigan, for its organized effort to support the infusion of technology into teaching and learning. Other schools are ahead of us in distance education infrastructure and support, but Penn has not identified this area as an institutional priority.

    Also worth noting is a new instructional support group at Stanford that is staffed with computer experts who also have discipline knowledge and work in departments to support faculty. The group was created on the recommendation of a committee on technology and learning, and established with money from the president and provost.

    In many institutions the library plays a key role in technology training, particularly at UC Davis, USC, and Washington.


    Industry appears to go further than higher education in setting standards, but doesn't allow for flexibility to meet the range of diverse needs we find in universities. Often in industry "the choice is vanilla and everybody gets it." In higher education, schools that are behind on the technology curve appear to be in a better position now to establish standards because they don't have to accommodate large installed bases. At Penn, installed bases and decentralized authority have made it more difficult to establish standards. One notable exception is the ResNet program. It was developed when the dorms were behind the curve in technology, and allowed Penn to institute strong standards. The ability to adopt an industry wiring standard in 1992 has made Penn a leader in ResNet today.

    Some companies are facing the same standards problem as universities, that is, divergent hardware platforms and database systems at their subsidiaries. Mergers and buyouts have caused some of these standards problems.

    Most higher education institutions use the "carrot approach" to encourage adherence to standards. They offer many of the incentives Penn employs: favorable pricing (or no cost access), technical support, and links to the emerging architecture. Typically, central IT plays a guiding role, with the campus participating in setting standards. The key recommendation here is to pay attention to the culture. A realistic view of the institution is required.

    Administrative Computing:

    In industry, corporations with a data warehouse, database standard, and client/server applications are seen as leading edge. These criteria put Penn in the leading edge category. This rating is also consistent with Penn's standing among its peers in the Ivy Plus consortium (a semi-annual information-sharing gathering of administrative computing directors of the Ivies plus MIT and others). We continues to hear from Ivy Plus members that Penn's original principles for Project Cornerstone, information technology architecture, and client/server deployment serve as models for others.

    Of companies surveyed by Computerworld, 86 percent are using some form of client/server, but maintain a healthy degree of skepticism. Some companies are backing off on client/server, maintaining that not all applications should or can be client/server. On average, 20 percent of IT budget is spent on client/server projects, with a growth rate of 45 percent. Fifty-seven percent report complexity of multivendor environments (many failure points) and 55 percent report lack of trained personnel as the most serious problems with client/server.

  3. How are other organizations delivering services? How are they staffed, managed, and funded?


    The organization of IT support in industry runs the gamut from highly centralized to highly decentralized. Some companies are decentralizing; others are centralizing after a period of decentralization. The trend, however, appears to be toward decentralization. But despite talk of decentralizing IT, IT budgets for the most part remain centrally controlled. Computerworld industry data show the following:

    • 79% of IT spending is managed centrally
    • 13% of IT spending is managed by divisional or functional IT managers
    • only 8% of IT spending is managed by non-IT managers

    In higher education, the data suggest that in most instances general computing support is provided in a mix of centralized and decentralized services. With a few exceptions, the trend appears to be toward general "shrink wrap" support at the central level and custom support at the local level.

    Primary Support:

    Different approaches are employed across institutions, but most of them are based on a mix of central and local IT support. UC Davis has shifted its focus to primary support at the local level. User groups, listservs, and newsgroups are highly recommended to support this approach, providing forums for people to share information. Wisconsin provides primary support at both the central and local levels, with central assuming responsibility for high-volume general support and local groups providing custom services.

    Secondary Support:

    Again, different approaches are employed, involving a mix of central and local support staff. UC Davis' central IT has shifted its focus to supporting primary support providers in the units. Central and local participation in user groups, listservs, and newsgroups helps make this model successful. To effectively provide secondary technical support at the central level, central IT must have staff who are at least as knowledgeable as the local gurus generally, and more knowledgeable in specific areas.

    Network Services:

    Across the board, network backbone is provided as a central infrastructure service and is almost always funded centrally. We found no examples of public utility commissions in use as a form of governance for network infrastructure services. A big issue facing many of our peers is supporting the ongoing cost of the network. UC Davis has a committee of faculty and IT staff exploring options, which will shortly present a funding model to the provost. Penn's Network Architecture Task Force makes recommendations for network improvements, but getting to implementation is often a hurdle due to the lack of a funding mechanism.

    In industry, software upgrades are provided as a network service (and as added incentive to support standards).

    The "intranet," as opposed to Internet, has been the subject of recent articles in the computer press. Intranet is described as a kind of enterprise internet in which the World Wide Web is used as the "client" in client/server applications and for enterprise-wide (or division/department-wide) information dissemination, collection, or exchange. The Penn Web is headed in this direction, and the Wharton School has already begun to implement an intranet system for its internal needs.

    Service Bureaus:

    It's difficult to summarize trends or practices in the area of service bureaus, or market-based "small businesses." We didn't ask specifically during the interviews, and higher education and industry refer to this type of service in a variety of ways. However, the examples we did uncover are all central. Michigan, for example uses central charge-back service bureaus; and Northwestern, though providing a baseline level of network services as a utility, sells "gold pool" modem services as a service bureau. Hawaii, on the other hand, discovered that the cost of metering services exceeded the cost of provision. They simply tack an additional charge onto the telephone bill to cover network services.

    This is an area that clearly needs followup. UC Davis mentioned the "competency center" concept specifically, but it's not clear how they are managing this type of service delivery. MIT's service bureau model is clear for training and applications development, but it is unclear how expertise is disseminated, managed, or charged. Stanford has computer experts with discipline knowledge in a competency center approach, but again specifics are unknown. Wisconsin is seeing a trend of local units going to the central group to hire local staff.

    Enterprise Planning and Brokering:

    Standards setting is one area of enterprise planning that is often coordinated at the central level, with participation across units. Another common enterprise service is site licensing. Penn appears to be ahead in the management of its program and central IT strategic license fund, but is behind other institutions in network distribution of software. In a good example of brokering, MIT's contracts management group is responsible for brokering, and the integration group is responsible for maximizing technology investments across the institution.

    Process Teams:

    Process teams are in wide use in industry, with teams being built as needed. In the past, budgets were built to allow for the temporary replacement of unit personnel who participated in project teams. With downsizing, these budget accommodations no longer exist and employees are expected to participate on teams and do their "real jobs." The problem appears to be widespread, but it is unclear how it's being resolved.

    MIT and Emory provide the two best examples of process teams in action among the universities we interviewed. MIT's central nomadic discovery and delivery team is plugged into projects as needed, and Emory's quality training and certification group is a cluster of cross-divisional teams working on cross-divisional projects and processes. Also worth noting are Michigan's cross-functional teams for administrative financial systems. In this model, process owners across the university decide what gets built, when, and for whom (student systems, personnel, alumni).


    In industry, 73 percent of companies surveyed by Computerworld said they outsource some IT function, with small companies doing more outsourcing than large companies. On average, 10 percent of IT spending goes to outsourcing. Companies with sales revenue below $500 million spend 15 percent on outsourcing, and companies above $1 billion spend 8 percent on outsourcing. Disaster recovery (48 percent) is the most common outsourced function; network operations (29 percent) is next, followed by applications development (25 percent).

    Survey respondents in industry were asked, "If you could do it over, what would you change in your outsourcing contracts?" Answers ranged from more accountability and tighter controls on deliverables and schedules to a dedicated individual to manage outsourcing and more flexibility in the contract.

    In a noteworthy industry example, the internal central IT organization submitted competitive bids against external vendors. Often, the internal IT group won the contract. Their knowledge of the business and the existing systems made them much more competitive.

    Among universities, the service most commonly outsourced is hardware repair and maintenance for desktop machines and UNIX workstations. Some institutions, such as Northwestern, are investigating outsourcing to meet rising demand for remote access to network services. UC Davis is looking to do the same, but also hopes to outsource support for these services (to relieve the associated demand for technical support for remote access).

    According to an informal outsourcing survey of university CIO's, the following functions are also being outsourced: programming/application development, consulting, mainframe operations, and wiring installation and maintenance. The survey concludes that outsourcing can be quite successful when used selectively to augment existing staff and for tasks where specific expertise is needed for a limited period of time. It also notes that vendors need to be managed more carefully than in-house staff, and that good contracts and specifications are key.

    Service Level Agreements:

    The use of service level agreements is growing. For example, 62 percent of Gartner Group survey respondents use them. Gartner Group points out an important problem associated with service level agreements: Often not everything that is needed to keep the customer happy is included in the service level agreement. IT groups can meet every detail in a service level agreement ("working to rule") but the customer is highly dissatisfied because something critical was overlooked in the original agreement, or priorities changed, or a new critical need developed.

    Our higher education peers who use service level agreements recommend their use as a mechanism for grounding expectations. Agreements should be as well-defined as possible, and reviewed/updated on a regular basis. In Michigan's "Partnership with the Deans" program, service level agreements are revisited at least annually. For each partnership, one representative from each side (school/central) meets regularly to determine where changes might be needed.

  4. What are others doing to support and nurture innovation? How is communication and integration across units handled?

    In industry, some organizations use advanced technology groups to foster innovation and develop new technologies. Moving personnel around to different project teams and different business units is another way some businesses support innovation and improve communications between staff in central IT and business unit IT

    Some schools, such as Princeton, have institutionalized advanced technology groups to assist in nurturing innovation. Others, such as Northwestern, have disbanded their advanced technology groups, responding to concerns that they concentrate on a select few end users, and their services are directed at special cases which usually aren't scalable.

    Michigan's Partnership with the Deans program has been very successful in advancing communication and integration across units. It has shown deans how central IT can better align with school groups. The goal of the program is to improve communications and increase overall technology access and capability in the university.

  5. What problems does Penn share with its peers and industry? What have we learned about how others dealing with these problems?

    We found 3 areas of universal concern among higher education institutions:

    • meeting demand for help desk services
    • meeting demand for remote dial-in access
    • meeting demand for seats in public-access labs

    To address these problems, some schools are defining as clearly as possible baseline services to be provided by the help desk (with possible value-added service bureau options). Columbia, for example, has instituted a 900 number for help desk calls. Others are investigating outsourcing for remote access. Auburn University, for example, now provides dial-up access for $12 per month to the university's host computers and the Internet, through a contract with MCI (offering 15 hours of 28.8 Kbps access per month, no busy signals, full graphic displays, and 24-hour technical support). To address the rising demand for access on campus, Northwestern is currently using computing labs and classrooms interchangeably.

    Issues on the industry side are somewhat different. According to the Gartner Group, the following critical issues are faced by IT organizations:

    1. Training IT staff in new technologies 81%
    2. Improving perception of IT among business management 74%
    3. Funding to keep up with demand and maintain quality 70%
    4. Educating business managers about how IT might help them 60%
    5. Developing a methodology to prove IT's return to the business 54%
    6. Educating IT staff on customers' business needs 54%
    7. Determining how much end users spend on IT 35%
    8. Improving the morale of IT staff 32%
    9. Competing with outsourcers 19%
    10. Changing IT from a cost center to a profit center 16%

  6. What can we expect before, during, and after the transition period?

    The perception of how well a transition is working can differ significantly among parties involved within the same institution. It's very important not to underestimate the need for clear, ongoing communication with both IT staff and end users.

    UC Davis emphasizes that formal agreements, structural changes, and new processes mean nothing unless people are willing to make a personal investment in the reorganization.

    MIT advises that the restructuring effort is not a sprint, it's a marathon. It takes time to turn organizations around. Over communicate. Make sure everyone on the inside is saying the same thing and is listening as well as talking. UC Davis stresses the importance of strong leadership to make organizational changes. They also stress that staff need to understand how they link to the organization and its goals. In its reorganization efforts, Minnesota seeks to build trust and confidence within and outside of the central IT organization. Wisconsin notes that the "construction zone" period is inevitable, and that IT staff and end users should be encouraged to expect this and understand it as part of the transition process.

    In industry studies, resistance to change and personnel retraining are cited most often as the biggest obstacles to overcome in making a smooth transition.

  7. What criteria are being used to determine excellence or success?

    A Gartner Group survey lists the top ten criteria management uses to assess the IT organization's performance. The criteria and the percent of companies using them are:

    1. Availability/reliability of installed systems 100%
    2. Ad hoc feedback from customers to management 98%
    3. Ability to reduce or contain costs 98%
    4. Speed/quality of applications development 96%
    5. Contribution of IT to specific business goals and strategies 92%
    6. Overall IT spending (e.g., $IT/$revenue) 86%
    7. Contribution of IT to financial performance of the company 85%
    8. Cost-efficiency of specific IT services 75%
    9. Formal customer satisfaction surveys 72%
    10. Formal service level agreements with customers 62%

    The universities we interviewed were unable to provide clear criteria. According to a 15-month study financed by an Education Department grant (The Chronicle of Higher Education, January 19, 1996), many college officials "don't have a clue" about how their campuses are affected by networking, how it is used, what works best, or how to improve it. The study's chief researcher, Charles McClure, a professor of information studies at Syracuse University says, "We have found it's sort of taken as a given now that you absolutely, positively must have a good information infrastructure – simply to attract students and retain faculty." "When you ask people 'Do you teach better because you have a good campus network?' they say, 'Oh, yeah, yeah.' But when you say, 'How do you know?' then the conversation becomes silent."

    Mr. McClure and his associates have completed the study's main byproduct – a manual entitled "Assessing the Academic Networked Environment: Strategies and Options." Academic decision makers "will increasingly ask others on campus for evidence that particular services and activities contribute to the overall success of the institution, that the networks are operated efficiently, and that they support specific institutional goals," a draft of the manual says. It says the evidence should focus on such matters as technology costs and usage patterns, and provide an understanding of users' views. Among the issues it suggests should be explored is whether people think that campus networks have changed their experiences in teaching, learning, research, and administration.

  8. What measures of customer satisfaction are being employed by other organizations?

    Although organizations are regularly surveying and measuring , there don't appear to be any institutionalized efforts that clearly translate into ensuring customer satisfaction. A lot of data are being collected, but it's unclear how they are being used.

    Michigan's service level agreements and Partnership with the Deans program appear to be steps in the right direction. Michigan recommends not counting or measuring more than you're prepared to take action on. At Northwestern, all departments are subject to internal and external reviews every five years. How the data are used to adjust services is unclear.

    Industry seems to depend heavily on ad hoc feedback from customers to judge success. According to a Gartner Group survey, 98 percent of IT organizations stated that customer ad hoc or informal feedback to management plays a very significant role.

  9. What else have we learned that is too interesting to ignore?

    Equipment Lifecycles:

    At Northwestern, the vice provost for information systems (VPIS) is on the verge of getting the university to build into its budget replacement costs for all campus computers, with deans putting money into a pot in exchange for additional central money (3-year life cycle). The VPIS has convinced the deans of the need for this.


    The UC Davis Information Technology Strategic Planning Committee's report, released in December 1992, includes several key findings which led them away from the then-current recharge (charge-back) model. The recommended model provides a base level of service, funded centrally by a campus allocation, at no visible cost to the individual user or department. This change was made to address concerns that the recharge (charge-back) model limited access to, and innovation with, information technologies, as people made decisions on the basis of recharge avoidance instead of the most effective and efficient solutions for both the unit and the campus. They also found that campus units were spending a substantial amount of time and money charging each other for services.

    Best Practices:

    Two major information technology publications, Computerworld and CIO, both produce annual reports of the top 100 companies most effective in managing information. In comparing the two reports for 1995, only 5 companies appear on both lists:

    • Charles-Schwab Corporation
    • The Coca-Cola Company
    • MacDonald's Corporation
    • Merck & Company, Inc.
    • Solectron Corporation

    Thus, even among well-regarded reviewers, criteria for determining "best practice" are inconsistent.


    Ninety-three percent of all companies surveyed by Computerworld are doing some form of re-engineering; approaches and results differ widely. Survey respondents focus re-engineering efforts on:

    • finance and accounting 57%
    • customer service 55%
    • order fulfillment 39%
    • marketing/sales 32%
    • manufacturing 32%
    • logistics 32%
    • product and service development 21%

    A wide range of support models are being used in highly successful companies, with no one model predominating. The one constant is that most companies are continually searching for ways to improve. Flexibility is key. Most companies want to be flexible enough and responsive enough to adapt quickly to changing circumstances. None of the universities we interviewed could recommend an institution with the best support model, although they could identify best practices in select areas.

  10. What followup benchmarking activities would we recommend to the Implementation Task Force?

    Penn State, which undertook a major benchmarking study several years ago, emphasizes that successful benchmarking takes time and money. True benchmarking is more than a quick "snapshot" of an organization at a given time. In order to fully understand benchmarking data, we need to understand the culture of an institution. Now that contacts have been established through initial interviews, we recommend ongoing information exchange with our benchmarking partners. As we identify areas of "best practice" among the institutions we've studied, we encourage making site visits to enhance our understanding of their services (e.g., Michigan's Partnership with the Deans Program).

    All of the data on industry came from published sources. We should follow up with personal interviews at the most interesting sites, using a similar question set to the one used with educational institutions. We recommend Merck & Company, Inc. as a starting point for several reasons: their ranking among the top five organizations in IT support in two major studies, the parallels between pharmaceutical and higher education institutions, their close proximity to Penn, and their broad reputation for excellent overall management.

    Members of the benchmarking and models workgroups will participate in a series of teleconference briefings with Gartner Group industry analysts to help us better answer the questions about implementing the new support model. We have devised a set of very focused questions to frame our discussions and to ensure that the most appropriate analysts are consulted.

    If Penn intends to make a strong commitment to benchmarking as an ongoing activity, we may consider joining a benchmarking clearinghouse or consortium. Although the best resources are expensive, they would allow us to stay in touch with best practices across industries. Some options include:

    • Benchmark Exchange, a Web-based resource
    • Gartner Group Best Practices Benchmarking Service
    • American Productivity & Quality Center International Benchmarking Clearinghouse

Appendix: Schools Participating

Cornell University
Emory University
Massachussetts Institute of Technology
Northwestern University
Penn State University (re: Benchmarking study)
Princeton University (Advanced Technology Group)
Rice University
Stanford University
University of California, Davis
University of Hawaii
University of Michigan
University of Minnesota
University of Southern California
University of Washington
University of Wisconsin


  1. 1994/95 Information Technology Budgets and Practices Survey, Gartner Group, November 18, 1994.

  2. Burns, Laurie, Cheryl Mun-Fremon (University of Michigan), "Partnerships with the Deans: Delivery of the Whole Product," CAUSE94.

  3. Duwe, Jack, Mark Luker, Tad Pinkerton (University of Wisconsin, Madison), "Restructuring a Large IT Organization: Theory, Model, Process, and Initial Results," CAUSE/EFFECT, Summer 1995, pp. 24-30.

  4. "The Eighth Annual CIO 100, Practice Makes Perfect: Delivering Best Practices Where They're Most Needed," CIO, The Magazine for Information Executives, August 1995.

  5. "Forecast '96," Computerworld, December 26, 1995.

  6. Huber, Richard L., "How Continental Bank Outsourced its Crown Jewels," Harvard Business Review, January-February 1993.

  7. Jackson, Robert L., "Study Finds Colleges are Ignorant of How Their Computers are Used," Chronicle of Higher Education, January 19, 1996, p. A22.

  8. "The Premier 100: The Most Effective Companies at Managing Information," Computerworld, October 9, 1995.

Information Systems and Computing
University of Pennsylvania
Comments & Questions

University of Pennsylvania Penn Computing University of Pennsylvania Information Systems & Computing (ISC)
Information Systems and Computing, University of Pennsylvania