Evaluating Research Productivity
no. 105 - June 2001

EVALUATING UNIVERSITY RESEARCH PRODUCTIVITY:
What's the ROI . and who cares?

R. W. Trewyn
Vice Provost for Research and Dean of the Graduate School
Kansas State University

Public research universities face many challenges in the 21st century, not the least of which involves documenting the value-added outcomes that derive from the teaching, research, and public service missions of the institution.  Governing boards, accrediting bodies, funding agencies, state legislators, taxpayers, and the American citizenry in general want to know.  Prospective students and their parents want to know: what sort of return on investment (ROI) can they expect?  In the new millennium, ROI is a concern of more than just investment bankers and stockbrokers. And universities - just like other entities seeking monetary investments - will be well served if they can provide compelling answers to questions about the ROI they generate in fulfilling their missions. 

Teaching Value-Added

The annual teaching/education value-added for an institution can be estimated from the number of alumni residing in the state by using U.S. Census Bureau data for mean annual earnings by level of education.  For example, there are 45,564 alumni in Kansas who received bachelor's degrees from Kansas State University (KSU). An individual with a bachelor's degree earns $19,114 more annually (on average) than someone with a high school diploma, so one can estimate the economic value gained from an undergraduate education by multiplying 45,564 x $19,114 = $870,910,296. 

A recent summary of the value-added earnings of KSU graduates in Kansas for all levels of education projected an annual value-added impact of more than $1 billion, with nearly $3 billion in total annual earnings (see attachment).  The ROI to the state of Kansas was estimated to be almost seven-fold, that is, for every $1 the state invested annually, it was estimated that $6.80 is added to annual earnings.  Moreover, KSU graduates are likely contributing significantly more in the way of tax revenues to the state than the state is providing to the university each year. 

Public Service Value-Added

The monetary value of university public service activities can also be estimated if appropriate tracking systems are in place to monitor the time spent by faculty, staff, and students in providing such services. At land-grant institutions, extension service activities can be quantified, so this offers one specific measure.  Unfortunately, most other university units don't track public service activities particularly well, if at all. 

This difficulty became apparent in a recent assessment of the economic impact of teaching, research, and service at KSU, where it was found that quantifiable public service outcomes were small compared to those generated by teaching or research.  As a result, these service end-points were merged with teaching or research in the final report. 

Research Value-Added

Determining the ROI for university research is unquestionably the most difficult parameter to establish.  Consequently, other parameters are sometimes used to assess the impacts of research.  Job creation, which is an important consideration from an economic development standpoint, is one such indicator that is commonly utilized. And funding for university research and scholarly activity creates jobs in the same way that R&D funding does in the private sector. 

As illustrated below in the fifteen-year summary of funding in support of research and scholarly activity at KSU, the base for FY 2000 approached $150 million.  These funds came from a variety of sources: competitive (extramural awards), donated (transfers from the KSU Foundation to the university), and appropriated (predominantly legislated land-grant support). A research and scholarly activity funding base at this level would create or support about 6,000 jobs as these funds are expended, based on a Kansas-specific multiplier of 40.6 jobs created per $1 million in research support (derived by the AAU using Commerce Department statistics).

KSU Research/Scholarly Activity Funding Base

While the ROI associated with basic research cannot be quantified in economic terms, we can make reasonable estimates from the outcomes of applied research.  Land-grant universities have been doing this for years in the agricultural realm where agricultural economists provide relevant data on a regular basis.  The 1998 publication entitled, Value Added: The Economic Impact of Teaching and Research at Kansas State University, estimated an annual impact of nearly $1 billion from research, and as already noted, this undoubtedly is a significant underestimate of the total impact because the outcomes from basic research cannot be quantified.  Moreover, many non-agriculture departments failed to track the impacts of their applied research. 

The insert at the left illustrates how the value-added benefits of applied research can be calculated. In this example from 1994, the specific contribution from KSU to the $1.5 billion Kansas wheat harvest amounted to $64 million.

The 1998 value-added report established an economic impact in Kansas of more than $2.4 billion from teaching and research combined.  Based on a state allocation to KSU of $145 million the previous year, a return on investment of seventeen-fold was established, that is, the university returned $17 to the state for every $1 of state funding.

Research productivity can also be evaluated by assessing the outcomes of university technology transfer efforts. The return on investment in this instance can come in many forms, including signing fees and royalty payments from traditional patent licensing arrangements, and related research funding that goes back to the institution.  Additionally, launching start-up companies can lead to job creation, equity acquisitions, and a variety of revenue streams. 

Communicating Value-Added Outcomes

All institutions - public and private - have customers and stakeholders that need information about ROI and other productivity information packaged in an institution-specific manner. Every effort should be made to provide such information, tailored to the needs of the particular customers or stakeholders, whenever possible. 

On the other hand, when a single governing board serves multiple universities and/or when multiple universities are located in a single state, it may be advantageous to provide pooled productivity data. 

For example, the chief research officers at the University of Kansas, Wichita State University, and Kansas State University presented a combined Research Infrastructure Report to the Kansas Board of Regents on March 16, 2000.  This report contained economic impact estimates for alumni of the three research universities who live in Kansas, and joint R&D expenditure data; no institution-specific information was included. Combined ROI information was also provided for the three universities, which included return on state investment in education and jobs created by Research and Development activities. The collective outcomes were substantial and impressive. 

Clearly, the most effective way of communicating value-added outcomes is to make them audience specific.  One size does not fit all.  And we need to listen to those customers and stakeholders. They care, and the ROI data we provide can return huge dividends.

ATTACHMENT: Estimated Economic Impact of Graduates in Kansas: Kansas State University

(1) The number of KSU graduates residing in Kansas.  Data provided by the Alumni Association as of November 1999. 

(2) Based upon U.S. Census Bureau data regarding the mean annual earnings by level of education for those in the workforce aged 18 and over.  For 1998, the mean annual earnings by degree/diploma level were: high school, $30,868; associate's, $38,483; bachelor's, $49,982; master's, $60,168; doctoral, $69,188; professional, $90,653.

(3) The value-added impact of the education for alumni living in Kansas was calculated as: associate's, $38,483 - $30,868 = $7,615; bachelor's, $49,982 - $30,868 = $19,114; master's, $60,168 - $49,982 = $10,186; doctoral, $69,188 - $49,982 = $19,206; professional, $90,653 - $49,982 = $40,671.  To reduce the potential for double counting and overestimating the impact, the value-added differential was calculated on the mean earnings for an associate's or bachelor's degree minus those for a high school diploma and on the mean earnings for a master's, doctoral, or professional degree minus those for a bachelor's degree.  As a result, individuals who received their bachelor's degree from one university and their master's, doctoral, or professional degree from another (a common occurrence), would have the appropriate portion of the education value-added earnings credited to the appropriate institution. 

(4) Professional degrees include only DVM degrees. 

(5) According to the Kansas Department of Revenue's Office of Policy and Research, the average for taxes paid in Kansas (State income tax, sales tax, and property tax) amounts to approximately 8.3% of personal income.  Based on $2.99 billion in estimated earnings, KSU graduates would be contributing $250 million annually to the tax rolls in Kansas, which is $100 million more than the state appropriates to KSU. 

(6)  In FY99, the State of Kansas appropriated $151.2 million to the general operating budget of KSU (Kansas Board of Regents Data Book, February, 1999).  Using this current-day financial subsidy, the state's annual return on investment (ROI) just for the education value-added portion of alumni earnings would amount to $6.8 million ($1,027.9 million divided by $151.2 million), i.e., for every $1 invested by the state, $6.80 is added to annual earnings in Kansas. However, that 7-fold return doesn't take into account the value-added benefits of research and public service activities at KSU, which increase the total ROI to 17-fold or more based on the results of a recent research value-added study.