Evaluating Research Productivity
June 13-15, 2001
2001 marked the fifth year for the Merrill retreat series: The Research Mission of Public Universities. Twenty-six university administrators and scientists came from the four-state region of Kansas, Nebraska, Iowa and Missouri. They compared best practices for ranking universities, promoting research, and evaluating faculty in the context of graduate education.
The keynote speaker was Dr. Joan Lorden, Associate Provost for Research and Dean of the Graduate School at the University of Alabama at Birmingham.
She critiqued the evaluation system and rankings published by the National Research Council in 1995. Her keynote address at the Merrill retreat was "Measuring Success: Lessons from the NRC Study of the Research Doctorate."
In the Know has a two-part interview with Joan Lorden on "Keeping the Value in Education."
The 2001 publication -- Evaluating Research Productivity (.pdf) -- is available online and in print from the Center.
Conference Summary by Joan Lorden:
1. Reasons to measure research productivity:
- to promote progress toward goals
- to improve institutional image
- to reward success
2. We become what we measure, so it is important to choose wisely and be consistent with university goals and missions.
3. The measures we use should be determined by the audience we are addressing. What is suitable for the academy may not play well outside it. Economic development measures and market measures may be appropriate for talking with elected officials, but they may also be insufficient. We must provide context and relevance with stories about the products of our research and their impact on our students and patients.
4. To measure faculty performance, we must use measures that are different from those we use to capture institutional or unit performance. Mindless counting is insufficient. In evaluating faculty, we should capture the originality of the work and the impact of the research.
5. Are federal dollars the "gold standard" for measuring research performance? This money would not come from the state and it provides jobs, purchases goods and services, etc. But are federal dollars a measure of quality? Federal research grants are peer-reviewed and funding from multiple sources suggests a consensus of one's peers, but this should not be the sole measure.
6. What is the right size for an academic program? Program size has a significant impact on the way univerisities are ranked by reputation. Programs need focus, but a niche program may never attract national attention. Through collaboration it may be possible to achieve an increased perceived size. Linking a group of specialty programs within a university or even across universities may offer a way to enhance visibility.
7. Enhancing productivity requires leadership at all levels, the right set of economic rules, and administrative structures that are not bureaucratic. It is important to align incentives with goals.