Evaluating Research Productivity
no. 105 - June 2001

Administrative Perspectives

KEEPING A BROAD PERSPECTIVE ON RESEARCH

Rollin C. Richmond
Provost
Iowa State University of Science and Technology

Introduction

The background and interests of administrators inevitably influence how we perceive the research contributions of our colleagues. I have spent the majority of my professional career engaged in the study of evolutionary processes mostly at the molecular level.  It is an exciting time to be an evolutionary biologist since the tools of molecular biology have enabled biologists to ask and often answer fundamental questions about the origins and evolution of organic diversity.  It is now a commonly, although not universally, accepted fact that evolutionary processes have created the organic world as we see it including the human species and the full array of its culture. 

E.O. Wilson (1998) in his sweeping volume, Consilience, propounds the view that human culture can be unified under the banner of the scientific paradigm. Wilson envisions a time when scholars in the humanities and the social sciences will recognize that they study phenomena that are as subject to scientific investigation as is the analysis of human disease or the fundamental properties of matter.  I wish to make a similar, though much more limited point, in this paper.  As academic administrators who evaluate the research contributions of our colleagues, we need to take a broader perspective on what we consider high quality research. Moreover, we need to realize that the academic disciplines, including that of education, share more in common, as Wilson would teach us, than is the general view. 

Much of the academic world accords higher prestige to research that seeks to elucidate the fundamental properties of our world.  Research whose goal is to advance the daily lives of people or to improve existing processes often is not accorded the kind of recognition academics reserve for the study of fundamental questions.  Given the often-biting criticism leveled at scholarship in the academic world, I believe it is time for us to reconsider the values we attach to research productivity in the hope that our research culture will respond to the real and often un-addressed needs of human society.

The Critics' View of Academic Research

Martin Anderson (1996) in his book, Imposters in the Temple, writes, 

"The dirty secret of academic intellectuals is that much of what they write and hold up to themselves and to the rest of the world as the highest expression of what they do is inconsequential and trifling. Taken as a whole, academic research and writing is the greatest intellectual fraud of the twentieth century." 

The management guru Peter Drucker in an interview in Science (July 18, 1997) provides a second critical view of academic research.  Drucker is reported to have opined, "I consider the American research university of the last forty years to be a failure.  The great educational needs of tomorrow are not on the research side but on the learning side."  This is a devastating criticism that academics worldwide must address.  Despite the fact that many academics believe teaching and learning are important - indeed the most important missions of their institution - few academics have engaged in scholarship designed to investigate the most effective ways of helping students learn the disciplines they care for so deeply.  Few scholars have even read the existing literature that provides valuable information on learning styles, including the factors of age and experience, and the most effective processes for learning.

So much is at stake in our world that ultimately must be addressed by teaching people how to analyze their situation and respond to it in an effective manner.  The great problems of the developing world - overpopulation, poverty, disease, illiteracy, and corruption - all must be confronted by the people who experience them.  However, the academic world could do much to help if we provided models of the best way to teach people and do it quickly.  Academics in the developed world have contented themselves for many centuries with the same approaches to education as they themselves experienced.  How is it that we have largely failed to apply the tools of science to learning?

A part of the answer to this question lies in the way in which we evaluate research and the values we espouse in making these judgments.  Let's begin by enumerating the characteristics of a scholar and the values that scholarship invokes. 

The Qualities of a Scholar

There are three general characteristics that good scholars exhibit (see Glassick, C., Huber, M., and Maeroff, G., 1997).  First, a scholar must have integrity.  By this is meant truthfulness, fairness, the absence of fraud and dissembling.  A scholar of integrity should use her talents to advance the general good of her society. A scholar of integrity gives credit to those whose work she builds upon and to those who have assisted her. Second, a scholar must persevere in his work and disseminate his scholarly findings to others.  Without the critical review of colleagues, scholarship may well lack rigor. Finally, a scholar must show courage, sometimes at significant personal risk, in her search for truth.  The system of providing academic tenure is a means of encouraging scholars to show courage and take risks with their scholarship so long as such efforts advance the search for truth. 

The Criteria for Good Scholarship at Research Universities               

Ernst Boyer of the Carnegie Institution for the Advancement of Learning argued in Scholarship Reconsidered (1997) that the evaluation of faculty performance in research universities should be changed to include the expectation that faculty will be both good teachers and good scholars.  He did not mean that faculty should place less emphasis on research.  Indeed he argued that  "original research and publication should remain the basic expectations and be the considered the key criteria" for promotion and tenure.  Boyer's work drew the attention of scholars to the research opportunities that analysis of learning entails.  He did not intend this message for a subgroup of faculty in the schools of education, but rather he hoped to change how academia views scholarship in all the missions of research institutions - learning, engagement, and research. While research, and often fundamental research, is the core mission of research universities, Boyer proposed that teaching undergraduates is as important for advancement as the discoveries made in fundamental mathematics. Research on learning would advance our ability to help our students, and would demonstrate to the citizens who support our research that we are concerned with issues of direct relevance to them.  Thus, Boyer would have us use a significantly broader definition of research in making judgments about the academic worth of our colleagues. He would have us apply the standards of good scholarship across all the missions of the research university, not just its research mission. 

One of the potentially important results of good teaching and effective learning among undergraduates at research universities is that it will ultimately help advance the disciplines themselves.  Academics often forget that among the students they teach are those who will themselves become academics.  Jaroslav Pelikan (1992) in his book, The Idea of the University, makes a point similar to E. O. Wilson's in his discussion of consilience.  Pelikan argues that great scholars are often much influenced in their research by what they learned in other disciplines as undergraduates.  If we are willing to accord prestige and value to high quality undergraduate instruction, we not only may enhance the likelihood that research universities will continue to be supported by society, but we may even be contributing to the development of knowledge that will help take academic disciplines to new levels of understanding. 

The land-grant institutions that this country so wisely created in 1862 are held responsible for providing education to students who might not otherwise have the opportunity or resources to obtain a university education.  The land-grants also carry responsibility for applied research that is designed to help people in their daily lives.  Applied research has not been accorded much prestige in American academia, and it is time for academic administrators to reconsider the significance of applied scholarship. John Maddox (1998), a theoretical physicist and former editor of the journal Nature, wrote in his book What Remains to be Discovered:  "...the science that has dramatically changed and improved the lives of people in the past century is applied science."  I turn now to the efforts Iowa State University has made in encouraging research across the spectrum of its missions, and especially in the areas of applied scholarship in the plant sciences, a discipline that is critical to the economic future of an agricultural state. 

The Plant Sciences Institute at Iowa State University

The State of Iowa and Iowa State University undertook a joint public-private effort to develop a new institute devoted to the disciplines that are critical for the continuing development of agriculture in Iowa.  In 1998, the Legislature set aside the first installment of state funds to allow the university to create an institute that would bring together existing faculty and draw new faculty to the university.  The Plant Sciences Institute (PSI) takes as its mission "enriching agriculture, the environment and our lives through science."  The institute now consists of nine centers covering a broad spectrum of the plant sciences disciplines and includes work on plant transformation and gene expression, designer crops, plant responses to environmental stresses, seed science, plant genomics, bioinformatics and statistics, plant breeding, crop utilization research and a center for designing foods to improve nutrition.  The PSI is actively recruiting faculty and also engages existing faculty from our colleges of Agriculture, Liberal Arts and Sciences, Family and Consumer Sciences, and Engineering.  Its public funding is increasing every year, and it receives wide bipartisan support from the State Legislature.  In addition, the university has received substantial private gifts in support of the institute and allied disciplines.  It is clear that the notion of an interdisciplinary group of faculty devoted to improving the lives of Iowans through science is attractive to both public and private organizations. The PSI and the values it projects underscore a wider effort at Iowa State University to reconsider how scholarship is valued and rewarded within the academic world. 

How Iowa State University Interprets and Values Scholarship

Over the last three years, the university worked to create a strategic plan that would support and encourage the development of scholarship across our missions in a manner that draws from the ideas advocated by Ernst Boyer.  The following diagram represents the interplay of our missions at the university and the scholarship of our faculty.

The three circles represent the missions of the university.  The identifying words differ from the standard missions of teaching, research and service because we wish to emphasize that each of these missions relate fully to faculty and students.  A faculty member engaged in teaching a course to her students is as likely to learn from organizing a course for students, as the students are to learn from her.  Moreover, students are likely to teach faculty not only about the efficacy of their pedagogical methods but sometimes about the discipline being investigated.  In a similar way, discovery and engagement are behaviors that both faculty and students will do.  The intersection of the three missions of the university is where we encourage most of our scholarship to occur. If we are able to change the culture of reward in our institution to favor faculty whose scholarship informs and supports each of the university's missions, we will have made Iowa State University a better place for all who work there. 

The university's strategic plan builds upon a revision of the university's tenure and promotion guidelines adopted by the Faculty Senate and the full faculty in 1999. This remarkable document builds on the ideas first espoused by Boyer and encourages faculty to use the principles and values of good scholarship in every aspect of their work.  Iowa State University characterizes good scholarship in five ways:

• Scholarship develops and communicates new understanding, new knowledge, insights, technologies, materials, uses, and beauty.
• The audiences for scholarship are: peers, undergraduate and graduate students, postdoctoral associates, users, patrons, and the public generally.
• Scholarship can be communicated to others through: teaching materials and methods, classes, curricula, publications, presentations, exhibits, performances, patents, copyrights, and the web.
• Scholarship is validated if it is: original, significant, accurate, replicable, of substantial scope, applicable, of depth and duration of influence, adopted by peers, and has impact or public benefits.
• Scholarship can be documented by: being validated by peers, communicated to peers and broader audiences, recognized, accepted, cited and adopted by others.

Potential Difficulties in the Evaluation of Research

The traditional means of evaluating research has many advantages.  It is relatively easy to count publications, determine the quality of journals used, and count the number of invited presentations at conferences or papers in invited volumes.  It is even easier to assess the amount of research money an investigator brings to her institution. However, these measures will not accurately assess the worth of research contributions or identify individuals who are truly making efforts to use the tools of good scholarship across the missions of their institutions.  We need a broader means of measuring the significance of scholarship, and we need to place it in the context of the institution it serves, not just the discipline it supports. Administrators need to examine their reward structures and ask if these rewards bring about the changes we seek.  Often administrators espouse an ethic of collaboration and breadth, but reward individuals whose scholarship is narrow and of real significance only to a limited academic audience. Finally we need to recognize and reward faculty who attempt to improve the quality of learning, our principle goal.  We are likely to experience increasing difficulties in attracting public and private support for our institutions unless we change the way that we reward academic scholarship.  The adoption of a broader perspective on research is perhaps the best means we have to ensure the continuation of one of the most productive, responsible and truthful institutions in our society -

References

Anderson, Martin (1996).  Imposters in the Temple.  Hoover Institution Press.

Boyer, Ernst (1997).  Scholarship Reconsidered.  Baltimore: Carnegie Foundation.

Drucker, Peter (1997, July 18). Interview in Science.

Glassick, Charles E., Huber, Mary T., and Maeroff, Gene I. (1997).  Scholarship Assessed. 
New York: Jossey-Bass.

Maddox, John (1998).  What Remains to be Discovered.  Free Press.

Pelikan, Jaroslav (1992).  The Idea of the University.  New Haven: Yale University Press.

Wilson, Edward O. (1998).  Consilience.  New York: Vintage Books.