Data Principles

Academic Analytics data in
the context of promotion and tenure

American research universities increasingly rely on quantitative information to enhance decisions and to implement those decisions.

Since 2005, Academic Analytics has been a key provider of the core data and related information enabling decision-makers to accurately gauge research activity. Academic Analytics believes that objective, accurate, and comprehensive research metrics are a valuable resource for institutional self-assessment. But, importantly, paired with other data, disciplinary expertise, and local knowledge, metrics describing research outcomes can foster productive discussion and promote mutual understanding of a wide range of campus stakeholders.

We have also observed the explicit use of quantitative descriptors of research outcomes in faculty promotion and tenure decisions at colleges and universities in the United States. Relying on research metrics to facilitate faculty promotion marks a departure from how databases traditionally have been used. The use has ranged from aggregate (institutional and within disciplines) self-assessment to the evaluation of an individual's scholarship.

Research metrics might be seen as a more objective record of scholarship than, for example, letters written by peers or student evaluations. But an over-reliance on bibliometric and other quantitative information without considering appropriate complementary knowledge can produce an incomplete picture of individual scholarly achievement.

The following principles can usefully guide administrators who seek to incorporate metrics into decision-making:

1. The use of verifiable quantitative data in promotion and tenure is only one important indicator of a scholar's productivity and must be supplemented with other kinds of information and with more nuanced indicators of quality.
2. Faculty members should have access to the research metrics used in promotion decisions about them, and they should have the opportunity to correct and supplement that record prior to any substantive review.
3. No database of research output is a comprehensive record of scholarly achievement. Journal articles, books, chapters, grant information, conference proceedings, and patents are traditional modes of research dissemination in many disciplines and often become bibliometric artifacts easily included in
   databases. But other modes of dissemination (e.g., exhibitions, storytelling, performances, choreography, musical composition, zine authorship, blog authorship, dataset production, software programs, opensource materials, public domain contributions, and media mentions) are valued in many disciplines. These venues are not captured in most databases, potentially disadvantaging scholars whose work is most often shared in those forms. Such gaps should be recognized and accounted for in evaluation processes.
4. Because research cultures vary widely across the disciplines, and even sub-disciplines (e.g., relative prominence of articles or books or conference proceedings; co-authorship vs. single authored work), administrative reviewers of faculty scholarship need to develop an informed understanding of realistic expectations for the disciplines under review.
5. Because the quantity of one's research output does not equate to the quality and impact of a scholar's research output, other means for evaluating significance, quality, and importance to the field of study should complement quantitative measures.
6. Research metrics cannot account for systemic biases or personal and social factors that may influence one's research. Gender, racial, ethnic, mental health, disability, and age biases in the American academy are increasingly well-documented and may impact the venue, quantity (but not necessarily the quality) of output.

To speak with an Academic Analytics representative about this, please contact us at:
info@academicanalytics.com