Preprints have been around for a few decades, but posting preprints to a repository has only become the new normal for scholars in recent years. Preprints allow researchers to stake a claim to their ideas and results by establishing a clear and timestamped record of their work, even if the peer review process drags on for months. Preprints also facilitate rapid communication among scholars, which can be critical during times of crisis; the COVID-19 pandemic, for instance, led to a surge in preprint publications across several fields of study.

The rise of preprints leads to many new research questions, including that asked by Pagliaro (2021) in a recently published paper in the journal Publications: do manuscripts change substantially between preprint posting and the final, peer-reviewed version of the article? Following in the footsteps of scholars who studied this question in Physics and Biological Sciences, Pagliaro studied a small sample of Chemistry articles, finding “the differences between preprints and the corresponding articles published after peer review are small.”

The implications of only “small” changes to preprints as they wind their way through the editorial and peer review process raises questions about the institution of peer review (also discussed by Pagliaro). There are also important implications for bibliometric and scientometric data aggregators (including Academic Analytics), however, who typically include “peer-reviewed” as a criterion for including a scholarly research artifact in their databases (effectively precluding preprints from having a role in the research strategy and planning exercises that are carried out based on these databases).

On one hand, including preprints in bibliometric databases necessitates substantial additional investment in disambiguation and data merging. The preprint (which often has its own DOI) eventually needs to be linked to the final published version of the paper so the same research output is not recorded as two artifacts of research (or at least so the end user can identify that the same manuscript resulted in two artifacts) rather than “double-counting” the manuscript. There are also cases where preprints never result in a peer-reviewed journal article; in these cases, counting preprints among the number of publications for an institution/department/scholar may incentivize the production of preprints for which the author has no intention of ultimately putting the ideas or results through peer-review.

On the other hand, excluding preprints from bibliometric databases signals that preprints are not valuable enough to be considered among the other artifacts produced by scholars (“value” here meaning the purported value conferred through the peer review process). Clearly this is not a fair characterization of preprints, which have tremendous value. With the efficacy of peer review increasingly called into question, it may be time for bibliometric database providers to mobilize resources to solve the problems of “double-counting” and what to do with preprints that never make their way into traditional journals.

We are eager to hear your thoughts on preprints and whether (and how) bibliometric databases can include them to more fully represent the research outputs of scholars.


Reference Cited:

Pagliaro M. Preprints in Chemistry: An Exploratory Analysis of Differences with Journal Articles. Publications. 2021; 9(1):5.