Social Media2018-09-07T17:44:00-04:00

Explore your research potential

Data to analyze, discover, and connect

Social Media

2602, 2021

Replay: Who’s Publishing Open Access Articles?

Friday, February 26, 2021|

Last week, AARC researchers Dr. Molly J. Wilson and Dr. Anthony J. Olejniczak discussed their recent paper Who’s writing Open Access (OA) articles? Characteristics of OA authors at Ph.D. granting institutions in the USA with a live audience via webinar. The discussion was excellent, and the playback is available here: link to request playback.

The next AARC webinar will be on April 13, 2021 at 1:30pm EST, about the publication outputs (journal articles, books, chapters, and conference proceedings) of senior scholars relative to other age cohorts. A preprint is available while the paper is in the peer review process, we hope you’ll join us!

1902, 2021

Who’s being honored in academia?

Friday, February 19, 2021|

Academic Analytics matches a huge number of honorific awards (10,000+) to individual scholars in the American academy. AARC researchers recently began digging through this data trove, and some summary statistics by discipline offer a glimpse into the deeper patterns we’re investigating. We started with academic department faculty lists for the 2019/2020 academic year. We then matched national or international awards (no state or local awards) bestowed upon those academics between 2017 and 2019, and created a table showing the number of awards won per faculty member in each discipline (the table can be sorted, and it’s paginated – only showing 10 rows at a time):

ComparisonGroupNameawdsPerFac
Accounting0.08
Aerospace Engineering0.193
Agricultural Economics0.151
Agricultural/Biological Engineering and Bioengineering0.321
Agriculture, various0.134
Agronomy and Crop Science0.149
American Studies0.166
Anatomy0.064
Ancient Studies0.162
Animal Sciences0.196
Anthropology0.115
Applied Economics0.113
Applied Mathematics0.13
Applied Physics0.276
Architecture0.115
Architecture, Design, Planning, various0.105
Area and Ethnic Studies, various0.163
Art History and Criticism0.138
Asian Languages0.102
Asian Studies0.121
Astronomy and Astrophysics0.168
Atmospheric Sciences and Meteorology0.233
Biochemistry0.115
Bioinformatics and Computational Biology0.13
Biological Sciences, various0.056
Biology/Biological Sciences, General0.09
Biomedical Engineering0.264
Biomedical Sciences, General0.077
Biomedical Sciences, various0.084
Biophysics0.097
Biostatistics0.125
Botany/Plant Biology0.161
Business Administration0.144
Business, various0.111
Cell Biology0.118
Chemical Engineering0.272
Chemical Sciences, various0.193
Chemistry0.18
Civil Engineering0.19
Classics and Classical Languages0.101
Clinical Psychology0.233
Cognitive Science0.268
Communication and Communication Studies0.216
Communication Disorders and Sciences0.069
Comparative Literature0.133
Composition, Rhetoric and Writing0.184
Computational Sciences0.148
Computer and Information Sciences, various0.18
Computer Engineering0.183
Computer Science0.212
Consumer and Human Sciences, various0.311
Counseling Psychology0.143
Counselor Education0.116
Criminal Justice and Criminology0.132
Curriculum and Instruction0.096
Developmental Biology0.134
Ecology0.218
Economics, General0.101
Education, General0.153
Educational Evaluation and Research0.171
Educational Leadership and Administration0.115
Educational Psychology0.136
Electrical Engineering0.205
Engineering Mechanics0.173
Engineering, General0.18
Engineering, various0.153
English Language and Literature0.102
Entomology0.125
Environmental Engineering0.203
Environmental Health Sciences0.082
Environmental Sciences0.147
Epidemiology0.088
Evolutionary Biology0.245
Family and Human Sciences, various0.147
Finance0.057
Fisheries Science0.1
Food Science0.184
Forest Resources/Forestry0.083
Foundations of Education0.156
French Language and Literature0.042
Gender Studies0.143
Genetics0.133
Geography0.088
Geological and Mining Engineering0.176
Geology/Earth Science, General0.19
Geophysics0.184
Germanic Languages and Literatures0.056
Health Professions, various0.096
Health Promotion, Kinesiology, Exercise Science and Rehab0.122
Health, Physical Education, Recreation0.083
Higher Education/Higher Education Administration0.159
History0.173
Horticulture0.153
Human and Medical Genetics0.132
Human Development and Family Studies, General0.181
Humanities/Humanistic Studies, General0.086
Immunology0.169
Industrial Engineering0.23
Information Science/Studies0.145
Information Technology/Information Systems0.088
International Affairs and Development0.151
Italian Language and Literature0.069
Languages, various0.051
Linguistics0.087
Management0.17
Management Information Systems0.096
Marine Sciences0.143
Marketing0.15
Mass Communications/Media Studies0.121
Materials Engineering0.24
Materials Science and Engineering0.29
Mathematics0.124
Mathematics Education0.148
Mechanical Engineering0.189
Medical Sciences, various0.093
Microbiology0.15
Molecular Biology0.126
Molecular Genetics0.114
Molecular Pharmacology0.117
Music specialties0.045
Music, General0.047
Natural Resources0.141
Near and Middle Eastern Languages and Cultures0.099
Neurobiology/Neuroscience0.13
Nuclear Engineering0.217
Nursing0.137
Nutrition Sciences0.125
Oceanography, Physical Sciences0.155
Oncology and Cancer Biology0.057
Operations Research0.169
Oral Biology and Craniofacial Science0.087
Pathology0.064
Performing and Visual Arts, various0.053
Pharmaceutical Sciences0.072
Pharmacology0.08
Pharmacy0.071
Philosophy0.072
Physics, General0.158
Physiology, General0.091
Plant Pathology0.123
Plant Sciences0.129
Political Science0.192
Psychology, General0.197
Psychology, various0.205
Public Administration0.182
Public Health0.081
Public Policy0.197
Religion/Religious Studies0.093
School Psychology0.137
Science Education0.164
Slavic Languages and Literatures0.095
Social Sciences, various0.104
Social Work/Social Welfare0.117
Sociology0.18
Soil Science0.116
Spanish Language and Literature0.044
Special Education0.083
Speech and Hearing Sciences0.096
Statistics0.115
Structural Biology0.1
Systems Engineering0.25
Teacher Education Specific Levels0.057
Teacher Education Specific Subject Areas0.093
Theatre Literature, History and Criticism0.028
Theology/Theological Studies0.032
Toxicology0.081
Urban and Regional Planning0.118
Veterinary Medical Sciences0.135
Wildlife Science0.128
Zoology0.163

Sorted by the number of awards per faculty member (ascending), the fewest awards per person tend to be in humanities fields (theater, languages, etc.). At the other end of the list (sorted descending), the greatest number of awards per person tends to be in engineering disciplines, with some exceptions: Agricultural/Biological Engineering and Bioengineering, Consumer and Human Sciences, Materials Science and Engineering, Applied Physics, and Chemical Engineering.

It’s fascinating to see the distribution of honorific awards, but it calls into question how representative the data are – in other words, its possible that Academic Analytics happens to capture more awards in engineering than in humanities due to a previously unrecognized collection bias. It’s also possible that there are simply more awards available to engineers – maybe there are more scholarly societies who bestow awards in engineering fields? In any case, it’s clear that honorific awards, for which there exists no equivalent of a standard metadata description or widely-accepted unique ID number (such as DOI) should be interpreted in the context of both availability and potential collection biases.

We advocate the use of honorific awards as a post hoc indicator of research excellence (and sometimes precursors to research excellence, in the case of honors bestowed upon early career researchers), but we also caution that they are not as uniformly distributed nor standardized as bibliometric data or data about research grants. Our research is leading us towards creative solutions to these issues, and we welcome your thoughts.

 

Anthony J. Olejniczak, Ph.D.

Director, Academic Analytics Research Center (AARC)

1202, 2021

The preprint conundrum for bibliometric databases

Friday, February 12, 2021|

Preprints have been around for a few decades, but posting preprints to a repository has only become the new normal for scholars in recent years. Preprints allow researchers to stake a claim to their ideas and results by establishing a clear and timestamped record of their work, even if the peer review process drags on for months. Preprints also facilitate rapid communication among scholars, which can be critical during times of crisis; the COVID-19 pandemic, for instance, led to a surge in preprint publications across several fields of study.

The rise of preprints leads to many new research questions, including that asked by Pagliaro (2021) in a recently published paper in the journal Publications: do manuscripts change substantially between preprint posting and the final, peer-reviewed version of the article? Following in the footsteps of scholars who studied this question in Physics and Biological Sciences, Pagliaro studied a small sample of Chemistry articles, finding “the differences between preprints and the corresponding articles published after peer review are small.”

The implications of only “small” changes to preprints as they wind their way through the editorial and peer review process raises questions about the institution of peer review (also discussed by Pagliaro). There are also important implications for bibliometric and scientometric data aggregators (including Academic Analytics), however, who typically include “peer-reviewed” as a criterion for including a scholarly research artifact in their databases (effectively precluding preprints from having a role in the research strategy and planning exercises that are carried out based on these databases).

On one hand, including preprints in bibliometric databases necessitates substantial additional investment in disambiguation and data merging. The preprint (which often has its own DOI) eventually needs to be linked to the final published version of the paper so the same research output is not recorded as two artifacts of research (or at least so the end user can identify that the same manuscript resulted in two artifacts) rather than “double-counting” the manuscript. There are also cases where preprints never result in a peer-reviewed journal article; in these cases, counting preprints among the number of publications for an institution/department/scholar may incentivize the production of preprints for which the author has no intention of ultimately putting the ideas or results through peer-review.

On the other hand, excluding preprints from bibliometric databases signals that preprints are not valuable enough to be considered among the other artifacts produced by scholars (“value” here meaning the purported value conferred through the peer review process). Clearly this is not a fair characterization of preprints, which have tremendous value. With the efficacy of peer review increasingly called into question, it may be time for bibliometric database providers to mobilize resources to solve the problems of “double-counting” and what to do with preprints that never make their way into traditional journals.

We are eager to hear your thoughts on preprints and whether (and how) bibliometric databases can include them to more fully represent the research outputs of scholars.

 

Reference Cited:

Pagliaro M. Preprints in Chemistry: An Exploratory Analysis of Differences with Journal Articles. Publications. 2021; 9(1):5. https://doi.org/10.3390/publications9010005

502, 2021

Should interdisciplinary comparisons of journal article publications use the mean or median?

Friday, February 5, 2021|

AARC scholars work with many datasets describing the publication outputs of research faculty. These datasets are almost always zero-inflated, or at least are skewed toward the lower end of the distribution. This phenomenon is so common we’ve even changed how we perform regression analyses to account for these skewed distributions (e.g., we ran hurdle regressions in our paper on Open Access publication trends). The histogram below shows the average number of journal articles published by scholars in departments classified as “Physics” over the past 10 years:

The data are clearly skewed towards the left side of the plot, between 0-100 articles per person over 10 years. The mean number of articles published over 10 years is 91.7 (green vertical line) and the median number of articles is 32 (red vertical line); however, a few physicists have as many as 1,100 articles over that 10-year span. These scholars are generally associated with massive multi-institution and multi-year projects such as CERN, so we looked at several other disciplines outside the natural sciences to see whether the pattern persists – the image below is for English Language and Literature journal articles over the same 10-year period:

Indeed, the distribution looks similar to that seen for physicists. In English, the mean number of journal articles per person over 10 years (green vertical line) is 3.8, while the median is 2.0 articles (red vertical line). A small number of English faculty members have published upwards of 50 journal articles over the 10-year period.

The skewness of these publication metrics complicates interpretation of discipline norms, with meaningful consequences for the faculty, administrators, and other committee members charged with comparing discipline publishing patterns for strategic planning. In English, the median number of journal articles published is about one half of the mean value, and in Physics the median is about one third of the mean value. Although means are commonly used in bibliometric comparisons, choosing the mean as the unit of comparison biases the data towards the few examples at the extreme right end of the publishing distribution.

For these reasons, we believe the median is often the more appropriate measure for intra- and interdisciplinary analysis and bibliometric comparisons. University-wide planning and evaluation are better served by focusing on discipline (or peer group) norms such as the median, rather than numbers incorporating the most extreme cases and perhaps setting unrealistic publishing expectations.

(the data above are from database version AAD2019-1470)

2901, 2021

What are the biggest fields in Ph.D. education?

Friday, January 29, 2021|

Scholars at AARC are working hard on a project aimed at quantifying the dizzying growth of Ph.D. education in the US over the past 25 years (more on that project in a few months). As we looked at the number of Ph.D. graduates and the number of programs over time, we realized that growth in those areas probably means growth in another metric- the number of faculty members mentoring those students and teaching in those programs. The table below shows the number of Ph.D. program faculty by broad field as of 2019 (Academic Analytics database version AAD2019-1470):

Field# Faculty% Faculty 
Biological and Biomedical Sciences 43,196 21.3%
Physical and Mathematical Sciences 34,041 16.8%
Engineering 26,836 13.2%
Social and Behavioral Sciences 26,338 13.0%
Humanities 22,102 10.9%
Health Professions Sciences 11,929 5.9%
Family, Consumer and Human Sciences 10,207 5.0%
Business 9,119 4.5%
Education 8,416 4.1%
Natural Resources and Conservation 5,596 2.8%
Agricultural Sciences 5,114 2.5%

Biological and biomedical sciences Ph.D. programs involve more faculty members than any other broad field of study, by about 5%. Breaking this down into individual disciplines, here are the 10 most populous (in terms of faculty members) Ph.D. program disciplines in the US:

Discipline# Faculty% FacultyRank (Population)
Neurobiology/Neuroscience7,7092.74%1
Molecular Biology7,6272.71%2
Cell Biology7,0962.52%3
Computer Science5,9232.11%4
Physics, General5,6722.02%5
Mathematics5,6422.01%6
Electrical Engineering5,5481.97%7
Biochemistry5,3971.92%8
Biology/Biological Sciences, General5,3521.90%9
Chemistry5,2481.87%10

Several disciplines from outside the life sciences make the “top 10” most populous list. What’s unclear is how recent the above table came to look this way – Neuroscience is the largest (by faculty population) Ph.D. discipline in the US, but was that the case 5 years ago? 10 years ago? It will be informative to uncover the history of growth over time in each field, and to attempt to trace that growth to the underlying causes.

Go to Top