Quantitative analysis of automatic performance evaluation systems based on the h-index

  • Since the h-index has been invented, it is the most frequently discussed bibliometric value and one of the most commonly used metrics to quantify a researcher’s scientific output. The more it is increasingly gaining popularity to use the metric as an indication of the quality of a job applicant or an employee the more important it is to assure its correctitude. Many platforms offer the h-index of a scientist as a service, sometimes without the explicit knowledge of the respective person. In this article we show that looking up the h-index for a researcher on the five most commonly used platforms, namely AMiner, Google Scholar, ResearchGate, Scopus and Web of Science, results in a variance that is in many cases as large as the average value. This is due to the varying definitions of what a scientific article is, the underlying data basis, and different qualities of the entity recognition problem. To perform our study, we crawled the h-index of the worlds top researchers according to two different rankings, all the Nobel Prize laureates except Literature and Peace, and the teaching staff of the computer science department of the TU Kaiserslautern Germany with whom we additionally computed their h-index manually. Thus we showed that the individual h-indices differ to an alarming extent between the platforms. We observed that researchers with an extraordinary high h-index and researchers with an index appropriate to the scientific career path and the respective scientific field are affected alike by these problems.

Download full text files

Export metadata

Additional Services

Search Google Scholar
Metadaten
Author:Marc P. HauerORCiD, Xavier C. R. HofmannORCiD, Tobias D. KrafftORCiD, Katharina Anna ZweigORCiD
URN:urn:nbn:de:hbz:386-kluedo-77928
DOI:https://doi.org/10.1007/s11192-020-03407-7
ISSN:1588-2861
Parent Title (English):Scientometrics
Publisher:Springer Nature - Springer
Document Type:Article
Language of publication:English
Date of Publication (online):2024/03/11
Year of first Publication:2020
Publishing Institution:Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau
Date of the Publication (Server):2024/03/11
Issue:123
Page Number:17
First Page:735
Last Page:751
Source:https://link.springer.com/article/10.1007/s11192-020-03407-7
Faculties / Organisational entities:Kaiserslautern - Fachbereich Informatik
DDC-Cassification:0 Allgemeines, Informatik, Informationswissenschaft / 004 Informatik
Collections:Open-Access-Publikationsfonds
Licence (German):Zweitveröffentlichung