Validation of journal impact metrics of Web of Science and Scopus
Abstract
Citation based metrics are widely used to assess the impact of research published in journals. This paper presents the results of a research study to verify the accuracy of data and calculations of journal impact metrics presented in Web of Science (WoS) and Scopus in the case of three journals of information and library science. Data collected from the websites of journals were compared with that of two citation extended databases. The study manually calculated Journal Impact Factor (JIF) and Impact per Publication (IPP) in accordance with formulas given in the databases. Data were also collected from the Google Scholar for the purpose of comparison. The study found discrepancies in two sets of data and bibliometric values, i.e., systematic values presented in WoS and Scopus and calculated in this study. Commercial databases presented inflated measures based on fabricated or erroneous data. The study is of practical importance to researchers, universities and research financing bodies that consider these bibliometric indicators as a good tool for measuring performance, assessing, and evaluating research quality as well as researchers.
References
Bar-Ilan, J. (2010). Citations to the “Introduction to Informetrics” Indexed by WOS, Scopus and Google Scholar. Scientometrics, 82(3), 495-506.
Bergman, E. M. (2012). Finding Citations to Social Work Literature: The relative benefits of using Web of Science, Scopus, or Google Scholar. The journal of academic librarianship, 38(6), 370-379.
Binswanger, M. (2014). Excellence by Nonsense: The competition for publications in modern science. In S. Bartling, & S. Friesike (Eds.), Opening Science (pp. 49-72). Springer International.
Blaise, C. (2014). Scholars and Scripts, Spoors and Scores. In C. Blaise, & C. R. Sugimoto (Eds.), Beyond Bibliometrics: Harnessing multidimensional indicators of scholarly impact. (pp. 3-21). MIT Press.
Brumback, R. A. (2008). Worshiping False Idols: The impact factor dilemma. Journal of Child Neurology, 23(4), 365-367.
Brumback, R. A. (2009). Impact Factor: Let's be unreasonable! Epidemiology, 20(6), 932-933.
Campbell, P. (2008). Escape from the Impact Factor. Ethics in Science and Environmental Politics, 8(1), 5-6.
Carrió, I. (2008). Of Impact, Metrics and Ethics. European Journal of Nuclear Medicine and Molecular Imaging, 35(6), 1049-1050.
Chew, M., Villanueva, E. V., & Van Der Weyden, M. B. (2007). Life and Times of the Impact Factor: Retrospective analysis of trends for seven medical journals (1994-2005) and their editors' views. Journal of the Royal Society of Medicine, 100(3), 142-150.
Day, R. E. (2014). The Data - It is me! In B. Cronin, & C. R. Sugimoto (Eds.), Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact (pp. 67-84). MIT Press.
De Bellis, N. (2014). History and Evaluation of (Biblio)Metrics. In B. Cronin, & C. R. Sugimoto (Eds.), Beyond Bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 23-44). MIT Press.
Dong, P., Loh, M., & Mondry, A. (2005). The "impact factor" Revisited. Biomedical Digital Libraries, 2(7), 1-8.
Elsevier. (2015). About Impact per Publication (IPP). Retrieved 08 23, 2015, from http://www.journalmetrics.com/ipp.php
Elsevier. (2015). Compare Journals. Retrieved 08 23, 2015, from http://www-scopus-com.lib.costello.pub.hb.se/source/eval.url
Elsevier. (2015). Journal Rankings. Retrieved 08 23, 2015, from http://www.scimagojr.com/journalrank.php
Falagas, M. E., & Alexiou, V. G. (2008). The Top-ten in Journal Impact Factor Manipulation. Archivum Immunologiae et Therapiae Experimentalis, 56(4), 223-226.
Frandsen, T. F. (2008). On the Ratio of Citable Versus Non-citable Items in Economics Journals. Scientometrics, 74(3), 439-451.
Garfield, E. (1964). Science Citation Index - A new dimension in indexing. Science, 144(3619), 649-954.
Glänzel, W., & Moed, H. F. (2002). Journal Impact Measures in Bibliometric Research. Scientometrics, 53(2), 171-193.
Golubic, R., Rudes, M., Kovacic, N., Marusic, M., & Marusic, A. (2008). Calculating Impact Factor: How bibliometrical classification of journal items affects the impact factor of large and small journals. Science and Engineering Ethics, 14(1), 41-49.
Harzing, A. W. (2007). Publish or Perish. Retrieved 08 23, 2015, from http://www.harzing.com/pop.htm
Jasco, P. (2001). A Deficiency in the Alogritm for Calculating the Impact Factor of Scholarly Journals: The journal impact factor. Cortex, 37(4), 590-594.
Kumar, M. (2010). The Import of the Impact Factor: Fallacies of citation-dependent scientometry. Bulletin of the Royal College of Surgeons of England, 92(1), 26-30.
Law, R. (2012). The Usefulness of Impact Factors to Tourism Journals. Annals of Tourism Research, 39(3), 1722-1724.
Law, R., & Li, G. (2015). Accuracy of Impact Factors in Tourism Journals. Annals of Tourism Research, 50, 19-21.
Leydesdorff, L., & Opthof, T. (2010). Scopus's Source Normalized Impact Per Paper (SNIP) Versus a Journal Impact Factor Based on Fractional Counting of Citations. Journal of the American Society for Information Science and Technology, 61(11), 2365-2369.
Martin, B. R. (2016). Editors' JIF-boosting Stratagems-Which are Appropriate and Which Not? Research Policy, 45(1), 1-7.
McVeigh, M. E., & Mann, S. J. (2009). The Journal Impact Factor Denominator: Defining citable (counted) items. JAMA, 302(10), 1107-1109.
Merton, R. K. (1957). Priorities in Scientific Discovery: A chapter in the sociology of science. American Sociological Review, 22, 635-659.
Moed, H. F. (2005). Citation Analysis in research evaluation (Vol. 9). Springer Science and Business Media.
Moed, H. F., Van Leeuwen, T. N., & Reedijk, J. (1999). Towards Appropriate Indicators of Journal Impact. Scientometrics, 46(3), 575-589.
Monastersky, R. (2005). The 'Number That’s Devouring Science. Chronicle of Higher Education, 52(8), 14.
Nelhans, G. (2014). Qualitative Scientometrics? Proceedings of the 35th IATUL Conference. The International Association of Scientific and Technological University Libraries (IATUL).
Rossner, M., Van Epps, H., & Hill, E. (2007). Show me the Data. The Journal of Cell Biology, 179(6), 1091-1092.
Rousseau, R. (2012). Updating the Journal Impact Factor or Total Overhaul? Scientometrics, 92(2), 413-417.
Salton, G. (1963). Associative Document Retrieval Technologies Using Bibliographic Information. Journal of the ACM (JACM), 10(4), 440-457.
Seglen, P. O. (1997). Why the Impact Factor of Journals Should not be Used for Evaluating Research. Bmj, 314(7079), 498-502.
Sevinc, A. (2004). Manipulating Impact Factor: An unethical issue or an editors's choice. Swiss Medical Weekly, 134(27-28), 410.
Simons, K. (2008). The Misused Impact Factor. Science, 322(5899), 165-165.
Smart, P. (2015). Is the Impact Factor the Only Game in Town? The Annals of the Royal College of Surgeons of England, 97(6), 405-408.
The PLoS Medicine Editors. (2006). The Impact Factor Game. PLoS Med, 291.
Thomson Reuters. (1994, 06 20). The Thomson Reuters Impact Factor. Retrieved 08 23, 2015, from http://wokinfo.com/essays/impact-factor/
Thomson Reuters. (2012). Journal Citation Reports. Retrieved 08 23, 2015, from http://admin-apps.webofknowledge.com/JCR/help/h_sourcedata.htm
Van Leeuwen, T. N., Moed, H. F., & Reedijk, J. (1999). Critical Comments on Institute for Scientific Information Impact Factors: A sample of inorganic molecular chemistry journals. Journal of Information Science, 25(6), 489-498.
Van Leeuwen, T., Costas, R., Calero-Medina, C., & Visser, M. (2013). The Role of Editorial Material in Bibliometric Research Performance assessment. Scientometrics, 95(2), 817-828.
Whitehouse, G. H. (2001). Citation Rates and Impact Factors: Should the matter? The British Journal of Radiology, 74(877), 1-3.
Wolthoff, A., Lee, Y., & Ghohestani, R. F. (2011). Comprehensive Citation Factor: A novel method in ranking medical journals. European Journal of Dermatology, 21(4), 495-500.
Wouters, P. (2014). The Citation: From culture to infrastructure. In B. Cronin, & C. R. Sugimoto (Eds.), Beyond Bibliometrics: Harnessing multidimensional indicators of scholarly impact. (pp. 47-66). MIT Press.
Wu, X. F., Fu, Q., & Rousseau, R. (2008). On Indexing in the Web of Science and Predicting Journal Impact Factor. Journal of Zhejiang University SCIENCE B, 9(7), 582-590.
Zupanc, G. K. (2014). Impact Beyond the Impact Factor. Journal of Comparative Physiology A, 200(2), 113-116.
Full Text: PDF
Refbacks
- There are currently no refbacks.
