ABSTRACT
The present study analyzed the main characteristics of retracted articles addressing coronavirus disease 2019 (COVID-19) published in medical journals in China and abroad. The results are anticipated to provide a reference for the publication and dissemination of articles addressing public health emergencies in the future. Information regarding the retraction of COVID-19 articles was obtained from the Retraction Watch website for the period between January 1, 2020, and October 31, 2023. The main characteristics of the withdrawn COVID-19 articles were analyzed according to the distribution of journals, authors, publication period, and the reasons for retraction. Among the 235 COVID-19 articles withdrawn, the main article types were original research and reviews, the main publishers were journals without impact factors and/or preprint platforms, and the authors' countries were mostly China and the United States. The publication period was mainly in 2020 and 2021, and the number of publications is gradually decreasing. The retraction delay was 0–741 days, with a mean delay of 5.9 months. In addition to mistakes from journals/publishers, most of the top reasons for retraction included academic misconduct, especially regarding data and results. The main characteristics of retracted COVID-19 articles included publication on preprint platforms and journals with an impact factor < 5. Furthermore, more retractions were due to journal/publisher errors and academic misconduct. During epidemic situations, such as COVID-19, medical journals should not only improve publishing efficiency, but also strive to strictly control academic quality.
Key words: medical journal, public health emergency, retraction
INTRODUCTION
Public health emergencies refer to incidents or situations that develop suddenly, causing or potentially causing serious harm to public health, including major infectious disease outbreaks, mass unexplained disease(s), significant food and occupational poisoning, and other events.[1,2] On January 30, 2020, the coronavirus disease 2019 (COVID-19) pandemic was designated a "Public Health Emergency of International Concern" by the World Health Organization.[3] Shortly after the COVID-19 outbreak, medical journals, both domestically and internationally, quickly responded, establishing new and efficient platforms for academic exchange, thus providing strong academic support for epidemic prevention and control.[4] Databases, such as Chinese National Knowledge Infrastructure (CNKI) and WanFang in China, set up first-release platforms for COVID-19 research results, and top international medical journals, such as the New England Journal of Medicine, The Lancet, and the Journal of the American Medical Association, established platforms for the rapid publication of studies addressing COVID-19.[5,6] The world's five major preprint platforms (MedRxiv, BioRxiv, SSRN, ArXiv, ChemRxiv) set up dedicated publishing platforms and search channels for COVID-19 articles.[7] As of October 31, 2023, a total of 395,078 COVID-19 articles have been indexed on PubMed. The rapid and large-scale publication of COVID-19 articles has, on the one hand, promoted the quick dissemination of related research results. On the other hand, in the pursuit of publication speed, reduced review time, or the number of reviews, studies with issues may have been rushed to publication. The Retraction Watch website set up a dedicated column for retracted COVID-19 articles at the end of April 2020, which is updated regularly.[8] The number of retractions rose from the initial 45 to 92 by the end of March 2021, and continues to increase, indicating that the retraction of COVID-19 articles is a matter of concern. We did not find any similar research reports on the revocation feature analysis of COVID-19 papers after reviewing the literature. Based on relevant data obtained from the Retraction Watch database, the present study collected and analyzed data from COVID-19 articles that were retracted. We examined the characteristics of these articles, and aimed to provide a reference for regulating the retraction of articles addressing topics pertaining to public health emergencies.
RESEARCH METHODS AND DATA SOURCES
The present study used the Retraction Watch database (http://retractiondatabase.org/) to search for and collect information related to the retraction of COVID-19 articles. The keywords used in the search included "COVID-19" or "Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2)" or "Coronavirus Disease 2019" or "2019-nCoV". The data collected were from the period between January 1, 2020 and October 31, 2023. A total of 290 relevant articles were retrieved in the search and, after detailed analysis, 55 were excluded, including 16 duplicates, 18 conference articles, 4 corrections, 15 expressions of concern (but not retracted), and 2 reinstatements, resulting in 235 retracted COVID-19 articles. The data were copied and imported into a spreadsheet (Excel 2013, Microsoft Corporation, Redmond, Washington, USA) for processing and analysis.
Information housed in the Retraction Watch database mainly includes title, professional classification, article type, journal name, publisher name, author name(s), author institution, author nationality, reason for retraction, publication period, and retraction time. The authors supplemented the information, such as journal impact factor, by querying the Web of Science database.
RESULTS
Journal and author distribution of the retracted articles
The distribution of retracted COVID-19 articles according to article type was as follows: original research (n = 164 [69.8%]), review (n = 27 [11.4%]), case report (n = 13 [5.5%]), clinical study (n = 12 [5.1%]), letter (n = 9 [3.8%]), commentary/editorial (n = 6 [2.6%]), and meta-analysis (n = 4 [1.7%]).
Regarding journals that published retracted COVID-19 articles, the 235 articles were distributed among 152 journals and 4 preprint platforms. Twenty-six (11.1%) articles were published on preprint platforms, 39 in journals without an impact factor (16.6%), 62 in journals with an impact factor < 3 (26.3%), 67 in journals with an impact factor between 3 and 5 (28.5%), and 41 in journals with an impact factor > 5 (17.4%). The journal with the highest impact factor was The Lancet (168.9). Journals with a high number of retracted COVID-19 articles included Cureus (n = 10), Health Science Reports (n = 9), Soft Computing (n = 7), Journal of Healthcare Engineering (n = 5), Brain and Behavior (n = 4), and The Lancet (n = 3). Preprint platforms with retracted COVID-19 articles included medRxiv (n = 12), SSRN (n = 8), BioRxiv (n = 5), and preprints with The Lancet (n = 1). The publisher with the most retracted COVID-19 articles was Elsevier, with a total of 61.
The distribution of countries of retracted COVID-19 articles was follows: China (n = 51), United States (n = 37), India (n = 20), Iran (n = 12), France (n = 8), Spain (n = 7), Italy (n = 6), Pakistan (n = 6), Greece, Saudi Arabia, Egypt (n = 5 each), Germany, Canada, Ghana, Sudan, Spain, Turkey (n = 4 each), Austria, Brazil, Russia (n = 3 each), Japan, Singapore, United Kingdom, South Korea (n = 2 each), and 32 other countries, including Belgium (n = 1 each). There were 62 studies that involved collaboration between ≥ 2 countries. The countries with the highest number of COVID-19 articles published during the same period were the United States (n = 78,502) and China (n = 41,312).
Time distribution of retracted articles
The publication time and retraction time distributions of the 235 retracted COVID-19 articles are shown in Figure 1. The earliest retracted COVID-19 articles were published in January 2020, with a total of 2. The main publication time of retracted COVID-19 articles was in 2020 and 2021, with > 80 articles per year, and the number of published articles decreased year by year. The earliest retraction of a COVID-19 article was by the preprint platform BioRxiv, which retracted a research article published 3 days earlier on January 28, 2020. The earliest journal to issue a retraction was The Lancet Global Health, which retracted a letter published on February 24 of the same year on February 26, 2020.
Figure 1. Distribution of publication and retraction time of retracted COVID-19 articles. COVID-19, coronavirus disease 2019.
Retraction latency refers to the time interval between the retraction of an article and its initial publication, which, to a certain extent, can reflect the "purification efficiency" of the scientific community. The retraction latency of the 235 retracted COVID-19 articles is illustrated in Figure 2. The shortest retraction latency was 0, with a total of 25 articles. The longest retraction latency was 741 days, for an article published in Health Psychology Research on June 4, 2021, and retracted on June 15, 2023, due to duplicate publication through errors by the journal/publisher. The most common retraction latencies were > 5 months (86 articles) and < 1 month (60 articles), with an average retraction latency of 5.9 months. There were also 23 articles for which the retraction or retraction statement did not clearly state the retraction time, thus making it exceedingly difficult to determine the retraction time and latency.
Figure 2. Retraction latency of retracted COVID-19 articles. COVID-19, coronavirus disease 2019.
Analysis of the reasons for retractions
Of the 235 COVID-19 articles retracted, excluding the 29 that did not specify the reason for retraction, there were a total of 360 recorded reasons for retraction. The top 10 reasons for retraction and the number of retractions they caused are summarized in Table 1. The most common reasons for retraction, aside from errors by the journal/publisher, were typically academic misconduct, including issues with data and results, plagiarism, and duplicate publication. Additionally, not obtaining permission from the institutional review board (IRB) or institutional animal care and use committees (IACUC) was also a common reason. Of the 34 articles retracted due to errors by the journal/publisher (including 22 "duplicate publication through errors by journal/publisher" and 12 "errors by journal/publisher"), 21 were from Elsevier. Of the 235 retracted COVID-19 articles, 60 were published by Elsevier.
| Rank | Reason | n | % |
| 1 | Unreliable results | 50 | 21.3 |
| 2 | Concerns/issues about data | 43 | 18.3 |
| 3 | Investigation by journal/publisher | 34 | 14.4 |
| 4 | Fake peer review | 31 | 13.1 |
| 5 | Duplicate publication through errors by journal/publisher | 22 | 9.4 |
| 6 | Concerns/issues about result | 19 | 8.1 |
| 7 | Errors in analyses | 16 | 6.8 |
| 8 | Errors in results and/or conclusions | 14 | 5.9 |
| 9 | Errors by journal/publisher | 12 | 5.1 |
| 10 | Duplication of article | 11 | 4.6 |
A previous study[9] categorized the 360 reasons for retraction by the responsible parties (including authors, journal/publisher, related to journal/publisher, reviewers, authors' institution, related to authors' institution, third parties, and related to third parties). The proportion of articles retracted due to author-related reasons was as high as 69.4%, followed by journal/publisher and journal/publisher-related reasons, which accounted for 18.9% of retractions. The retraction records of each responsible party are reported in Figure 3. According to that study,[9] the objects of responsibility mainly included data, images, results, text, authorship, reviewer comments, and references. The distribution of the 360 reasons for the retraction of COVID-19 articles was as follows: results, 23.3%; text, 18.0%; data, 16.4%; reviewer comments, 8.6%; ethical, 7.8%; experimental issues, 6.9%, none 14.7%, authorship 3.1%, copyright and reference 0.6%, respectively. The retraction records of each object of responsibility are reported in Figure 4.
Figure 3. Responsible party of retracted COVID-19 articles. COVID-19, coronavirus disease 2019.
Figure 4. Object of responsibility of retracted COVID-19 articles. COVID-19, coronavirus disease 2019.
DISCUSSION AND IMPLICATIONS
Results of the present analysis revealed that, although there were 235 retracted articles, this number represents only a small proportion of published articles addressing COVID-19. According to Retraction Watch, as of December 30, 2020, the retraction rate for COVID-19 articles was approximately 0.03%,[10] which is still lower than the overall retraction rate of 0.04% for scientific articles.[11]
General characteristics of retracted COVID-19 articles
Among the article types retracted, original research accounted for the majority, with 164 of 235. The journal with the highest impact factor that published retracted COVID-19 articles was The Lancet. However, most of the retracted articles (n = 65) were published in journals with no recognized impact factor.
As a new academic publishing model, the preprint platform has been widely recognized in various disciplines. The world's five major preprint platforms, MedRxiv, BioRxiv, SSRN, ArXiv, and ChemRxiv, focus on the fields of medicine, biology, the humanities and social sciences, physics and mathematics, and chemistry, respectively.[12–15] Before research results are published in formal publications, researchers can publish them on preprint platforms to establish the priority of research results and engage in academic exchange.[16,17] Preprint articles can be published without peer review, and peers can review them online or by sending e-mails.[18,19] In the face of sudden public health events, such as the COVID-19 pandemic, researchers may publish unverified or unapproved research results on preprint platforms in attempts to claim priority.[20]
In this study, the reasons for retracting COVID-19 articles published on the preprint platforms medRxiv, SSRN, and BioRxiv mainly included problems with results, copyright ethical issues, and issues with informed patient consent. Although preprint articles are not peer-reviewed and not "formally" published, they may be forwarded or cited by other authors.[21] Problematic COVID-19 articles published and disseminated on preprint platforms may spread inaccurate information to the public, causing unnecessary confusion and/or panic. In the early stages of the COVID-19 outbreak in February 2020, scientists from the Indian Institutes of Technology published a preprint paper on bioRxiv claiming that the novel coronavirus exhibited inserted fragments of the human immunodeficiency virus gene structure, suggesting it could be a human-made virus.[22] This view, published without review at the beginning of the outbreak when the public was still uninformed about COVID-19, attracted widespread attention and caused undue panic. Although it was quickly proven false, and based on incorrect data and results, it was retracted 2 days later; nevertheless, it had a negative impact.
Time periods of retracted COVID-19 articles
The publication periods of retracted COVID-19 articles were concentrated in 2020 and 2021, with 87 and 83, respectively. This is consistent with the development trend of the COVID-19 pandemic and the growth trend of the number of COVID-19 articles, as reported by Ni et al.[5] who found that the number of COVID-19 articles began to increase rapidly in April 2020. The retraction time periods of COVID-19 articles were concentrated in June and December 2020, with 12 and 9 articles, respectively. In other months, the number of retracted COVID-19 articles was 2–6 per month.
The retraction latency of retracted COVID-19 articles was the shortest at 0 days and longest at 741 days. The most common retraction latencies were > 5 months (n = 86) and < 1 month (n = 60), with an mean retraction latency of 5.9 months. As many as 42 articles were retracted within 10 days of publication. The mean retraction latency of COVID-19 articles was shorter than the mean retraction latency of basic life sciences or scientific articles in the literature (269 days and 667 days, respectively).[23,24] This demonstrates that sudden public health events are the focus of the academic community. Compared with general scientific articles, their self-purification efficiency is higher, which is conducive to promoting the academic community's in-depth research investigating sudden public health events and improving public understanding of them.
Reasons for retraction of COVID-19 articles
The most common reasons for the retraction of COVID-19 articles included errors by the journal/publisher, academic misconduct, and lack of IRB or IACUC approval. As many as 14.4% of retracted COVID-19 articles were due to errors by the journal/publisher, which are different from the reasons for the retraction of general scientific articles reported in most of the literature.[25] Of the 34 articles retracted due to errors by the journal/publisher, 21 were from Elsevier, and Elsevier also had the highest number of retracted COVID-19 articles, with as many as 60. Elsevier is an international publishing group with a history of > 140 years and > 2500 journals.[26] Although it publishes a large number of articles each year, the high proportion of retractions in the field of COVID-19, many of which were due to duplicate publication of text, suggests that the rigor of this publishing group in academic publishing needs to be improved.
In sudden public health events, medical journals respond quickly, use online first and open-access publishing to shorten the publishing cycle, and disseminate the latest research results quickly and efficiently.[27–30] However, under such circumstances, there are authors who use unverified research results for submission to get their articles published quickly, and some journals speed up or even simplify the review process to publish COVID-19 articles rapidly, resulting in a large number of low-quality COVID-19 articles. Editors of medical journals should enhance their sense of responsibility and mission, strengthen the review of articles addressing sudden public health events, and focus especially on results, data, and ethics. They should not omit the requirement for important information, such as original materials, data sources, and ethics approval in the pursuit of efficiency, to avoid publishing problematic manuscripts and causing adverse effects on epidemic prevention and control.
CONCLUSION
Sudden public health events pose new challenges to the publication and dissemination of information by medical journals. Currently, the retraction rate of COVID-19 articles has not exceeded the overall retraction rate of scientific articles, indicating that most medical journals continue to maintain strict control over the academic quality of COVID-19 articles, and are able to retract problematic articles in a timely manner. However, journal-related issues that appear in large numbers among the reasons for retracting COVID-19 articles should be given due attention. While quickly publishing and disseminating articles, retractions caused by journal errors should be avoided. Finding an acceptable balance between improving publishing efficiency and strictly controlling manuscript quality is a key measure that medical journals must implement during public health emergencies such as the COVID-19 pandemic. However, there are still certain limitations to this study, and no further analysis has been conducted on the impact of these retracted articles, such as citations, downloads, and even relevant positive comments. In the future, further research will be conducted on the impact of retracting articles related to public health emergencies compared to general articles.
DECLARATION
Author contributions
Fang YC: Writing—Original draft, Data collection, Data analysis. Wu JJ, Wu QL, Xu J: Conceptualization, Writing—Review and Editing. Yu J: Writing—Original draft. All authors have read and agreed to the published version of the manuscript.
Source of funding
This research received no external funding.
Ethics approval
Not applicable.
Conflict of interest
The author has no conflicts of interest to declare.
Data availability statement
Data used to support the findings of this study are available from the corresponding author at wujj39@mail.sysu.edu.cn.
REFERENCES
- The State Council of the People's Republic of China. [Emergency Regulations for Public Health Emergencies]. Accessed November 11, 2023. http://www.gov.cn/zhengce/content/2008–03/28/content_6399.htm
- Wilder-Smith A, Osman S. Public health emergencies of international concern: a historic overview. J Travel Med. 2020;27(8):taaa227. DOI: 10.1093/jtm/taaa227 PMID: 33284964
- WHO. [Coronavirus disease (COVID-19) pandemic]. Accessed November 15, 2023. https://www.who.int/zh/emergencies/diseases/novel-coronavirus-2019.
- Sha S, Jiang L. [Quick response and social responsibility of medical journals in the period of public health emergency:case study on the outbreak of COVID-19]. Acta Editologica. 2020;32(4):385–389.
- Ni J, Li P, Liu HX, et al. [Publication characteristics of Chinese and foreign biomedical journals during public health emergencies: Case study on COVID-19 papers]. Chin J Sci Tech Period. 2021;32(2):262–268. DOI: 10.11946/cjstp.202008120737
- Lucas-Dominguez R, Alonso-Arroyo A, Vidal-Infer A, Aleixandre-Benavent R. The sharing of research data facing the COVID-19 pandemic. Scientometrics. 2021;126(6):4975–4990. DOI: 10.1007/s11192–021–03971–6 PMID: 33935332
- Zhang W, Leng HM. [Information transmission channels of scientific journals in public health emergencies based on media convergence]. Acta Editologica. 2020;32(5):566–569.
- Retracted coronavirus (COVID-19) papers. Accessed November 15, 2023. https://retractionwatch.com/retracted-coronavirus-covid-19-papers/
- Ren YQ, Jin W, Weng YQ. [New characteristics of academic misconduct in retracted papers and enlightenment]. Chin J Sci Tech Period. 2019;30(12):1251–1258. DOI: 10.11946/cjstp.201903090150
- List of retracted COVID-19 papers grows past 70. Accessed November 15, 2023. https://retractionwatch.com/2020/12/30/list-of-retracted-covid-19-papers-grows-past-70/
- Brainard J, You J. What a massive database of retracted papers reveals about science publishing's 'death penalty'. Accessed March 23, 2023. https://www.sciencemag.org/news/2018/10/what-massive-database-retracted-papers-reveals-about-science-publishing-s-death-penalty
- Xu N, Miao XZ, Chen JX. [Enlightenment of preprint "great prosperity" to academic journals]. Acta Editologica. 2019;31(3):282–289.
- Xie HJ, Liu XM, Jing R. [Empirical research on the impact of preprint platform bioRxiv and suggestions]. Chin J Sci Tech Period. 2019;30(11):1218–1224. DOI: 10.11946/cjstp.201909110630
- Kirkham JJ, Penfold NC, Murphy F, et al. Systematic examination of preprint platforms for use in the medical and biomedical sciences setting. BMJ Open. 2020;10(12):e041849. DOI: 10.1136/bmjopen-2020–041849 PMID: 33376175
- Verma AA, Detsky AS. Preprints: a timely counterbalance for big data-driven research. J Gen Intern Med. 2020;35(7):2179–2181. DOI: 10.1007/s11606–020–05746-w PMID: 32141042
- Kaiser DE, Crossetti MDGO. Open Science and the emergence of preprints. Rev Gaucha Enferm. 2021;42:e20210030. DOI: 10.1590/1983–1447.2021.20210030 PMID: 33852673
- Irawan DE, Zahroh H, Puebla I. Preprints as a driver of open science: Opportunities for Southeast Asia. Front Res Metr Anal. 2022;7:992942. DOI: 10.3389/frma.2022.992942 PMID: 36225341
- Fu JY, He Y. [Token incentive mechanism of self-organizing peer review in preprint]. Chin J Sci Tech Period. 2021;32(1):23–27. DOI: 10.11946/cjstp.202005260531
- Johansson MA, Saderi D. Open peer-review platform for COVID-19 preprints. Nature. 2020;579(7797):29. DOI: 10.1038/d41586–020–00613–4 PMID: 32127711
- Massarani L, Neves LFF. Reporting COVID-19 preprints: fast science in newspapers in the United States, the United Kingdom and Brazil. Cien Saude Colet. 2022;27(3):957–968. DOI: 10.1590/1413–81232022273.20512021 PMID: 35293473
- Oh HC, Kang H. Current concerns on journal article with preprint: Korean Journal of Internal Medicine perspectives. Korean J Intern Med. 2023;38(3):332–337. DOI: 10.3904/kjim.2023.099 PMID: 37041691
- Pradhan P, Pandey AK, Mishra A, et al. Withdrawn: Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag. Accessed November 15, 2023. https://www.biorxiv.org/content/10.1101/2020.01.30.927871v2
- Zhang LH, Tian D. [Comparative Analysis of the Retraction Reasons and Time-To-Retraction in Different Disciplines]. Digit Libr Forum, 2019(8):40–44. DOI: 10.3772/j.issn.1673–2286.2019.08.006
- Li AD, Zhang X, Liang B. [Analysis on the phenomenon of retracted papers from international scientific journals written by Chinese scholars]. Chin J Sci Tech Period. 2019;30(10):1034–1038. DOI: 10.11946/cjstp.201907080492
- Wu RL, Deng ZQ, Wu SQ. [Analysis of the reasons for retraction of open access journals from multi-dimensional view: Based on the data from Retraction Watch Database]. Chin J Sci Tech Period. 2020;31(3):346–355. DOI: DOI:10.11946/cjstp.201910170709
- [Elsevier]. Baidu Encyclopedia. Accessed November 15, 2023. https://baike.baidu.com/item/%E7%88%B1%E6%80%9D%E5%94%AF%E5%B0%94?fromtitle=elsevier&fromid=2357645
- San Torcuato M, Bautista-Puig N, Arrizabalaga O, Méndez E. Tracking openness and topic evolution of COVID-19 publications January 2020-March 2021:comprehensive bibliometric and topic modeling analysis. J Med Internet Res. 2022;24(10):e40011. DOI: 10.2196/40011 PMID: 36190742
- Wang JZ, Cui JY, Tan CL, Sun SY. [Investigation and thinking on the online priority publication of COVID-19 research results on CNKI]. Chin J Sci Tech Period. 2020;31(4):483–489. DOI: 10.11946/cjstp.202003070182
- Zhou HQ, Xiao DB. [A Study on Academic Publication of Public Emergencies in Science and Technological Journals: a Case Study of COVID-19 OA Publication of CNKI]. Publ J. 2020;28(3):59–66. DOI: 10.13363/j.publishingjournal.2020.03.010
- Smith MJ, Upshur REG, Emanuel EJ. Publication ethics during public health emergencies such as the COVID-19 pandemic. Am J Public Health. 2020;110(7):e1–e2. DOI: 10.2105/AJPH.2020.305686 PMID: 32407155
