I recently revisited an insightful piece of writing that got me thinking. This article talks about how even though the peer review process underlies all of science, it is understudied and its defects are more prominent than its merits. This article also mentions how peer review is often famously compared with democracy—as "a system full of problems but the least bad we have".^[1] This piece was published in 2006. Today, over 15 years later, peer review continues to occupy an important place in research, especially in grant funding and journal publishing. However, it has certainly been analyzed, discussed, and studied a lot more than it was before.^[2–5]

While there are several published works that delve into the workings of the peer review process, its challenges, advantages, and demerits, two global events that have opened up unlimited conversations about peer review among the scholarly publishing community are the International Congress on Peer Review and Scientific Publication^[6] and Peer Review Week.^[7] If you are unfamiliar with these events, here's a quick primer.

The International Congress on Peer Review and Scientific Publication (hereinafter referred to as Peer Review Congress): This in-person conference is organized by the Journal of the American Medical Association (JAMA) Network, British Medical Journal (the BMJ), and the Meta-Research Innovation Center at Stanford and is held once every four years. The Peer Review Congress brings together publishing industry professionals interested in sharing and discussing in-depth research and studies on the peer review process and journal publishing systems. The topics discussed at the congress include, but are not restricted to, peer review, scholarly publication, reporting practices, identifying and dealing with bias, measuring research impact, threats to research integrity, experiments with peer review, and new developments in scholarly publishing and peer review. The recurring theme of the Peer Review Congress is "Enhancing the quality and credibility of science", and the 2022 event focused on this broad theme too. Through plenary sessions, invited lectures, and in-person and online poster presentations, the presenters initiated thought provoking discussions on topics such as research misconduct, bias and spin, authorship, paper mills, author and reviewer guidance and training, data sharing and preprints, open science, reproducibility, and social media.

Peer Review Week: Peer Review Week is an annual virtual global event that celebrates the role of peer review in the scholarly publishing process. It is a fully community-led and community-focused event organized by a diverse steering committee that includes participant volunteers representing researchers, reviewers, editors, publishers, institutions, librarians, etc. Every year, the Peer Review Week Steering Committee runs with a specific theme to celebrate Peer Review Week. Past themes include identity (2021), trust (2020), quality (2019), diversity (2018), transparency (2017), and recognition (2016). Peer Review Week 2022 focused on "Research Integrity: Creating and supporting trust in research"—a theme that was chosen through an open global poll. This theme encouraged the community to think and talk about how research integrity is more valuable than ever today, when we are dealing with a surge in information sources and a reproducibility crisis. These conversations aimed to encourage the community to identify ways to ensure and promote quality peer review, which in turn will help increase our confidence in research.

As someone who has had the opportunity to be associated with both the Peer Review Congress and Peer Review Week in different ways over the years, I find them highly stimulating, thought provoking, and essential for facilitating global conversations about peer review and scholarly publishing. What do people in the industry think about peer review? Is everyone aligned with each other's expectations? What do researchers want from peer review? How useful do peer reviewers and journal editors find it? What are some of the problems and challenges associated with peer review? What are journals and publishers doing to improve their understanding of the peer review process? Where do we see peer review heading? These are just some of the broad questions both events encourage us to think about. I have attended the eighth (2017) and ninth (2022) Peer Review Congresses as well as have been part of the Peer Review Week Steering Committee (as a member from 2016 and later as Co-Chair of the 2021 and 2022 editions). The conversations I have had with authors, reviewers, and editors through both events have given me a lot of food for thought about peer review and I would like to share with you some of my key takeaways and top discussion points from the 2022 editions of the Peer Review Congress and Peer Review Week.

WE NEED MORE RESEARCH ON PEER REVIEW AND PUBLISHING

Much of what we know about how peer review and journal publishing work is not supported by hard data/evidence/peer-reviewed studies. Even though the scholarly publishing industry has made some strides in this area, there is a need for more in-depth studies and research to gather data about experiments, patterns, and trends in the peer review and scholarly publishing processes as well as potential solutions to some of the most pressing challenges we face (e.g., dealing with increasing volumes without an impact on the integrity of the information published).

PEER REVIEW TRAINING COULD PLAY AN IMPORTANT ROLE

Both formal and informal training and mentoring programs/initiatives could play an important part in helping equip reviewers with the best practices to be followed while evaluating a paper. This concept could also be extended to editors to help them navigate complex scenarios in the publishing process. In the long term, such an approach could help increase reviewer diversity and inclusivity as well as help build stronger peer reviewer pools for journals.

AUTOMATION IS ESSENTIAL, BUT IS IT THE ANSWER TO EVERYTHING?

Over the past few years, the scholarly publishing industry has been relying on technology to introduce efficiencies at various stages of the publication process. The implementation and uptake of artificial intelligence (AI)-based solutions has also increased. As a result, we have access to more complex technology and a greater willingness to experiment. However, while technology will help reduce human effort and deal with submissions at scale, some of the most important decisions and interactions in scholarly publishing must and will need human intervention/the human element. How this discussion continues and the direction this aspect will take in the future remains to be seen.

THERE IS A GAP IN PEER REVIEW

This was, for me, the most interesting takeaway from the Peer Review Congress and Peer Review Week. Academic research and scholarly publishing is an incredibly complex and constantly evolving universe that includes several processes, stages, and stakeholders. Of these, peer review is, arguably, one stage or process that shoulders the immensely heavy burden of expectations from not one but four stakeholder segments—authors, peer reviewers, journal editors, and readers (academic and non-academic). So, the humble peer review is not unlike Atlas who carried the weight of the world on his shoulders!

How can so many expectations be associated with one process? Here's a simplistic breakdown. Authors expect peer review to be fair and help elevate the quality of their manuscripts. Peer reviewers expect an opportunity to be part of what could be a path breaking development in their field or even the prestige associated with performing a review. Journal editors expect that peer review reports will assist them with solid recommendations to publish robust and credible research. The audience or end readers expect the authors, reviewers, and editors to have done their due diligence and ensure that what they read (for improving their knowledge or to seek evidence for critical decisions) has been vetted through peer review. But we often see a gap in understanding and expectations, especially between authors and reviewers.

One way to bridge this gap is to acknowledge the fact that authors are human as are reviewers and editors and that everyone is working towards the same goal. More inclusive and transparent discussions and communication about the scope of authors' and reviewers' roles could also help set the right level of expectations—the CRediT taxonomy developed by the National Information Standards Organization (NISO) is a great example of an effort in this direction.^[8] Finally, increased transparency among all stakeholders involved in the scholarly publishing workflow would be very helpful. Steps taken towards making research openly accessible, the increasing focus on preprints, and explorations of models such as open peer review are a step forward in this direction. However, the adaptation of open practices in scholarly communications could be more a social than a logistical issue, with a great need to educate more and more stakeholders about these practices.

THE RESEARCH CULTURE NEEDS TRANSFORMATION

Perhaps, most of the challenges we deal with at the scholarly publishing level have their roots in the highly competitive and somewhat unforgiving research culture. Using publications as the currency of progress is not the best approach to rewarding and acknowledging progress. Similarly, it is important to acknowledge that a majority of the peer reviews performed today are not recognized or remunerated despite the huge effort they require. For authors, we need to incentivize the right behavior at the beginning (ethical practices) instead of focusing on the end result (publication). We also need better structures and systems to recognize peer review effort. Introducing incentives for the right behaviors will help reward more authors and reviewers as well as increase the pool of willing reviewers.

These were just a few personal observations and takeaways from the Peer Review Congress and Peer Review Week. Over the years, at both these events, I have seen that researchers, reviewers, publishers, and other scholarly publishing professionals are now more eager and willing to have sustained and open discussions about how we can work together as a community to ensure that best and ethical practices are followed to disseminate credible scientific findings. I look forward to greater strides and developments in the scholarship around peer review and scholarly publishing.

DECLARATIONS

Author contributions

Jayashree Rajagopalan: Conceptualization, Writing- Original draft preparation, Writing- Reviewing and Editing.

Conflict of interest

The author has no conflicts of interest to declare.

REFERENCES

1. Smith R. Peer review: a flawed process at the heart of science and journals. J R Soc Med. 2006;99(4):178–182.    DOI: 10.1177/014107680609900414    PMID: 16574968
2. Publons. 2018: Global State of Peer Review. Accessed December 17, 2022. https://publons.com/static/Publons-Global-State-Of-Peer-Review-2018.pdf    DOI: 10.14322/publons.gspr2018
3. Checco A, Bracciale L, Loreti P, Pinfield S, Bianchi G. AI-assisted peer review. Humanit Soc Sci Commun. 2021;8:25.    DOI: 10.1057/s41599-020-00703-8
4. Acuna DE, Teplitskiy M, Evans JA, Kording K. Author-suggested reviewers rate manuscripts much more favorably: a cross-sectional analysis of the neuroscience section of PLOS ONE. PLoS One. 2022;17(12):e0273994.    DOI: 10.1371/journal.pone.0273994
5. Jefferson T, Wager E, Davidoff F. Measuring the quality of editorial peer review. JAMA. 2002;287(21):2786–2790.    DOI: 10.1001/jama.287.21.2786
6. International Congress on Peer Review and Scientific Publication. Accessed December 17, 2022. https://peerreviewcongress.org/
7. Peer Review Week. Accessed December 17, 2022. https://peerreviewweek.wordpress.com/
8. The National Information Standards Organization. CRediT, Contributor Roles Taxonomy. Accessed December 17, 2022. https://groups.niso.org/higherlogic/ws/public/download/26466/ANSI-NISO-Z39.104-2022.pdf    DOI: 10.3789/ansi.niso.z39.104-2022