Artificial Intelligence and Peer Review: Through the Editor’s Lens
The outcome of peer review can make or break a researcher’s career. With this weighty responsibility in mind, could we ever bestow this task – partly or completely – upon artificial intelligence technology? We asked our editors for their views.
From 25-29 September 2023 we celebrate Peer Review Week (PRW), which this year explores the relationship between peer review and the future of publishing.
For many academic authors, peer review manifests itself in an emailed list of recommendations about their manuscript provided by an anonymous expert in their field. Now, consider a scenario where this expert is not a human scholar, but an artificial intelligence (AI) tasked with deciding between acceptance, revision and rejection. Given that new technologies such as large language models have the ability to mimic human speech, would people be able to tell the difference? And if the comments made sense, would we even care who or what is behind it?
On the occasion of Peer Review Week 2023, we asked some of our books and journals editors about their experiences and the perceived benefits and pitfalls of AI-assisted peer review. We wanted to know whether they would trust AI in the peer review process and what aspects it could take over. Basically, do they think we’re there yet?
Words From Our Editors
“One might feel that we are ready to use AI everywhere, given the recent explosion of its popularity in almost every aspect of our lives. However, the review process requires extreme accuracy and confidence, which cannot be provided by the current large language models. I believe we can use various forms of AI to make writing, reviewing and publishing easier, more accurate and faster. But we are not there yet. It seems inevitable that we will be in a few years’ time though.”
Dr. Konrad Sarzyński
Managing Editor of the journal Economics
“The use of AI in peer review would have the benefit of assisting the reviewer in several tasks such as comparing the result with existing literature, plagiarism, correcting references, misprinting, and verifying the English. The disadvantages arise only when you expect what AI cannot (yet) deliver: evaluate the scientific importance of a paper and verify its correctness. If AI is used correctly there are only benefits, otherwise it could become a monster capable of killing real science.”
Prof. Vincenzo Vespri
Editor-in-Chief of the journal Open Mathematics
“I think AI can be very useful to correct general errors such as spelling mistakes, grammar and even the correct use and placement of references. However, I doubt that AI can do a good job in science. AI is based on existing knowledge, and I am not sure if it is capable to accept new scientific findings and draw the right conclusions.”
Dr. Christoph Steiner
Editor of the journal Open Agriculture
“Whether we like it or not AI will play a role in peer review, if it does not already. However, it will probably never come to the point of replacing traditional peer review as that depends not only on expertise and recognition of past scholarship (AI with a large database will exceed human capabilities of recall and identification) but also on personal judgement.”
Prof. Paulo de Medeiros
Editor of the book series Culture and Conflict
“As we know, AI works on the basis of past information. Thus, in research that has an innovation aspect, AI will not be useful. But, for sections such as the introduction as well as in the case of research similar to existing findings, the use of AI will have many advantages.“
Dr. Alireza Haghighi Hasanalideh
Editor of the journal Open Agriculture
“I asked ChatGPT 3.5 (openai.com) this question: ‘Have you ever encountered AI being used in the peer review process? If so, what role did it play and did it do a good job? Answer in 500 characters, including spaces.’ The response was very interesting and coherent:
‘Yes, AI has been used in peer review for tasks like language checking, plagiarism detection, reviewer matching, and content analysis. Its effectiveness varies; it aids language correction and plagiarism detection but struggles with nuanced judgment. Balancing AI’s role with human review is vital to maintain quality and prevent bias.’
The fact is I do use ChatGPT for language improvement in my peer review reports, but always proofread the changes ChatGPT makes in the text.”
Dr. Shahid Hussain
Editor of the journal Open Agriculture
What are your thoughts on the topic?
Stay tuned for more blog posts on Peer Review Week 2023 and join the conversation on our social media channels using the hashtag #DeGruyterPRW!
To learn more about the basics of peer review, check out our resources for authors and editors.
[Title image by Andrea De Santis via Unsplash]