Fact or Fake? The Question of Anonymity in Peer Review

Expert reviews are the lifeblood of refereed scholarly journals. They remain the dominant method to validate content and are critical in maintaining high scientific standards. Yet the process is far from perfect.

* Blind reviews are also called “masked reviews” to avoid discrimination of blind people. However, the term “blind reviews” has been used for so long and widespread that to avoid misunderstandings, I am using the same term as most people will understand it. I support replacing the term with “masked reviews” when it is used more commonly in academia.

In this essay, I will discuss the various assumptions about blind* reviews and the practices of authors and reviewers in the referee process, that make blind reviews not truly blind but just fake blind.

This work is based on my 13 years of experience as associate editor and editor-in-chief of Journalism and Mass Communication Quarterly, the oldest academic journal in the field of journalism and mass communication and the flagship journal of the Association for Education in Journalism and Mass Communication (AEJMC).

Double-Blind Misconceptions

In the scholarly review process, there are a number of potential biases with sometimes harmful consequences to authors and reviewers.

Journal publishers and editors usually first think of protecting the identity of reviewers so they can honestly comment on a manuscript without having to fear retribution from the authors. This consideration has led to the system of single-blind reviewing in which the authors do not know the identity of the reviewers while the reviewers know the identity of the authors. Most scientific journals have adopted this system and continue to use it up to this day. In many other scholarship review settings, such as research grants and external reviews for faculty tenure and promotion, the single-blind review system has also been adopted.

However, in the social sciences and humanities, double-blind peer review, in which both reviewers and authors do not know each other’s identity throughout the review process, remains the norm. A study by Wiley has shown that out of the 1,593 journals it publishes, 95% of the health and physical sciences journals use single-blind reviews while only 15% of its social sciences journals do so.

Particularly in the humanities and social sciences, the outcome of reviews is prone to the subjectivity and values of reviewers. Generally, knowing about an author’s identity can greatly influence review results and disadvantage those who are junior in ranking, minorities, women, or not from well-known universities. Increased awareness of this discrimination has given rise to the use of double-blind reviewing.

“If double-blind is such a good system, why don’t all other scientific journals follow?”

Ever since its implementation, many scholars of disadvantaged groups have embraced the double-blind review system, believing it will allow them to publish without being prejudiced. However, if double-blind is such a good system, why don’t all other science journals follow? Before we compare the merit of double-blind versus single-blind review systems, let’s revisit the basic assumptions of the former:

1. Reviewers will not recognize who the author is if their name is not directly visible on the manuscript.
2. Reviewers will not accidentally find out the author’s identity by checking preprints or preliminary reports on the topic posted online.
3. Authors will not use other ways to reveal their identities intentionally or unintentionally.
4. There will no longer be bias against authors with their names removed from the manuscript.
5. The editor will choose reviewers who are experts in the field and use a balanced mix of reviewers.

Based on my experience, these five assumptions are probably too idealistic and naïve, and they do not stand with the following realities:

1. The best reviewers should be true experts in their field. They know what their peers are doing and have attended conferences, where they already learned about the submitted research. If a well-known scholar writes a manuscript, they inevitably leave traces of their identity by either citing many of their own works or referring to their own research. In that case, reviewers will immediately recognize who the author is. If they decline the invitation based on that, you might lose an expert who is truly knowledgeable about the subject matter. This would leave you finding some novice reviewers who don’t know much about the subject area and are not authoritative experts in the topic.
2. With easily available preprints on various research sharing platforms, such as ResearchGate, Academia.edu, etc., it is not hard to find out about an author’s identity if one wishes to do so.
3. Reviewers are tempted to guess who the author is, which may lead to misidentification. For instance, excessive citations may be caused by the author being a student of a well-known scholar or being heavily influenced by an established scholar. Ultimately, this misidentification based on citation patterns or else might result in a more favorable or less favorable judgment of the manuscript.
4. If the reviewer is an expert on the topic, they probably will check online to see if there are similar texts or if the findings look familiar. I have had several incidents where a reviewer reported to me that the submitting author “plagiarized” someone else’s paper while in fact the paper was from the submitting author himself or herself. However, by explaining this to the reviewers I lose them, as they are now aware of the author’s identity.
5. A good, ethical and experienced author should not reveal their identity in any shape or form in the manuscript. However, it is hard for a novice scholar to know what an excessive number of citations is – and what isn’t. Also, the pressure for citations in academia nowadays pushes junior authors to include their own works as much as possible (sometimes at the expense of other scholars’ works on similar topics). Some experienced scholars also cannot help to include their own works or try to boost the credibility of the manuscript by listing their own works.
6. A native English-speaking reviewer can rather easily tell whether a manuscript was written by a non-native English speaker. An experienced scholar also has a good chance of guessing whether a manuscript was written by a young scholar, based on common mistakes in manuscript organization, citations, or the implementation of tables and figures. Occasionally, reviewers still misidentify authors. I have had several cases in which a reviewer claimed, “This must be a graduate student’s work,” while in fact it was written by a professor. All these guessing games show that double-blind reviews may lead to either misidentification of authors or correct identification of the authors, which makes the review outcome biased.
7. Although the editor can try their best to find a good mix of reviewers, due to reviewer availability and timing, the ideal mix may not be attainable, and the editor has to live with whomever they can find at that time.

Considering the above-mentioned “fake” blind review practices and constraints, the double-blind review system seems to be in jeopardy. How to choose a review process that ensures a manuscript is judged solely by its merits and no other factors?

My proposed solution is not a single review system but a dual track system containing both single-blind and double-blind review options. The single-blind option would apply to manuscripts that cannot be truly anonymized due to the reasons listed above and/or when they can be discovered easily online. The double-blind option, on the other hand, would become effective when manuscripts have not been posted online and do not display excessive self-citations or citations of a specific scholar to be identified or misidentified. Since knowing the identity of an author would not necessarily be a disqualifier anymore, such a dual system could also maximize the availability of reviewers.

Louisa Ha is the Editor-in-Chief of the Open Access journal “Open Cultural Studies”

Louisa Ha, Peiqin Chen, Ke Guo (eds.)Online Media and Global CommunicationPeer-reviewed, scholarly journal for high quality articles on theories and methods about the role of online media in global communication.

[Title image by fizkes via iStock / Getty Images Plus]