Exploring the Human Dimension of Peer Review
Our scientific identities shape our notions of quality, value, and relevance, adding an essential human dimension to the evaluation of science. It is therefore time to think (and act) outside the box and to explore peer review as an identity-informed rather than identity-agnostic mechanism in science.
Imagine you are attending a virtual talk on “Identity in Peer Review” and you are asked to use the chat to share three words that the theme brings to mind. What might you write? And what might you see? Take a moment—before reading further—and imagine a possible list. Think broad and wide. Below is my imagined list.
“To be seen”
“Science without borders”
“Transparency, accountability, rigor”
“Representation is key”
“Global scholarship”
“Bias, gender, race”
“Love this theme!”
“Science is color blind”
“Equity not equality”
My imagined list may or may not have much in common with yours. But it tells you something about what the topic means to me. My list, like yours, is informed by my identity—scientific and personal, my identity as a member of the academy, as a citizen, etc. Intersectionality, understood as the unique juxtaposition of identities within each one of us, is reflected on my list. And so are my experiences—professional and personal, and my scholarly pursuits. The latter include writing and advising on the global gender gap in science and mathematics, and working on transforming STEM and the academy to affirm identity by design.
What is identity? What aspects of it play a role in peer review? And what is a peer, from an identity centered lens? How does identity enter into “maintaining scientific quality”—the central aim of peer review? Is “scientific quality” inclusive of scientific value or scientific relevance? Whose values, whose scientific priorities might we implicitly center through peer review?
Names and Peer Review
Depending on the context, a person’s full name is either equivalent to their identity (as in “concealing an author’s identity”) or a poor proxy for it (as in “who am I, as a mother?”). Names—of authors, but also of institutions, editors, funders—are arguably the most concrete way in which “identity” enters the sphere of peer review. In this context, identity, meaning names and author characteristics we might infer from them (e.g., gender, ethnicity, nationality) are in principle independent of the quality of the scholarly output being evaluated. As reviewers we leave them aside, or try to, aided by blind (or in spite of open) review policies of various kinds.
Single blind reviews—by and large the standard in science and mathematics—only hide reviewers’ identities; author(s) names and institutional affiliations are included with the material to be reviewed. Double-blind reviews conceal both authors and reviewers identities; triple-blind reviews—used in disciplines like philosophy—further conceal an editor’s name. Through concealing identity, double- and triple-blind reviews aim to ensure fairness and promote trust in the evaluation of scientific quality. Yet, as policies, they are more controversial than expected. Their efficacy depends on the fidelity with which they are applied, and their perceived success on the data and methods used to evaluate them. In science and mathematics, the dominant culture still vies for single-blind reviews as the rule, in spite of evidence of reviewer bias favoring institutional and professional prestige and, more contestably, male gender—both aspects of an author’s identity.
“Perhaps blinding, in its various forms, suggests a deficit view of identity.”
Perhaps blinding, in its various forms, suggests a deficit view of identity—we hide to be fair and, in privileged cases, we identify ourselves to be recognized. What might a strengths-based view of identity look like and imply for peer review? Why (and how) might this matter? A flourishingly diverse scientific community calls for considering the question with certain urgency. In science and mathematics, recent advocacy for increased diversity among editors and reviewers might be as step towards strengths-based approaches to peer review. Increased representation based on demographic data alone won’t suffice, however. How might author identity matter beyond demographic characteristics and a name?
Identity beyond a Name
In recent interviews, colleagues and I asked university undergraduates to describe who they felt they were, both as persons and as college students taking mathematics. The second part of the prompt meant to elicit students’ perceived identity within a space, or a discipline—their math identity of their science identity. Such identities are not entailed in a name, but permeate unescapably not just what we write and why we write it, but also our very scholarly pursuits, research goals and agendas. Indeed, the very research questions we prioritize in science are critically informed by who we are and what motivates us, from finding a more efficient and elegant proof to a cryptography theorem, to working with local communities to understand and model and endemic water problem.
On the flip side of ensuring fairness in peer review, identity gives science its essential ethos—its color. And it always has. Any perception to the contrary suggests dominant identities—pervasive within science since time immemorial—have naturally blended into the fabric of science itself. Identity gives science essential color, color that stands to be diversified and renewed, color that is progressively critical for science’s continued value and relevance. Can peer review not just protect but actually enhance critically diverse identities in science?
Identity and the Meaning of “Peer”
Who is your peer? The question may be addressed from the angle of expertise, but it is also fundamentally important from the angle of identity. When we review, our science identities are engaged. These identities are tied to values, experiences, and realities. They influence our evaluation of quality when author names and affiliations are blinded and when they are not.
“An enhanced notion of peer requires commitment to evidence-based equity work, a strengths-focused view of identity, and organizational disposition to change.”
A peer review process encompassing deliberate attention to identity might call for considering an enhanced notion of peer—an expert not just open to considering the value identity adds to scientific output, but actually committed to learning how to do this in practice (not the detriment of scientific rigor and quality, but for their survival). An enhanced notion of peer goes beyond (and is perhaps separate from) increasing representation of traditionally minoritized populations within science. It requires commitment to evidence-based equity work, a strengths-focused view of identity, and organizational disposition to change.
Identity and Scientific Quality, Value, and Relevance
Our scientific identities—in being identities, shape our notions of quality, value, and relevance, adding an essential human dimension to the work of evaluating science. Ensuring “scientific quality” is part and parcel of peer review. Yet the concept of quality is unavoidably subject to experts’ and reviewers’ interpretation. So are ties between scientific quality and value. Insights into such interpretive views—what they are, how they vary across reviewers identities, how they influence social dimensions of science—are few and remain largely unexplored.
Open or public review mechanisms (in which reviewers’ identities might be known and their reviews publicly shared) might shed light on how identity (reviewers’ and authors’) underpins perspectives on scientific quality and its interplay with value. But furthering that understanding should likely happen by design. How is identity central to value and relevance in science? How is that potential centrality accounted for in peer review?
While largely a product of a gendered and arguably elitist academy, peer review also enjoys a useful independence from it. The theme of identity presents an opportunity to think (and act) outside the box, an opportunity to explore peer review as an identity-informed (rather than identity-agnostic) mechanism in science. This perspective suggests peer review might serve to actualize normative views of scientific quality, and enhance the strengths of a growingly diverse and global scientific community—including authors, reviewers, editors and funding agencies. With themes like identity as beacons, peer review can be a vehicle for transformative change.
You might also be interested in this essay
[Title image by PeopleImages/iStock /Getty Images Plus]