Letting the Cat Out of the Bag on the Editor’s Role in Peer Review: It’s no Piece of Cake

We received our first respective peer-reviews in what seems like last century, shedding copious tears when reviewers asked basic questions like if we knew the difference between mg and ug. When invited to be associate editors, we loftily imagined dismantling systemic inequalities of peer review[i][ii][iii], offering sage advice to budding scientists, contributing to the communication and promotion of excellent science in our disciplines, and illuminating enlightening insights to lightly castigate reviewer 2. Broken down by the challenges of keeping up with manuscript submissions and reviewer recruitment, our sights are perhaps more realistic. Below, we offer catchy idioms and 15 hypothetical retorts we’d say to those wondering how to help editors as they decide whether to click "accept" or "decline" in that invitation to review. 

  1. Choosers can be Beggars. Our job as editors can sometimes feel like we are begging people to review. Really.  Editors aim to get excellent, informed, and diverse perspectives from reviewers.  We choose whom to invite by content expertise, prior reviews, location, reputation, and match.  But if our efforts are met with perpetual “I decline” responses, we might end up sending those personalized emails that implore the far reaching edges of our academic networks to please, please, please agree. Please be understanding if you are the recipient of one of these.
  2. The Ball is in Your Court. Respond to the invitations.  Behind the auto-generated spam is an exhausted person clicking through dozens of prompts asking if they’re sure they want to invite you -- yes, you! -- to review. 
  3. Don’t Bite Off More Than You Can Chew. We get it. You’re busy. Decline if you must … but respond. Ghosting is okay only with exes and Halloween.
  4. Pass the Buck. If you can’t take a review, please take a moment to suggest reviewers. This is your area of expertise. This is a chance to put your future-editor hat on and ponder the collective expertise of people like you. Not just the big fish in this small academic pond. This is a chance to lift up others in your field who might not be on the editor’s radar.
  5. Diversity Makes For a Rich Tapestry. Reviewers sculpt their craft to become editors and study section members. Reviewing shapes priority agendas as it simultaneously improves scientific writing. Representation matters -- and not just for papers with population demographics in the title. The peer review process is invisible labor and invisible networking. Recommend other reviewers strategically.
  6. A Question is Worth a Thousand Words. Hesitant about asking for something from the editor?  Need feedback about the quality of your review? Proposing a mentored review? Just ask. Editors are looking to train new reviewers, so they’ll often welcome a good mentored review. They are going to read your review anyway so it isn’t much extra effort on their part.
  7. A Stitch in Time Saves 9. Submit on schedule, but if you can’t, give the editor a heads up.
  1. Better Late than Never. If your review is late, it is okay.  Still do it.  Don’t get bogged down in the guilt of a late review.
  2. Review Unto Others As You Would Have Them Review Unto You. Every submitted manuscript represents an enormous amount of work.  Reviewers’ feedback is a learning opportunity that will be better received if delivered compassionately. The goal is to improve the quality of the paper not to make a grad student cry. Don’t be Reviewer 2.
  3. You Catch More Flies With Honey Than Vinegar. Write the weaknesses with the goal that authors will want to make the changes, not get defensive. Don’t forget about the strengths!
  4. Honesty is the Best Policy. This isn’t a place for subtlety…  err on the side of hyperbolizing strengths and being very clear about the magnitude and impact of the weaknesses. The editor is like a fly on the wall, listening in to scientific conversation between the reviewer and authors. Good reviews help them eavesdrop. It is much easier to make editorial decisions that echo reviewers than to get short, positive reviews that ignore important weaknesses or make minor weaknesses seem like critical flaws. Aim for clarity with kindness.
  5. Don’t Throw the Baby Out With The Bathwater. If the weakness is not a critical flaw, don’t make it into one. Recognize that every study has weaknesses and limitations & the best feedback is constructive: Do the authors draw conclusions with weaknesses in mind? Do we learn something valuable, despite them?
  6. All That Glitters is Not Gold. Assess whether fancy methods and designs are backed up with theory. Resist the allure of jargon and long-digited p-values. Even highly technical methods or statistics need to produce conceptually meaningful results.
  7. Missing the Forest for the Trees. Highlight themes that run throughout the manuscript. Details are important and can be helpful in the context of big picture concerns and especially as examples, but problems or mistakes that look like ranker lists may be hard for the authors to walk through.  Remember, brevity is the sister of talent.
  8. Ending on a High Note. Take this with a grain of salt but, reviewer, you are the best thing since sliced bread; the bee’s knees. There are other fish in the sea, but you really cut the mustard.  It takes reviewer #2 to tango and we’re glad you didn’t miss the boat. If we could, we’d give a penny for your thoughts. Or maybe a dime? A dozen?

[i] Lundine, J., Bourgeault, I. L., Glonti, K., Hutchinson, E., & Balabanova, D. (2019). “I don't see gender”: Conceptualizing a gendered system of academic publishing. Social Science & Medicine, 235, 112388.

[ii] Resnik, D. B., & Smith, E. M. (2020). Bias and groupthink in science’s peer-review system. Groupthink in science, 99-113.

[iii] Hojat, M., Gonnella, J. S., & Caelleigh, A. S. (2003). Impartial judgment by the “gatekeepers” of science: fallibility and accountability in the peer review process. Advances in Health Sciences Education, 8(1), 75-96.


 

Author Bios:

Birdie Shirtcliff is Associate Editor of Biological Psychology, Associate Blog Editor for Society for Research on Adolescence, and serves on several editorial boards and ad hoc study sections. She is a research professor at University of Oregon, Center for Translational Neuroscience. Her favorite idiom is “feeding two birds with one scone” and if she could create an idiom, this night owl would say “the early bird gets the crows feet.”

Christy M. Buchanan has served as Associate Editor for JRA and Developmental Psychology, and as Guest Editor and a member of the Editorial Board for other outlets. She is currently an Associate Blog Editor for SRA. She is a Professor of Psychology at Wake Forest University. 

Share this post:

Comments on "Letting the Cat Out of the Bag on the Editor’s Role in Peer Review: It’s no Piece of Cake"

Comments 0-0 of 0

Please login to comment