Peer Review : from Blinded to Open : the Way Science is Communicated
Open Peer Review : what is it? What are the advantages, downsides and challenges for researchers ? Does open review approach give credit to reviewers for the time and effort they put into improving the quality of the manuscripts ? ...
In 2017, OpenAIRE released a report - entitled “OpenAIRE survey on open peer review: Attitudes and experience amongst editors, authors and reviewers” - that gauged the views towards open peer review (OPR) of over 3,062 editors, authors and reviewers. The results show that OPR is moving mainstream, with high levels of enthusiasm and experience amongst those surveyed.
From drawback of Traditional Peer Review ...
‘Often, the closest “peers” in someone’s area of research are also that researcher’s direct competitors ! One solution is to remove the authors’ names from the manuscript, but this double-blind system is not fool-proof, and a reviewer will still often recognize which lab a paper comes from. In addition, any bias towards competitors of the reviewer still remain, even if that competitor is anonymized. Another drawback of traditional peer review is that the referee reports are visible only to the authors and the editor...’ (F1000Research).
... to “Open Peer Review” (OPR)
Despite being a major pillar of Open Science, OPR has neither a standardized definition nor an agreed schema of its features and implementations. The literature reflects this, with a myriad of overlapping and often contradictory definitions:
* While the term is used by some to refer to peer review where the identities of both author and reviewer are disclosed to each other,
OPR : Facts, Advantages, Disadvantages, Challenges
* The results of the aforementioned online cross-disciplinary survey (conducted for the OpenAIRE2020 project) show the majority of respondents to be in favour of OPR becoming mainstream scholarly practice, as they also are for other areas of Open Science, like Open Access and Open Data.
A survey has also discovered high levels of experience with OPR (76.2% respondents reporting having taken part in an OPR process as author, reviewer or editor). There were also high levels of support for OPR workflow in terms of open interaction, open reports and final-version commenting.
Taken together, the survey's findings are very encouraging for OPR’s prospects though due care must be taken to avoid a “one-size fits all” solution and to tailor such systems to different (especially disciplinary) contexts. OPR is an evolving phenomenon and hence future studies are to be encouraged, especially to further explore differences between disciplines and monitor the evolution of attitudes.
* The benefits of Open Peer Review (OPR) - for authors, readers and reviewers - are indisputable:
Hence, OPR can be seen as a supportive and collaborative process between referees and authors, as well as an ongoing dialogue (rather than a selection system) between groups of scientists to progressively assess and predict the quality of published research.
* Advantages and disadvantages : reporting from the field … "Isn't this what science is all about?"
“Our recent research […] is now available for commentary and formal peer review in two preprint repositories: SJS (@social_sjs) and bioaRxiv (@bioarxivpreprint). Each of these repositories comes with advantages and disadvantages.
BioaRxiv is already backed by a large community, provides a DOI for indexing and citing, and tracks article usage statistics across the web. Its big disadvantage is that, just like in any other repository, articles simply sit there waiting to be published in a traditional journal in order to acquire some quality indicator —no matter how inaccurate and perverse— that will inform readers and be useful for authors in the advancement of their careers.
SJS, on the other hand, is the first and only repository that facilitates a formal peer review process. Its big disadvantage is that it is not yet supported by a big community that would ensure sustainability and greater visibility...
So, while our article is sitting quietly on bioaRxiv, we are sending personalised invitations to a number of experts in the field to formally review the version hosted on SJS. In less than a week after uploading the article, we already received two signed reviews by known experts in the field. One reviewer is suggesting a revision, making excellent suggestions for improvements that we will have to incorporate in the next version of the manuscript.
All reviewer comments are posted alongside the article, making it easy for readers to follow the discussion paragraph by paragraph. Openness and transparency make the review process a collaborative effort to detect and correct errors, enrich interpretations and generally improve validity, reproducibility, and quality - Pandelis Perakakis, PhD, Academic Website, July 1, 2017
* What are the challenges of OPR ? … Some opinions :
@ 'As well as research to assess whether different models of peer review lead to better quality peer reviewer reports, we more importantly need research to assess whether they lead to better quality articles – we need to not lose sight of the fact that that is after all what the process is trying to achieve', - Stephanie Boughton, 15 Jun 2016, BioMed Central blog
@ 'Some researchers, especially those in more junior positions, are afraid that if they are vocal or negatively critical about another researcher’s work in public, especially one who is more senior, then there could be a potential backlash against them for it. Documenting this sort of power dynamic abuse is difficult to see or measure, but certainly impacts upon earlier career researchers, who are perhaps those more willing to engage in more open research practices', - What are the barriers to post-publication peer review?
@ ‘Although open peer review is becoming more common, and addresses several of the issues of anonymous review, a few challenges still remain. A study in the early days of open review suggested that naming referees slightly reduced the likelihood of finding reviewers but did not affect the quality of review. Conversely, other studies suggest that open review provides more constructive reports’ , - F1000Research.
Some examples of Platforms, Journals and Publishers with OPR system implemented
- The F1000Research Open Research publishing platform (with 18 Gateways; including GODAN (Global Open Data for Agriculture and Nutrition) publishes open referee reports (example article).
- The Atmospheric Chemistry and Physics journal publishes pdfs with peer reviews alongside the final version of the paper (example article).
- Also Molecular Systems Biology publishes a “Review Process File” (example).
- In BMC Cancer the reviewers’ names are published with the review reports (example).
- The journal Royal Society Open Science publishes reviewer reports under “Review History” for each paper (example).
- Peer and Nature Communications journals have a system where both the reviews and the names of the referees can be optionally made open.
- BioMedCentral includes both the reviewer names and the peer review history (pre-publication) alongside published manuscripts in the medical journals of the BMC series.
- The Frontiers series now publishes all referee names alongside articles, and EMBO journals publish a review process file with articles, with referees remaining anonymous but Editors being named.
- Royal Society Open Science make its referee reports public under an open access license, CC-BY.
- ScienceOpen recently had a case where groups of students tested the reproducibility of a paper on this platform. Each post-publication peer review was courteous, detailed, and constructive. The authors of the paper responded that they were delighted with the attempts to critique and improve their research, with no ill side effects to those performing the evaluations (What are the barriers to post-publication peer review?, Jon Tennant, OpenAIRE blog, 2017).
Get more insights into peer review:
- OPENing UP new methods, indicators and tools for peer review, dissemination of research results, and impact measurement
- A Curious Blindness Among Peer Review Initiatives (The Scholarly Kitchen, 2018)
- OpenAIRE blogs about Open Peer Review
- How do you feel about open peer review?
- Should peer review reports be published?
- Study Reports Open Peer Review Attracts Fewer Reviews, Quality Suffers
- Turning a Critical Eye on Reference Lists
- Peer review: open sesame?
- Who’s Afraid of Open Peer Review?
- ScienceOpen : a freely accessible research network
- From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics (Free PMC Article, 2015)
- Getting credit for review
- Anonymous or signed reviews
- Pre- or post-publication peer review
- Citations are peer review by the world
- Challenges on Open Peer Review (slides)
- OKAD & F1000Research: a very different approach to publishing agricultural research (slides)
- Publish openly but responsibly (Science, 14 Jul 2017)