When the process works

Any scientist can attest to when the peer review process has gone well. In fact, most of us credit our best work to that modified based on editorial and reviewer feedback. In this respect, editors and reviewers can (and do) recommend additional steps that strengthen reporting and clinical interpretations. Examples include encouraging authors to incorporate additional literature, to conduct or evaluate additional analyses (eg, adjustment for multiple testing, reporting absolute effect sizes), attending to alternative clinical or scientific interpretations (eg, placing findings back into the clinical context) and not allowing conclusions to go beyond those allowed by the data. When done well, editors and peer reviewers drive and maintain the high standards of health science reporting.

When it does not

At the same time, the process does not always move smoothly. Many factors contribute to this, including greater competition for publication in prestigious journals, an abundance of review requests for a relatively smaller pool of peer reviewers, mistaken ideas about what the scope of peer review should be and most critically, an absence of explicit guidance about a journal's expectations and requirements in the peer review process. The result is that constructive and timely advice may be forgone in favour of re-creating and dismantling the science. Numerous unnecessary and detailed revisions can result in substantive delays (from months to years from submission to acceptance) without demonstrable improvement in the final product. Worse, these steps can (and do) introduce lower quality science (eg, post hoc analyses) with an inordinate loss of time and effort to the researcher, reviewer and editor and delays for the public. The question is whether these recommendations and the attendant costs, are critical to high-quality reporting and positioning of that scientific work for publication in that journal. Does peer review interfere with the path of communication between scientist and public?

Avenues for improvement

We recommend that editors review manuscripts from the start to judge publication priority (eg, Does the paper advance the science? Does it fit with the mission/priorities of the journal?). Determining whether a paper meets journal priorities can be done prior to the peer review, known as the ‘insta-reject’ (AD Bryan, email communication, October 2009). While this may feel brusque to an author team, it is a scientific saving grace. Letting authors know that the manuscript, as it is currently conceived or developed, is not a good fit for the journal, allows authors to quickly, with no revision, pick a better and more suitable home. Papers sent for peer review should be only those likely to be published if a detailed peer review confirms the editors’ initial impressions. The only papers that should be sent for revision are those that will be published provided the authors are responsive to the revision requests.

We also encourage editors to provide reviewers with clear journal guidelines, requiring that they ‘affirm’ proof of this knowledge prior to review (eg, via checkbox within the journal review site). We challenge journals to require reviewer attention to priority over preference, with revisions only requested in instances where attentive responses will improve the manuscript and position it for eventual publication. To facilitate transparency, we support open review, with reviewer names available to author teams. Finally, we support a mediational role for editors, whereby editors actively synthesise reviewer comments to guide authors about which reviewer comments to respond to, which to ignore and what is ultimately expected from the author team to best position their submission for eventual publication within that journal (eg, whether to respond to each comment, how to manage conflicting queries from different reviewers, whether or not to conduct new analyses). This represents a substantive shift from the convention of providing a templated letter with appended reviewer comments, without any editorial summary or guidance.

Recommendations for improvement

Ultimately, peer review is not perfect. However, it is the best process that we have to ensure the integrity of our science and the public dissemination of our scientific work. We therefore offer these simple recommendations in a format that editors can easily share with peer reviewers (see box 1).

For the journal:

1. Provide peer reviewers with concrete guidelines about how to review a manuscript, including attending to questions such as (A) originality, (B) importance of the work to general readers and (C) scientific reliability. Examples include: BMJ: http://www.bmj.com/about-bmj/resources-reviewers/guidance-peer-reviewers; Nature: http://www.nature.com/authors/policies/peer_review.html

2. Require that reviewers read the relevant journal's review guidelines before beginning a review for that specific journal. This would be akin to the disclosure and proof of knowledge/ethics form that most journals require for authors in the manuscript submission process.

3. Conduct open review (eg, BMJ), whereby authors will know who has reviewed their work, to open the process of transparency and communication.

4. Peer reviews should be timely. Ideally, recommendations to authors should arrive within 4 weeks of submission.

For the peer review team:

• Review the work before you. Suggesting the authors do a different study is not helpful to the process of improving the current manuscript (unless it is a recommendation for the discussion section). Do not belabour the process by adding steps that reflect reviewer preference rather than scientific integrity.

• Be courteous, clear and concrete. Provide clear and actionable recommendations. If the requested changes cannot be made quickly and efficiently (eg, requiring collection of a new sample), it is not a good fit.

For the editorial team:

• Be prepared to undertake a mediational role, filtering and synthesising reviewer comments so that authors clearly know what steps should be taken to best position their paper for acceptance within that journal.

• Quickly notify authors if the paper does not fit editorial priorities, or if the revisions will not make the paper acceptable for eventual publication in that journal.