Screening research papers by reading abstracts

The BMJ receives approaching 8000 manuscripts each year and accepts only about 7% of them. Editors reject about 60-70% of original articles without external review. When a paper is clearly unsuitable for the BMJ just one editor can make the decision to reject it. When the decision is less clear other editors are involved.

The low acceptance rate makes the BMJ a big rejection machine and leaves many of our customers dissatisfied. But triaging papers at an early stage allows us to spend as much time and effort as possible on the peer review, commissioning, and editing of material that we think will be relevant, useful, and important to our readers, material that we want to publish. Furthermore, rejecting unsuitable papers quickly allows the authors to submit their work to another journal. That delay may be as little as a few hours. Daily duty editors make initial decisions within 24 hours of submission of research papers and can reject manuscripts, send them for eternal review, or pass them to colleagues for a further opinion almost instantly using our online manuscript processing system (submit.bmj.com).

How do BMJ editors make decisions about research papers? During initial screening, the first editor makes judgments about originality, importance, and relevance. The ideal paper, given that the BMJ is a general medical journal with an international readership, would be useful to as many readers as possible around the world and appeal to a broad medical readership. Its findings would be directly relevant to patient care or to healthcare policy that would affect patients. The research question would be one that really needed answering, and the findings would be credible and would add enough to existing evidence, rather than simply comprising another small brick in the wall of knowledge. The authors would have used the right research design to answer the question, and any weaknesses in the design would be outweighed by important strengths. Our focus is on the research question and then the methods used. We do not decide the paper's fate on whether the findings are positive or negative.

A couple of years ago it became clear that several BMJ editors were making at least preliminary decisions on submitted research papers by reading only the abstracts, and we decided to test whether this was valid. We conducted an experiment to see if editors at the BMJ could make decisions about research papers based on reading only the abstracts, and to compare how each initial decision differed from the final one after reading the whole submission.1 Only original research papers containing a structured abstract were included in the study. Medical editors acting as first readers of BMJ submissions had to read the abstract of each manuscript allocated to them and read no other material related to the submission.

Editors recorded the time taken to read each abstract and either their decision (immediate rejection, send to external peer reviewer, need for further in house consultation) or their inability to make a decision based on the abstract alone. Having made a decision based on the abstract, editors then read the whole manuscript and recorded on a separate form the time taken to do this and their decision based on reading the whole submission. The papers then continued through the rest of the process as normal, and the BMJ researchers followed them up and recorded the outcomes.

For about two thirds of submissions seen during the study, editors were able to make decisions based on reading only abstracts. They went on to make similar decisions when they read the corresponding whole submissions. For all papers that editors thought should be rejected after reading the abstract, the final decision after full processing was still rejection. The BMJ researchers concluded that it would be valid for editors to reject a submission after reading only the abstract if they were confident about that decision, and that when they felt they needed to read a full submission to make a decision, they should do so.1

BMJ editors now begin screening original research papers by reading only the abstract. Our estimate is that an initial decision is made on the abstract alone in 15-25% of papers. In a further 30-40% of papers editors look at the full paper for one or two specific points, which usually concern the paper's originality or methods. The remaining papers are read more fully before an initial decision is made. This is not as radical as it might sound: it is routine, for example, for conference panels to screen submissions by assessing abstracts. The 30-40% of papers that we send to external peer reviewers are read more fully by editors, and by reviewers and—if they survive external peer review—in painstaking detail by several people at our editorial advisory (or hanging) committees.

If you intend to submit an original research paper to the BMJ please ensure that its structured abstract is as complete, accurate, and clear as possible—but not unnecessarily long—and has been approved by all authors (see our full advice on writing structured abstracts at http://bmj.bmjjournals.com/advice/sections.shtml). All too often abstracts are poorly written, incomplete, misleading, and plain wrong.2 3 And, anecdotally, we know that the abstract is often the last piece written and is left to the least experienced author to produce.