Background: We compared the quality of structured abstracts of original research articles from the British Medical Journal (BMJ), Canadian Medical Association Journal (CMAJ), and the Journal of the American Medical Association (JAMA) from 1991 to 1992 and 2001 to 2002 between journals.
Methods: A random, stratified sample of 54 abstracts from 2001 to 2002 in BMJ, CMAJ, and JAMA was compiled and coded. Two blinded raters reviewed 27 abstracts each against 33 objective criteria, separated into eight categories (purpose, research design, setting, subjects, intervention, measurement of variables, results, and conclusion). The quality score was the proportion of criteria present (range = 0-1).
Results: The overall mean quality score (0.74) for 2001-2002 was significantly higher than the 1988-1989 unstructured abstracts (mean = 0.57; p<0.001) but not different from the 1991-1992 structured abstracts (mean = 0.74; p>0.05). In 2001-2002, abstracts of CMAJ and JAMA (both means = 0.76) improved significantly over 1991-1992 (p<0.05) and scored significantly higher than BMJ (mean = 0.71; d.f. = 16, p<0.05). Some individual criteria scores (intervention, statistical information) improved but information was found consistently under-represented in areas that imply shortcomings of the studies.
Interpretation: We found a consistency in abstract quality regardless of the precise format used by different journals. This indicates that the framework for research articles already in place should be maintained and further modification of the framework may not necessarily improve the abstract quality.