Article Text
PDF
Statistics from Altmetric.com
In 1904, Dr James Herrick evaluated a 20-year-old patient from Grenada who was studying in Chicago and suffered from anaemia and a multisystem illness. The patient was found to have ‘freakish’ elongated red cells that resembled a crescent or a sickle. Dr Herrick concluded that the red cells were not artefacts because the appearance of the cells was maintained regardless of how the smear slide was prepared. He followed the patient who had subsequently received care from other physicians until 1907 and questioned whether this was syphilis or a parasite from the tropics. Then in 1910, in a published case report, he concluded that this presentation strongly suggested a previously unrecognised change in the composition of the corpuscle itself.1 Sickle cell disease became a diagnosis thereafter.
Case reports and case series have profoundly influenced the medical literature and continue to advance our knowledge in the present time. In 1985, the American Medical Association reprinted 51 papers from its journal that had significantly changed the science and practice of medicine over the past 150 years, and five of these papers were case reports.2 However, concerns about weak inferences and the high likelihood of bias associated with such reports have resulted in minimal attention being devoted to developing frameworks for approaching, appraising, synthesising and applying evidence derived from case reports/series. Nevertheless, such observations remain the bread and butter of learning by pattern recognition and integral to advancing medical knowledge.
Guidance on how to write a case report is available (ie, a reporting guideline). The Case Report (CARE) guidelines3 were developed following a three-phase consensus process and provide a 13-item checklist that can assist researchers in publishing complete and meaningful exposition of medical information. This checklist encourages the explicit presentation of patient information, clinical findings, timeline, diagnostic assessment, therapeutic interventions, follow-up and outcomes.3 Yet, systematic reviewers appraising the evidence for decision-makers require tools to assess the methodological quality (risk of bias assessment) of this evidence.
In this guide, we present a framework to evaluate the methodological quality of case reports/series and synthesise their results, which is particularly important when conducting a systematic review of a body of evidence that consists primarily of uncontrolled clinical observations.
Definitions
In the biomedical published literature, a case report is the description of the clinical course of one individual, which may include particular exposures, symptoms, signs, interventions or outcomes. A case report is the smallest publishable unit in the literature, whereas case series report aggregates individual cases in one publication.4
The median number of patients in articles with ‘case series’ in the title was found to be seven (range 1–6432). The median (range) of the number of cases of articles with ‘case report’ as a publication type was four (1–178).5 Case reports/case series are usually retrospective although can occasionally be prospective, such as the Herrick’s first case report of sickle cell disease.1 Case reports/series can also define their subject by exposure or outcome (analogous to a cohort study and case–control study). Therefore, a specific number of patients, the temporal direction of follow-up or even the definition by case/exposure are not differentiating characteristics of case report/series. One unique feature, however, is that case report/series are uncontrolled (non-comparative) and have a relatively small number of individuals.
If a case series is prospective, differentiating it from a single-arm uncontrolled cohort study becomes difficult. In one clinical practice guideline, it was proposed that studies without internal comparisons can be labelled as case series unless they explicitly report having a protocol before commencement of data collection, a definition of inclusion and exclusion criteria, a standardised follow-up and clear reporting of the number of excluded patients and those lost to follow-up.6
Evaluating methodological quality
Pierson7 provided an approach to evaluate the validity of a case report based on five components: documentation, uniqueness, objectivity, interpretation and educational value, resulting in a score with a maximum of 10 (a score above 5 was suggested indicate a valid case report). This approach, however, was rarely used in subsequent work and seems to conflate methodological quality with other constructs. For case reports of adverse drug reactions, other systems classify an association as definite, probable, possible or doubtful based on leading questions.8 9 These questions are derived from the causality criteria that was established in 1965 by the English epidemiologist Bradford Hills.10 Lastly, we have adapted the Newcastle Ottawa scale11 for cohort and case–control studies by removing items that relate to comparability and adjustment (which are not relevant to non-comparative studies) and retained items that focused on selection, representativeness of cases and ascertainment of outcomes and exposure. This tool was applied in several published systematic reviews with good inter-rater agreement.12–16
Proposed tool
The previous criteria from Pierson,7 Bradford Hills10 and Newcastle Ottawa scale modifications11 converge into eight items that can be categorised into four domains: selection, ascertainment, causality and reporting. The eight items with leading explanatory questions are summarised in table 1.
Tool for evaluating the methodological quality of case reports and case series
For example, a study that explicitly describes all the cases who have presented to a medical centre over a certain period of time would satisfy the selection domain. In contrast, a study that reports on several individuals with unclear selection approach leaves the reader with uncertainty to whether this is the whole experience of the researchers and suggests possible selection bias. For the domain of ascertainment, self-report (of the exposure or the outcome) is less reliable than ascertainment using administrative and billing codes, which in turn is less reliable than clinical records. For the domain of causality, we would have stronger inference in a case report of an adverse drug reaction that has resolved with cessation of the drug and reoccurred after reintroduction of the drug. Lastly, for the domain of reporting, a case report that is described with sufficient details may allow readers to apply the evidence derived from the report in their practice. On the other hand, an inadequately reported case will likely be unhelpful in the course of clinical care.
We suggest using this tool in systematic reviews of case reports/series. One option to summarise the results of this tool is to sum the scores of the eight binary responses into an aggregate score. A better option is not to use an aggregate score because numeric representation of methodological quality may not be appropriate when one or two questions are deemed most critical to the validity of a report (compared with other questions). Therefore, we suggest making an overall judgement about methodological quality based on the questions deemed most critical in the specific clinical scenario.
Synthesis of case reports/series
A single patient case report does not allow the estimation of an effect size and would only provide descriptive or narrative results. Case series of more than one patient may allow narrative or quantitative synthesis.
Narrative synthesis
A systematic review of the cases with the rare syndrome of lipodystrophy was able to suggest core and supportive clinical features and narratively summarised data on available treatment approaches.17 Another systematic review of 172 cases of the infrequently encountered glycogenic hepatopathy was able to characterise for the first time patterns of liver enzymes and hepatic injury in this disease.18
Quantitative synthesis
Quantitative analysis of non-comparative series does not produce relative association measures such as ORs or relative risks but can provide estimates of prevalence or event rates in the form of a proportion (with associated precision). Proportions can be pooled using fixed or random effects models by means of the various available meta-analysis software. For example, a meta-analysis of case series of patients presenting with aortic transection showed that mortality was significantly lower in patients who underwent endovascular repair, followed by open repair and non-operative management (9%, 19% and 46%, respectively, P<0.01).19
A common challenge, however, occurs when proportions are too large or too small (close to 0 or to 1). In this situation, the variance of the proportion becomes very small leading to an inappropriately large weight in meta-analysis. One way to overcome this challenge is to transform prevalence to a variable that is not constrained to the 0–1 range and has approximately normal distribution, conduct the meta-analysis and then transform the estimate back to a proportion.20 This is done using logit transformation or using the Freeman-Tukey double arcsine transformation,21 with the latter being often preferred.20
Another type of quantitative analysis that may be utilised is regression. A meta-analysis of 47 published cases of hypocalcaemia and cardiac dysfunction used univariate linear regression analysis to demonstrate that both QT interval and left ventricular ejection fraction were significantly correlated with corrected total serum calcium level.22 Meta-regression, which is a regression in which the unit of analysis is a study, not a patient, can also be used to synthesise case series and control for study-level confounders. A meta-regression analysis of uncontrolled series of patients with uveal melanoma treated with proton beam therapy has shown that this treatment was associated with better outcomes than brachytherapy.23 It is very important, however, to recognise that meta-regression results can be severely affected by ecological bias.
From evidence to decision
Several authors have described various important reasons to publish case reports/series (table 2).7 24 25
Role of case reports/series in the medical literature
It is paramount to recognise that a systematic review and meta-analysis of case reports/series should not be placed at the top of the hierarchy in a pyramid that depicts validity.26 The certainty of evidence derived from a meta-analysis is contingent on the design of included studies, their risk of bias, as well as other factors such as imprecision, indirectness, inconsistency and likelihood of publication bias.27 Commonly, certainty in evidence derived from case series/reports will be very low. Nevertheless, inferences from such reports can be used for decision-making. In the example of case series of aortic transection showing lower mortality with endovascular repair, a guideline recommendation was made stating ‘We suggest that endovascular repair be performed preferentially over open surgical repair or non-operative management’. This was graded as a weak recommendation based on low certainty evidence.28 The strength of this recommendation acknowledged that the recommendation might not universally apply to everyone and that variability in decision-making was expected. The certainty in evidence rating of this recommendation implied that future research would likely yield different results that may change the recommendation.28
The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach clearly separates the certainty of evidence from the strength of recommendation. This separation allows decision-making based on lower levels of evidence. For example, despite low certainty evidence (derived from case series) regarding the association between aspirin and Reye’s syndrome in febrile children, a strong recommendation for using acetaminophen over aspirin is possible.29 GRADE literature also describes five paradigmatic situations in which a strong recommendation can be made based on low quality evidence.30 One of which is when the condition is life threatening. An example of which would be using hyperbaric oxygen therapy for purpura fulminans, which is only based on case reports.31
Discussion
Guideline developers and decision-makers often struggle when dealing with case reports/case series. On occasions, they ignore such evidence and focus the scope of guidelines on areas with higher quality evidence. Sometimes they label recommendations based on case reports as expert opinion.32 We propose an approach to evaluate the methodological quality of case reports/series based on the domains of selection, ascertainment, causality and reporting and provide signalling questions to aid evidence-based practitioners and systematic reviewers in their assessment. We suggest the incorporation of case reports/series in decision-making based on the GRADE approach when no other higher level of evidence is available.
In this guide, we have made the case for publishing case reports/series and proposed synthesis of their results in systematic reviews to facilitate using this evidence in decision-making. We have proposed a tool that can be used to evaluate the methodological quality in systematic reviews that examine case reports and case series.
References
Footnotes
Contributors MHM drafted the paper and all coauthors critically revised the manuscript. All the authors contributed to conceive the idea and approved the final submitted version.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.