Article Text
PDF
Abstract
Aim: To evaluate the proportion of interventions that are evidence based in the acute care unit of a regional eye hospital.
Methods: A prospective clinical audit was carried out at Hong Kong Eye Hospital in July 2002 to investigate the extent to which ophthalmic practices were evidence based. The major diagnosis and intervention provided were identified through chart review. A corresponding literature search using Medline and the Cochrane Library was performed to assess the degree to which each intervention was based on current, best evidence. Each diagnosis intervention pair was accordingly analysed and graded. The level of best, current evidence supporting each intervention was graded and analysed.
Results: A total of 274 consecutive consultation episodes were examined. 22 cases were excluded since no diagnosis or intervention was made during the consultation. 108 (42.9%) patient interventions were found to be based on evidence from systematic reviews, meta-analyses, or randomised controlled trials (RCT). Evidence from prospective or retrospective observational studies supported the interventions in 86 (34.1%) patients. In 58 (23.0%) cases, no evidence or opposing evidence was found regarding the intervention. The proportion of evidence based on RCT or systematic reviews was higher for surgical interventions compared with non-surgical interventions (p=0.007). The proportion of interventions based on RCT or systematic reviews was higher for specialist ophthalmologists than trainee ophthalmologists (p=0.021).
Conclusion: This study demonstrated that the majority of interventions in the ophthalmic unit were evidence based and comparable to the experience of other specialties.
- evidence based medicine
- ophthalmology
- Hong Kong
Statistics from Altmetric.com
Evidence based medicine (EBM) is the “conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.”1 The concept of EBM has gained worldwide popularity in the past decade and aims at encouraging clinical practice to be grounded in scientific inquiry.2 In addition to keeping physicians up to date with the most current medical knowledge, EBM also sets out to support the provision of quality care and minimisation of medical error and to facilitate a more equitable provision of services.2–4
As the quality of scientific evidence varies in the literature, examining the study design may provide clues to determine whether and to what degree the findings are valid and conclusive. The Oxford Centre for Evidence-Based Medicine has developed reviews and a “levels of evidence” scheme for classifying studies based on the epidemiologic design.5 Systematic reviews and randomised controlled trials (RCT) are rated as the highest level of evidence followed by cohort studies and case-control studies. Case series and expert opinion give the lowest level of evidence.
Before the 1995 study by Ellis et al, it has been suggested that only 10% to 20% of current medical interventions were supported by scientific evidence.6 Ellis and colleagues demonstrated that 53% of interventions in a general medical unit were based on evidence from RCT, while an additional 29% were based on reliable non-experimental or observational evidence. Although criticisms have been raised about the generalisability of this study, a later study by Michaud et al reported similar findings.7 Subsequent studies in various other specialties including general practice,8 paediatric surgery,9 dermatology,10 anaesthesia,11 and general surgery12 have evaluated the extent to which common interventions were evidence based. Together, these studies estimated the proportion of interventions that were supported by observational and/or experimental studies ranged from 71% to 97%.
To our knowledge, there has been no formal study on the proportion of evidence based interventions in ophthalmology. Furthermore, this type of study has not been performed in any specialty in Asia. The purpose of this study was to investigate the proportion of ophthalmic interventions that were evidence based in a large, regional teaching eye hospital.
PATIENTS AND METHODS
All patients presenting to the acute care unit (that is, emergency department) of Hong Kong Eye Hospital during seven consecutive sessions in July 2002 were prospectively recruited. The hospital is a tertiary ophthalmic centre serving the Kowloon East cluster with a coverage population of approximately 1.6 million out of 6.8 million in Hong Kong. Medical staff consists of four consultants, 13 specialist ophthalmologists, and 21 trainee ophthalmologists. To minimise Hawthorne's phenomenon (the potential bias associated with subjects being aware that they were being observed), none of the participating ophthalmologists involved was aware of the study during the recruitment period. After each session, all medical charts were retrieved for analysis. A predefined, standardised set of data was collected for each consultation including patients' characteristics and the rank of the ophthalmologist who performed the consultation. The primary diagnosis and primary intervention were independently determined by two of the authors (TL and VW) who were not involved in attending the consultations during the study period. The primary diagnosis was defined as the problem recorded for the consultation that was most responsible for the patient's presentation and the primary intervention as the most important attempt to manage or treat the patient in respect to the primary diagnosis.6 The primary interventions were classified as medical interventions which may include conservative or expectant treatment or the use of pharmaceutical products, or as surgical interventions if procedures included open surgery or laser treatment. Cases were excluded when a primary diagnosis could not be ascertained. Patients who refused or did not consent to the prescribed interventions were also excluded. In cases where there was disagreement in assigning diagnosis or intervention, the authors resolved these by consensus.
Literature review was performed after all the diagnosis and intervention pairs were confirmed. The same two authors performed a comprehensive search through Medline (1966 to August 2002) and the Cochrane Library, Issue 3, 2002, independently and disagreement was again settled by consensus. Only publications in English were selected. The evidence obtained was then graded into five hierarchical categories as described by Kingston et al (Table 1).12 This classification is similar to the classification by the Oxford Centre of Evidence-based Medicine, in which systematic reviews or RCT were given the highest level of evidence.5 However, expert opinion was not considered as proper scientific evidence in this classification. When there was more than one article for an intervention, the study with a higher level of evidence was chosen. For articles at the same level, the more recent article was selected.
Levels of evidence according to Kingston et al12 (n=252)
Statistical analyses were performed using spss for Windows v 10.0. Categorical outcomes were compared using two tailed Pearson's χ2 test.
RESULTS
During the study period, 274 consultations were performed by 16 different ophthalmologists (five specialist ophthalmologists and 11 trainee ophthalmologists). The mean number of consultations per ophthalmologist was 17.1 (range 9–36). Patients' mean age was 49.5 years (range 1–87 years). Males comprised 51.8% of the sample population. For the ascertainment of the primary diagnosis and the primary intervention, there was agreement between the two authors in 270 of 274 (98.5%) and in 251 of 252 (99.6%) patients respectively. Twenty two (8.0%) patients were excluded since no diagnosis were made during the consultation (Fig 1). In the remaining 252 cases, 26 (10.3%) and two (0.8%) of the interventions were supported by systematic reviews and meta-analyses, respectively (Table 2). Evidence from RCT supported the management of 80 (31.7%) patients (Table 3). Prospective and retrospective observational studies provided evidence for the interventions in 34 (13.5%) and 52 (20.6%) patients, respectively (Tables 4 and 5). The total number of interventions which were evidence based—that is, with level 1 to 5 evidence, was 194 (77.0%).12 For the remaining 58 (23.0%) diagnosis-intervention pairs, no evidence could be identified to substantiate or refute the intervention rendered (Table 6).
Interventions based on systematic reviews or meta-analyses (n=28)
Interventions based on randomised controlled trials (n=80)
Interventions based on prospective case series (n=34)
Interventions based on retrospective case series (n=52)
Interventions in which no evidence was found or evidence found to oppose the intervention (n=58)
Study organisation.
The primary intervention was classified as a medical intervention in 168 (66.7%) cases, while the remaining 84 (33.3%) cases were surgical interventions. The proportion of interventions that were evidence based was higher in the surgical group compared to the medical group but was not statistically significant (p=0.09). Among the 194 evidence based interventions, the quality of evidence was better in the surgical group with 48 of 70 (68.6%) interventions being supported by RCT or better compared to 60 of 124 (48.4%) interventions in the medical group (p=0.007).
Among all 252 interventions, 65 (25.8%) were delivered by specialist ophthalmologists and 187 (74.2%) were performed by trainee ophthalmologists. There was no difference in the proportion of evidence based interventions performed by the two groups (p=0.50). The respective proportions of medical and surgical interventions performed by each group were also similar (p=0.88). However, the quality of evidence based interventions performed by the specialist group was better compared with the trainee group, with evidence based on RCT or better supporting 36 of 52 (69.2%) interventions and 72 of 142 (50.7%) interventions in the specialist ophthalmologist and the trainee ophthalmologist groups respectively (p=0.021).
A total of 86 different interventions could be identified. The eight most common interventions accounted for the treatment of 121 (48.0%) patients (Table 7). All but one of these interventions were evidence based.
The eight most common interventions accounted for the treatment of 121 patients
DISCUSSION
Our study demonstrated that the majority of interventions (77.0%) in our regional ophthalmic unit were evidence based. The proportion of evidence based interventions in our study was comparable to findings from studies in other specialties, where around 80% of interventions were found to be grounded in scientific evidence.6–10,12 Furthermore, evidence from RCTs (level 3) or better could be identified in 108 (42.9%) patients. This was also within the range of previous studies, in which 11% to 53% of interventions were found to be based on evidence from RCT or better.6–12
Ophthalmology is a unique specialty in which a mix of both medical and surgical interventions are routinely performed. It is well known that randomised controlled trials are generally harder to conduct for surgical treatments and previous studies have shown that the proportion of evidence from RCTs was lower in surgical studies compared with studies in internal medicine.12 In our study, we found that the proportion of interventions based on RCT or better was significantly higher for surgical interventions compared to non-surgical ophthalmic interventions. This might have been because of the rapid advancement in ophthalmic surgical techniques and new instrumentations, thereby encouraging more RCTs being performed for surgical interventions. In addition, the non-surgical interventions in our study were generally prescribed for less serious and self limiting conditions (for example, subconjunctival haemorrhage, acute posterior vitreous detachment), where it may be difficult or even unnecessary, to carry out RCTs to generate evidence.
Of note, the proportion of interventions with evidence based on RCTs or better was higher in the specialist ophthalmologist group compared with the trainee ophthalmologist group. This may be due to the more experienced specialist group being more aware of the evidence available in the literature. The seniority of the specialist ophthalmologists may also allow them to be involved in the development of clinical guidelines and protocols for the unit and this process would have allowed them to be better equipped in practising EBM. Therefore, our results suggest that the more senior doctors are actually practising EBM rather than “eminence” based medicine as some have previously suggested.75
In this study, we only considered evidence to be valid if it was published in the medical literature. We did not use any textbooks or expert panels as the source of evidence since these sources have been suggested not to constitute “good” evidence and therefore should be excluded from studies of this nature.12 Other potential limitations in our study also bear mention. During the literature review, we may not have identified all the relevant literature since not all available scientific databases were searched, although Medline and the Cochrane library have often been cited as the “gold” standard for this type of study.12,76 Also, language bias may have occurred as only publications in English were assessed. The reductionist approach of having a primary diagnosis and primary intervention pairing may have led to a loss of practical reality in the actual clinical setting. Our clinical setting also made the results less generalisable since the study was conducted within the acute care unit and the proportion of evidence based intervention might differ among different subspecialty clinics as well as between other hospitals. Future research should extend this study methodology to different hospital subspecialty clinics and other centres to increase the external validity and representativeness. Lastly, we only selected the single best study on the basis of the study design for analysis to preserve clarity and simplicity. However, one of the weaknesses of EBM is the heavy emphasis on RCT where other types of evidence may be devalued, making a poor quality RCT more influential than valid non-randomised studies.77 Ideally, the quality of each individual randomised controlled trials should be examined and any conflicting results from various randomised controlled trial should be resolved through systematic reviews or meta-analyses.
In summary, our study has demonstrated that most of the current ophthalmic interventions in the acute care unit of the Hong Kong Eye Hospital were evidence based. Routine audits similar to our study may be beneficial since they can provide quality assurance to the provision of optimal patient care.7 Through this study, we have highlighted areas in which there was limited evidence and have identified areas of deficiency in the literature for future research.
Acknowledgments
We would like to thank Drs CC Chi and NM Lam for their support for the study and Mr John Cheng for retrieving all the medical charts.
Conflicts of interest (financial or otherwise): None.
Financial support: None.