Dear Editor,

In response to the Editor's invitation calling for short items on EBM related issues, we would like you to consider this statistical problem for possible publication.

The double-blind Randomised Controlled Trial (RCT) is the basis of good evidence based medicine because it eliminates problems of bias and confounding. However systematic reviews show different RCTs arriving at diametrically opposite conclusions. The reason for this is that the samples for the RCTs are drawn from different populations and it reflects the truth in those various populations. This matter is often overlooked when met-analysis is done. When RCTs are aggregated in a meta-analysis we have to aggregate the populations they represent – not the sample sizes. Large samples from small populations will get undue weightage otherwise. Meta-analysis as done presently can be misleading and unreliable.

Dear Editor,

We agree with doctors Evans and Hadler that our clinical decision rule[1] should be validated in another setting and that it should be shown to cause more good than harm before it could be widely used. Our study was a first step in the process of developing a predictive tool for the occupational outcome of back pain. In a next phase of development, not only should the rule be validated in a new group of subjects, but it should also be compared with the judgment of clinicians; a decision tool that is no better than the clinician’s own judgment is of no interest. If the tool adds significantly to the clinician’s judgment, then the impacts of its systematic use could be further studied.[2]

Evans and Hadler are also right to be cautious about the performance of the rule. However, it includes a subset of predictors selected among more than 100 variables, several of which measured at baseline, 6 weeks and 12 weeks. We are confident to have identified the best predictors. As we have recognized ourselves in the paper, the rule is however far from perfect. But in its appreciation, it is also very important to understand the limitations of the methods that are currently available to quantify the performance of our model: for example, because classical measures of concurrent validity ask for dichotomous outcomes, we had to regroup the outcome categories and this tended to dilute the performance of the rule. The alternative approach of comparing each group with the “Success” group would have resulted in much better indices of predictive validity, but they would however have been artificial, since the physician always works from the complete pool of patients. Again, the real test will come from comparisons with the clinicians’ judgment.

We do not believe that “return to work in good health” is an “unstable psychosocial construct”. It was clearly defined and is certainly clinically important. In many instances, this is the kind of outcomes on which the primary care physician is asked to give a prognosis. It is, obviously, not easy to predict. Should we then abandon our efforts because the task is too difficult?

The interpretation Evans and Hadler seem to give to our results, as to the association of the patient’s “own premonition” with their future status, is clearly flawed. In a predictive statistical approach as the one we have used, there is no intent to identify causal associations and there is no adjustment for confounders.[3] All that can be said about this association is that this variable is one of the most useful to predict the outcome. Why not use it then? Others have also found patients’ recovery expectations to be strongly linked with different health outcomes.[4-7]

Finally, interpreting Figure 2 of the paper to say that the outcome at 12 weeks had a predictive accuracy of >90% is, unfortunately, too simple: at 12 weeks, only 52% of subjects were in the “Success” category. Forty-eight percent were in the other categories, and there was a difference of 7.3% in “Failures” between 12 weeks and 2 years. Given that the largest part (>80%) of the costs of back pain comes from a small minority (<10%) of patients, this is not trivial.

References

1. Dionne CE, Bourbonnais R, Frémont P, Rossignol M, Stock SR, Larocque I. A clinical return-to-work rule for patients with back pain. Cmaj 2005;172(12):1559-1567.

2. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. Jama 1997;277(6):488-494.

3. Kleinbaum DG, Kupper LL, Muller KE. Applied regression analysis and other multivariable methods. 2 ed. Boston: PWS-KENT Publishing Company; 1988. Chapter 11.

4. Cole DC, Mondloch MV, Hogg-Johnson S. Listening to injured workers: how recovery expectations predict outcomes--a prospective study. Cmaj 2002;166(6):749-754.

5. Kalauokalani D, Cherkin DC, Sherman KJ, Koepsell TD, Deyo RA. Lessons from a trial of acupuncture and massage for low back pain: patient expectations and treatment effects. Spine 2001;26(13):1418-1424.

6. Kiyak HA, Vitaliano PP, Crinean J. Patients' expectations as predictors of orthognathic surgery outcomes. Health Psychol 1988;7(3):251- 268.

7. Mondloch MV, Cole DC, Frank JW. Does how you do depend on how you think you'll do? A systematic review of the evidence for a relation between patients' recovery expectations and health outcomes. Cmaj 2001;165(2):174-179.

This is indeed an important finding on several levels, yet it remains difficult in translating this into clinical practice. I have found myself even more ambivalent about suggesting SMBG to patients reasonably well-controlled on oral anti-diabetes medications.

In an effort to translate these findings, I propose the following practical suggestions.

1. For patients struggling to comply with health care recommendations, we now know that SMBG does not need to be as high a priority in our counseling.

2. For patients struggling to afford all the components necessary to actually perform SMBG (e.g., glucometer, strips, lancets), this now becomes an area of potential cost savings. Freeing up their limited discretionary income might allow them to afford a nutrition consultation, important medications or other expensive interventions with better evidentiary support.

3. For patients fully complying with SMBG, this might be a time to allow them to check their BG less frequently (e.g., Tuesday and Friday, fasting and post-prandial, and prn for signs and symptoms of hypoglycemia and/or hyperglycemia) instead of at their current rate.

Finally, since there is potential to have a critical study in the future overturn these recommendations, I am choosing to use caution both in how I explain this study and in discontinuing anyone from SMBG.

It's quite a while since I read EBM but I received an eTOC today and was pleased to discover I could read it via my NHS Athens login. I couple of abstracts caught my including this one, because as a GP, LUTS and concern about PSA are relatively commonly seen. The commentary seemed to have a lot of detail about prostate disease, which was interesting but perhaps not entirely relevant to the paper it was commenting upon. I was very surprised to see RRRs and no ARRs in the summary of the results, surely a mainstay of EBM is to provide both otherwise the perception of the results is almost inevitably biased. I turned to the abstract of the article (after perusing the full text article re finasteride), unfortunaly the full text was not available. What I discovered from the abstract was that the ARR seems to be about 5% for all biopsy detected prostate cancer (about 25% placebo and 20% dutasteride) which I guess is an NNT of 20 for 4 yrs, to prevent a positive biopsy, so debatable to what extent this is a patient oriented outcome. What was puzzling is that the abstract also mentions that Gleason 7-10 biopsies were lower in years 1-4 of the study and the Gleason 8-10 biopsies were higher in years 3-4 (12 in dutasteride, 1 in placebo). I am not really clear why the 2 different intervals and why the 2 different Gleason ranges, but I would be concerned about the apparently higher incidence of high grade biopsies with worst prognosis at 3 - 4 years. The net result is I am not clear whether dutasteride has any patient benefit or possibly even makes things worse. It would have been nice if the commentary could have addressed this question in more detail, because the tone of commentary is that dutasteride is beneficial and I am unclear that the data shows this.

David Bossano

Conflict of Interest:

None declared