Dear Editor,

We agree with doctors Evans and Hadler that our clinical decision rule[1] should be validated in another setting and that it should be shown to cause more good than harm before it could be widely used. Our study was a first step in the process of developing a predictive tool for the occupational outcome of back pain. In a next phase of development, not only should the rule be validated in a new group of subjects, but it should also be compared with the judgment of clinicians; a decision tool that is no better than the clinician’s own judgment is of no interest. If the tool adds significantly to the clinician’s judgment, then the impacts of its systematic use could be further studied.[2]

Evans and Hadler are also right to be cautious about the performance of the rule. However, it includes a subset of predictors selected among more than 100 variables, several of which measured at baseline, 6 weeks and 12 weeks. We are confident to have identified the best predictors. As we have recognized ourselves in the paper, the rule is however far from perfect. But in its appreciation, it is also very important to understand the limitations of the methods that are currently available to quantify the performance of our model: for example, because classical measures of concurrent validity ask for dichotomous outcomes, we had to regroup the outcome categories and this tended to dilute the performance of the rule. The alternative approach of comparing each group with the “Success” group would have resulted in much better indices of predictive validity, but they would however have been artificial, since the physician always works from the complete pool of patients. Again, the real test will come from comparisons with the clinicians’ judgment.

We do not believe that “return to work in good health” is an “unstable psychosocial construct”. It was clearly defined and is certainly clinically important. In many instances, this is the kind of outcomes on which the primary care physician is asked to give a prognosis. It is, obviously, not easy to predict. Should we then abandon our efforts because the task is too difficult?

The interpretation Evans and Hadler seem to give to our results, as to the association of the patient’s “own premonition” with their future status, is clearly flawed. In a predictive statistical approach as the one we have used, there is no intent to identify causal associations and there is no adjustment for confounders.[3] All that can be said about this association is that this variable is one of the most useful to predict the outcome. Why not use it then? Others have also found patients’ recovery expectations to be strongly linked with different health outcomes.[4-7]

Finally, interpreting Figure 2 of the paper to say that the outcome at 12 weeks had a predictive accuracy of >90% is, unfortunately, too simple: at 12 weeks, only 52% of subjects were in the “Success” category. Forty-eight percent were in the other categories, and there was a difference of 7.3% in “Failures” between 12 weeks and 2 years. Given that the largest part (>80%) of the costs of back pain comes from a small minority (<10%) of patients, this is not trivial.

References

1. Dionne CE, Bourbonnais R, Frémont P, Rossignol M, Stock SR, Larocque I. A clinical return-to-work rule for patients with back pain. Cmaj 2005;172(12):1559-1567.

2. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. Jama 1997;277(6):488-494.

3. Kleinbaum DG, Kupper LL, Muller KE. Applied regression analysis and other multivariable methods. 2 ed. Boston: PWS-KENT Publishing Company; 1988. Chapter 11.

4. Cole DC, Mondloch MV, Hogg-Johnson S. Listening to injured workers: how recovery expectations predict outcomes--a prospective study. Cmaj 2002;166(6):749-754.

5. Kalauokalani D, Cherkin DC, Sherman KJ, Koepsell TD, Deyo RA. Lessons from a trial of acupuncture and massage for low back pain: patient expectations and treatment effects. Spine 2001;26(13):1418-1424.

6. Kiyak HA, Vitaliano PP, Crinean J. Patients' expectations as predictors of orthognathic surgery outcomes. Health Psychol 1988;7(3):251- 268.

7. Mondloch MV, Cole DC, Frank JW. Does how you do depend on how you think you'll do? A systematic review of the evidence for a relation between patients' recovery expectations and health outcomes. Cmaj 2001;165(2):174-179.

When RCTs are consistent across a variety of populations and settings, we should feel more secure about the applicability of the intervention. If it works in low risk and high risk, young and old, East and West, it will probably work in my patient. However, as Puliyel and Sreenivas point out, RCTs don't always agree, and sometimes diverge widely. When that happens, we would like to know why. It could be any of the PICO elements: the populations studied, the way the intervention is delivered (dose, vehicle, route, timing, etc), the comparator and background treatments, or when or how the outcomes were measured.[1] Or it can be that the PICOs are the same, but that some of the trials are flawed (poor randomisation, poor followup, non-blinding, etc) and some not, leading to confounding by trial quality. Systematic (and unsystematic!) reviews should look for such differences, and if they occur use them as an opportunity to learn more about when and why a treatment works or does not. However, that requires considerable care to separate out the possible true and artefactual causes of apparent disagreement between studies.

The Editors

References

[1] Glasziou PP, Sanders SL. Investigating causes of heterogeneity in systematic reviews. Stat Med. 2002;21:1503-11.

I see many results in which potassium falls (a little) above the normal range.[1] I was wondering at what level I should be concerned about sudden death being the presenting symptom.

I reflected that the context in which the result was found affects my decision making.

A routine test which identifies a raised value, is less alarming than a test which is done for a reason, such as the presence of symptoms or perhaps because of a medication change.

This is analagous to the significance of a symptom or sign in the context of a focussed query versus routine enquiry (or screening).

One way of looking at it is in terms of the test characteristics of a diagnostic test versus a screening test. A screening test may have a higher cut point to increase specificity and reduce false positives. A diagnostic test may have a lower cut point to increase sensitivity and reduce false negatives.

Alternatively, looking at it as a Bayesian probability calculation, when doing the test for a reason (eliciting a symptom or sign for a reason) the prior probability (of a problem) is greater and hence one is influenced more even if the test (or symptom/sign) has a relatively small positive LR.

This is a very roundabout way of saying that if in doing routine annual bloods on a patient with hypertension treated with an ACE, I discovered a K of 6 say, I would perhaps feel this was not a cause for urgency. Conversely if someone presented with weakness and palpitations having recently commenced an ACE for diabetic nephropathy I might judge the same value of K to be more serious.

On the basis of this I would be interested to know what type of patients were included in the prognostic studies identified. Were they a broad selection of the population, such as seen in primary care, or were they perhaps patients more at risk of hyperkalemia, and its consequences, such as patients with diabetic nephropathy? Similarly, what was the context in which ECGs were related to potassium levels?

Reference

1. de Palma J R, Glasziou P. The first symptom of hyperkalaemia is death. Evid Based Med 2004; 9: 134-135