Dear Editor,
We would like to thank DeFrank and Brewer for their interest in our
recently published paper: Long-Term Psychosocial Consequences of False-
Positive Screening Mammography.(1)
In the Methods section DeFrank and Brewer write: "Brodersen and
colleagues conducted a study of 454 adult women in Denmark screened in the
same time period who had normal mammography screening results, false-
positives or breast cancer diagnoses." We (Brodersen & Siersma)
included 454 adult women in Denmark who had abnormal mammography screening
results (false-positives or breast cancer diagnoses) and 864 women with
normal screening results - all 1318 screened in the same time period.
In the Methods section DeFrank and Brewer also state that the
questionnaire we used: the Consequences Of Screening in Breast Cancer (COS
-BC), is a revised version of the question: the Psychosocial Consequences
Questionnaire (PCQ). This is not the case. Together with the developer of
the original Australian version of the PCQ - deceased Professor Jill
Cockburn - we published a systematic reviewed on "The adequacy of
measurement of short and long-term consequences of false-positive
screening mammography".(2) Part of our conclusion was: "The PCQ is an
adequate questionnaire for measuring short-term consequences, and the PCQ
is preferable to other measures because of its higher sensitivity.
However, there is little evidence that the PCQ is able to adequately
detect all long-term consequences of screening mammography. Given the
inadequacy of the measurement instruments used, any current conclusions
about the long-term consequences of false-positive results of screening
mammography must remain tentative".
In a qualitative study including six focus interviews with women having
false-positive screening mammography we found that the PCQ includes
ambiguous items, does not cover all psychosocial consequences of false-
positive screening mammography and three items from the PCQ were deleted
because they were judged by interviewees to be irrelevant.(3) We also
found that the women's experiences in the critical period from abnormal
screening mammography until final false-positive diagnosis differed
entirely from their experiences after the final diagnosis.(3) Therefore,
fifteen new items were generated to cover the negative psychosocial
consequences of abnormal and false-positive screening mammography
comprehensively.(3) Five new items were produced that concerned the
consequences of screening mammography during the period after being
declared ''free from'' suspicion of cancer.(3) Response options for the
positive items were changed to allow responses in both positive and
negative directions. Our conclusion was: "Because of the major changes to
both parts of the PCQ the measure derived from this study should be
regarded as a new questionnaire with two parts: Consequences Of Screening
in Breast Cancer (COS-BC). Part II focuses on the long-term consequences
of a false-positive screening mammography".(3)
Because we had developed a new questionnaire (based on Jill Cockburn's
work) we conducted two psychometric studies using the Item Response Theory
Partial Credit Rasch model for polytomous items plus Classical Test Theory
to validate the COS-BC.(4;5) Therefore, it is slightly confusing that
DeFrank and Brewer calls COS-BC for PCQr when it actually is a new
measure.
Due to this extensive qualitative and statistical psychometrical work we
have provided robust high quality evidence that the subscales of the COS-
BC measure each distinct constructs that are unidimensional. It is
therefore wrong when DeFrank and Brewer in the Commentary section write
that some "subscales mix multiple constructs". Furthermore, we have
revealed that the PCQ lacks content validity and encompass ambiguous
items. Because the COS-BC has high content validity and adequate
psychometric properties we can conclude that the COS-BC and the PCQ do not
measure the same thing. It is therefore not appropriate that DeFrank and
Brewer compare the scores of the PCQ with the scores of the COS-BC.
Furthermore, standard methods, like the correlation coefficients are
different from our linear regression methods, and do not adjust for the
differential dropout. As stated in our method section: "We used
generalized estimating equations methods to account for repeated
measurement on the same individual. To adjust for possible bias resulting
from differential dropout from the study, the scores available at each
follow-up time point were weighted by the inverse of an estimate of the
probability of this score being observed at that time point".(1) Those
people that are worse off have the tendency to drop out in surveys. In our
case, women diagnosed with breast cancer and the women that are most
affected by the false-positive screening result could have a higher
tendency to drop out; analysing only the subjects that did not drop out
will then artificially decrease effects. By using inverse probability
weighing and correcting the confidence intervals accordingly using
generalized estimating equations we have adjusted for this differential
drop out. The relatively small effect sizes found and commented on by
DeFrank and Brewer using our data may well be an artefact of not adjusting
for dropout.
In the Commentary section DeFrank and Brewer also write that the COS
-BC "has a subscale for breast self-examination, a behaviour that many
guidelines now recommended against". However, the subscale has not
anything to do with breast self-examination recommendations. In our
qualitative study we found that women with false-positive screening
mammography examined their breasts with their hands and examined their
breasts in a mirror as a psychosocial consequence of the false-positive
results and not because of any recommendation.
In the Commentary section DeFrank and Brewer also write: "The second
half of the PCQr assesses how patients...". Most of these women are not
patients. The main part of the women in our survey had a normal or a false
-positive screening result and besides that they were healthy. In the same
sentence DeFrank and Brewer continue:"....think the testing affected them
and uses an uncommon scoring system." We do not find the scoring system as
a limitation since the response categories and scoring system for part 2
of the COS-BC were developed during the previous mentioned focus group
interviews.(3) Furthermore, part2 of the COS-BC was validated using the
empirical knowledge from these interviews,(3) the theory of cognitive
dissonance, the theory of sense of coherence, and as mentioned above Item
Response Theory Partial Credit Rasch model for polytomous items. (4;6)
In the Commentary section DeFrank and Brewer lastly write: "Finally,
the absence of analyses that demonstrate whether changes over time
differed by test results, the internal consistency of measures and their
reliability over time. Study findings would be strengthened by the
inclusion of outcomes assessed before screening or learning of test
results." Test for interaction between time and the three screening groups
could have been added to the tables, but such test may not have been very
informative because of the large differences between the trajectories in
the start of follow-up: most tests for interaction were highly
significant. Where items are shown to fit a Rasch model the measure can be
shown to posses criterion-related construct validity,(7) to be objective
(invariant),(8) sufficient(9) and, therefore, also reliable.(10)
Furthermore, we have provided evidence that the items in the subscales do
not posses differential item functioning in relation to time.
References
(1) Brodersen J, Siersma VD. Long-Term Psychosocial Consequences of
False-Positive Screening Mammography. The Annals of Family Medicine 2013;11(2):106-15.
(2) Brodersen J, Thorsen H, Cockburn J. The adequacy of measurement
of short and long-term consequences of false-positive screening
mammography. J Med Screen 2004 Mar 1;11(1):39-44.
(3) Brodersen J, Thorsen H. Consequences Of Screening in Breast
Cancer (COS-BC): development of a questionnaire. Scand J Prim Health Care
20081;26(4):251-6.
(4) Brodersen J. Measuring psychosocial consequences of false-
positive screening results - breast cancer as an example. Department of
General Practice, Institute of Public Health, Faculty of Health Sciences,
University of Copenhagen: M?nedsskrift for Praktisk L?gegerning,
Copenhagen. ISBN: 87-88638-36-7; 2006.
(5) Brodersen J, Thorsen H, Kreiner S. Validation of a condition-
specific measure for women having an abnormal screening mammography. Value
in Health 2007;10(4):294-304.
(6) Brodersen J, Thorsen H, Kreiner S. Consequences Of Screening in
Lung Cancer: Development and Dimensionality of a Questionnaire. Value in
Health 2010;13(5):601-12.
(7) Rosenbaum PR. Criterion-related construct validity. Psychometrika
1989 ;54(4):625-33.
(8) Rasch G. An Informal Report on a Theory of Objectivity in
Comparisons. In: Van der Kamp LJTh, Vlek CAJ, editors. Psychological
Measurement Theory.Leyden: University of Leyden; 1967:1-19.
(9) Andersen EB. Sufficient Statistics and Latent Trait Models.
Psychometrika 1977;42:69-81.
(10) Bartholomew DJ. The Statistical Approach to Social measurement.
San Diego: Academic Press; 1996.
Conflict of Interest:
None declared
Dear Editor,
We would like to thank DeFrank and Brewer for their interest in our recently published paper: Long-Term Psychosocial Consequences of False- Positive Screening Mammography.(1)
In the Methods section DeFrank and Brewer write: "Brodersen and colleagues conducted a study of 454 adult women in Denmark screened in the same time period who had normal mammography screening results, false- positives or breast cancer diagnoses." We (Brodersen & Siersma) included 454 adult women in Denmark who had abnormal mammography screening results (false-positives or breast cancer diagnoses) and 864 women with normal screening results - all 1318 screened in the same time period.
In the Methods section DeFrank and Brewer also state that the questionnaire we used: the Consequences Of Screening in Breast Cancer (COS -BC), is a revised version of the question: the Psychosocial Consequences Questionnaire (PCQ). This is not the case. Together with the developer of the original Australian version of the PCQ - deceased Professor Jill Cockburn - we published a systematic reviewed on "The adequacy of measurement of short and long-term consequences of false-positive screening mammography".(2) Part of our conclusion was: "The PCQ is an adequate questionnaire for measuring short-term consequences, and the PCQ is preferable to other measures because of its higher sensitivity. However, there is little evidence that the PCQ is able to adequately detect all long-term consequences of screening mammography. Given the inadequacy of the measurement instruments used, any current conclusions about the long-term consequences of false-positive results of screening mammography must remain tentative". In a qualitative study including six focus interviews with women having false-positive screening mammography we found that the PCQ includes ambiguous items, does not cover all psychosocial consequences of false- positive screening mammography and three items from the PCQ were deleted because they were judged by interviewees to be irrelevant.(3) We also found that the women's experiences in the critical period from abnormal screening mammography until final false-positive diagnosis differed entirely from their experiences after the final diagnosis.(3) Therefore, fifteen new items were generated to cover the negative psychosocial consequences of abnormal and false-positive screening mammography comprehensively.(3) Five new items were produced that concerned the consequences of screening mammography during the period after being declared ''free from'' suspicion of cancer.(3) Response options for the positive items were changed to allow responses in both positive and negative directions. Our conclusion was: "Because of the major changes to both parts of the PCQ the measure derived from this study should be regarded as a new questionnaire with two parts: Consequences Of Screening in Breast Cancer (COS-BC). Part II focuses on the long-term consequences of a false-positive screening mammography".(3) Because we had developed a new questionnaire (based on Jill Cockburn's work) we conducted two psychometric studies using the Item Response Theory Partial Credit Rasch model for polytomous items plus Classical Test Theory to validate the COS-BC.(4;5) Therefore, it is slightly confusing that DeFrank and Brewer calls COS-BC for PCQr when it actually is a new measure. Due to this extensive qualitative and statistical psychometrical work we have provided robust high quality evidence that the subscales of the COS- BC measure each distinct constructs that are unidimensional. It is therefore wrong when DeFrank and Brewer in the Commentary section write that some "subscales mix multiple constructs". Furthermore, we have revealed that the PCQ lacks content validity and encompass ambiguous items. Because the COS-BC has high content validity and adequate psychometric properties we can conclude that the COS-BC and the PCQ do not measure the same thing. It is therefore not appropriate that DeFrank and Brewer compare the scores of the PCQ with the scores of the COS-BC. Furthermore, standard methods, like the correlation coefficients are different from our linear regression methods, and do not adjust for the differential dropout. As stated in our method section: "We used generalized estimating equations methods to account for repeated measurement on the same individual. To adjust for possible bias resulting from differential dropout from the study, the scores available at each follow-up time point were weighted by the inverse of an estimate of the probability of this score being observed at that time point".(1) Those people that are worse off have the tendency to drop out in surveys. In our case, women diagnosed with breast cancer and the women that are most affected by the false-positive screening result could have a higher tendency to drop out; analysing only the subjects that did not drop out will then artificially decrease effects. By using inverse probability weighing and correcting the confidence intervals accordingly using generalized estimating equations we have adjusted for this differential drop out. The relatively small effect sizes found and commented on by DeFrank and Brewer using our data may well be an artefact of not adjusting for dropout.
In the Commentary section DeFrank and Brewer also write that the COS -BC "has a subscale for breast self-examination, a behaviour that many guidelines now recommended against". However, the subscale has not anything to do with breast self-examination recommendations. In our qualitative study we found that women with false-positive screening mammography examined their breasts with their hands and examined their breasts in a mirror as a psychosocial consequence of the false-positive results and not because of any recommendation.
In the Commentary section DeFrank and Brewer also write: "The second half of the PCQr assesses how patients...". Most of these women are not patients. The main part of the women in our survey had a normal or a false -positive screening result and besides that they were healthy. In the same sentence DeFrank and Brewer continue:"....think the testing affected them and uses an uncommon scoring system." We do not find the scoring system as a limitation since the response categories and scoring system for part 2 of the COS-BC were developed during the previous mentioned focus group interviews.(3) Furthermore, part2 of the COS-BC was validated using the empirical knowledge from these interviews,(3) the theory of cognitive dissonance, the theory of sense of coherence, and as mentioned above Item Response Theory Partial Credit Rasch model for polytomous items. (4;6)
In the Commentary section DeFrank and Brewer lastly write: "Finally, the absence of analyses that demonstrate whether changes over time differed by test results, the internal consistency of measures and their reliability over time. Study findings would be strengthened by the inclusion of outcomes assessed before screening or learning of test results." Test for interaction between time and the three screening groups could have been added to the tables, but such test may not have been very informative because of the large differences between the trajectories in the start of follow-up: most tests for interaction were highly significant. Where items are shown to fit a Rasch model the measure can be shown to posses criterion-related construct validity,(7) to be objective (invariant),(8) sufficient(9) and, therefore, also reliable.(10) Furthermore, we have provided evidence that the items in the subscales do not posses differential item functioning in relation to time.
References
(1) Brodersen J, Siersma VD. Long-Term Psychosocial Consequences of False-Positive Screening Mammography. The Annals of Family Medicine 2013;11(2):106-15.
(2) Brodersen J, Thorsen H, Cockburn J. The adequacy of measurement of short and long-term consequences of false-positive screening mammography. J Med Screen 2004 Mar 1;11(1):39-44.
(3) Brodersen J, Thorsen H. Consequences Of Screening in Breast Cancer (COS-BC): development of a questionnaire. Scand J Prim Health Care 20081;26(4):251-6.
(4) Brodersen J. Measuring psychosocial consequences of false- positive screening results - breast cancer as an example. Department of General Practice, Institute of Public Health, Faculty of Health Sciences, University of Copenhagen: M?nedsskrift for Praktisk L?gegerning, Copenhagen. ISBN: 87-88638-36-7; 2006.
(5) Brodersen J, Thorsen H, Kreiner S. Validation of a condition- specific measure for women having an abnormal screening mammography. Value in Health 2007;10(4):294-304.
(6) Brodersen J, Thorsen H, Kreiner S. Consequences Of Screening in Lung Cancer: Development and Dimensionality of a Questionnaire. Value in Health 2010;13(5):601-12.
(7) Rosenbaum PR. Criterion-related construct validity. Psychometrika 1989 ;54(4):625-33.
(8) Rasch G. An Informal Report on a Theory of Objectivity in Comparisons. In: Van der Kamp LJTh, Vlek CAJ, editors. Psychological Measurement Theory.Leyden: University of Leyden; 1967:1-19.
(9) Andersen EB. Sufficient Statistics and Latent Trait Models. Psychometrika 1977;42:69-81.
(10) Bartholomew DJ. The Statistical Approach to Social measurement. San Diego: Academic Press; 1996.
Conflict of Interest:
None declared