Introduction New medical instruments and tests are constantly being developed with the aim of providing cheaper, non-invasive, more convenient and safe methods. It is imperative that all medical instruments are accurate and precise. A failure of medical instruments to be accurate may lead to diagnosis errors with potentially affect patient’s management. Agreement is one of parameters for testing the quality of instrument and signifies the accuracy of that certain instrument. The Bland-Altman method is the most popular method used to assess the agreement of medical instruments. The main concern about this method is the presence of proportional bias. The aim of this study is to propose a new method (Agreement Model) to complement the Bland-Altman method in the analysis of agreement.

Methods A cross-sectional study was conducted to collect data in two population settings; an institutional and community setting. Data were collected for blood glucose level, systolic blood pressure, diastolic blood pressure, body weight, and peak expiratory flow rate to assess methods used in agreement studies. Evaluations of statistical methods were carried out on the original clinical data and simulated data. The agreement evaluation involved a comparison of the most commonly used method (Bland-Altman Limits of Agreement), and a new proposed method (Agreement Model). Both Agreement Model and Bland-Altman analysis were performed to identify various pattern of simulated bias: (a) Constant systematic error, (b) Inconsistent error, and (c) Different proportion of bias.

Results The Bland-Altman method gives accurate predictions of disagreement for all 45 simulated datasets. The Agreement Model gives accurate predictions of agreement for all 55 clinical datasets. Both statistical methods provide a consistent prediction of agreement. The Agreement Model successfully predicts the agreement of all clinical datasets. However, the Bland-Altman Limits of Agreement shows disagreement for all instruments. The main problem with the Bland-Altman method is the overestimation of bias in the limits of agreement. When the Bland-Altman analysis resulted in the agreement of instruments, it is very likely that the instruments are truly in agreement. The Agreement Model is less likely to give incorrect prediction of disagreement. Thus this method is most suitable to confirm disagreement.

Conclusion The tendency of the Bland-Altman method to overestimate bias makes this method is most suitable to confirm agreement. However, it is best to be complemented by the Agreement Model to prevent overestimation of bias.