Validity and reliability of the Spanish version of the ACE tool for assessing competencies in evidence-based medicine in medical students

Objective: To translate and culturally adapt the tool 'Assessing Competency in evidence-based medicine (EBM)' (ACE) to Spanish and to implement it in a cohort of medical students for the evaluation of the instrument's psychometric properties.

Design: Bilingual translators produced a translation and backtranslation of the original instrument, with interim consensus in each stage with oversight and input by a group of experts. We then performed cognitive interviews to adapt the wording of the tool culturally. Finally, we implemented the final version in a cohort of medical students on a virtual general practice course with EBM modules.

Setting: Medical School in Buenos Aires, Argentina. Due to restrictions to in-person teaching during the COVID-19 pandemic, we conducted this study in the context of virtual learning.

Participants: We included 125 fourth and fifth-year medical students.

Main outcome measures: We measured internal consistency with the Kuder-Richardson coefficient (>0.6 as a threshold for reliability) and construct validity through a Pearson's correlation between the examinations carried out with the translated instrument and the results of the regular examinations of EBM in the same students (expected values of at least 0.3 to 0.7). We also compared the total score of the instrument of fifth-year students to fourth-year students.

Results: As for measurements for internal consistency, the coefficient Kuder-Richardson resulted in a value of 0.268, below our prespecified threshold. For construct validity, the Pearson correlation between the sum of the items and regular examinations was 0.139, also below our prespecified threshold. However, fifth-year students averaged 0.94 points more than fourth-year students (95% CI 0.24 more to 1.65 more).

Conclusion: The translated and cross-culturally adapted version of the ACE tool into Spanish had low reliability and validity in an MBE course taught and evaluated in a virtual environment.

Trial registration: Not applicable.