Objectives This research project aims to determine the potential differential impact of two curricular approaches to teaching evidence-based medicine (EBM) on student performance on an EBM assignment administered during the first year of clerkship. A meaningful result would be any statistically significant difference in scores on the assignment given to measure student performance.

Design In order to assess and compare student learning under the different curricula, the principal investigator and a team of five faculty members blinded to assignment date and other possibly identifying details used a modified version of the previously validated Fresno rubric to retrospectively grade 3 years’ worth of EBM assignments given to students in clerkship rotations 1–3 (n=481) during the Internal Medicine clerkship. Specifically, EBM performance in three separate student cohorts was examined.

Setting The study took place at a large Midwestern medical school with nine campuses across the state of Indiana.

Participants Study participants were 481 students who attended the medical school and completed the Internal Medicine clerkship between 2017 and 2019.

Interventions Prior to the inception of this study, our institution had been teaching EBM within a discrete 2-month time period during medical students’ first year. During a large-scale curricular overhaul, the approach to teaching EBM was changed to a more scaffolded, integrated approach with sessions being taught over the course of 2 years. In this study, we assess the differential impact of these two approaches to teaching EBM in the first 2 years of medical school.

Main outcome measures We used clerkship-level EBM assignment grades to determine whether there was a difference in performance between those students who experienced the old versus the new instructional model. Clerkship EBM assignments given to the students used identical questions each year in order to have a valid basis for comparison. Additionally, we analysed average student grades across the school on the EBM portion of step 1.

Results Four hundred and eighty-one assignments were graded. Mean scores were compared for individual questions and cumulative scores using a one-way Welch Analysis of Variance test. Overall, students performed 0.99 of a point better on the assignment from year 1 (Y1), prior to EBM curriculum integration, to year 3 (Y3), subsequent to EBM integration (p≤0.001). Statistically significant improvement was seen on questions measuring students’ ability to formulate a clinical question and critically appraise medical evidence. Additionally, on the United States Medical Licensing Examination (USMLE) step 1, we found that student scores on the EBM portion of the examination improved from Y1 to Y3.

Conclusions Results of this study suggest that taking a scaffolded, curriculum-integrated approach to EBM instruction during the preclinical years increases, or at the very least does not lessen, student retention of and ability to apply EBM concepts to patient care. Although it is difficult to fully attribute students’ retention and application of EBM concepts to the adoption of a curricular model focused on scaffolding and integration, the results of this study show that there are value-added educational effects to teaching EBM in this new format. Overall, this study provides a foundation for new research and practice seeking to improve EBM instruction.

Trial registration number IRB approval (Protocol number 1907054875) was obtained for this study.