Article Text
PDF
Abstract
Postgraduate training is moving from face-to-face workshops or courses to online learning to help increase access to knowledge, expertise and skills, and save the cost of face-to-face training. However, moving from face-to-face to online learning for many of us academics is intimidating, and appears even more difficult without the help of a team of technologists. In this paper, we describe our approach, our experiences and the lessons we learnt from converting a Primer in Systematic Reviews face-to-face workshop to a 6-week online course designed for healthcare professionals in Africa. We learnt that the team needs a balance of skills and experience, including technical know-how and content knowledge; that the learning strategies needed to achieve the learning objectives must match the content delivery. The online approach should result in both building knowledge and developing skills, and include interactive and participatory approaches. Finally, the design and delivery needs to keep in mind the limited and expensive internet access in some resource-poor settings in Africa.
- evidence-based practice
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Systematic reviews (SRs) are necessary to make evidence-informed decisions and thus have an important role to play in informing policy and practice. Evidence-based healthcare (EBHC) and the use of SRs are gaining momentum in the African region,1–3 and academic programmes promoting the use of research to inform policy and practice are also on the rise. However, the majority of teaching is still delivered face-to-face through academic programmes, especially in Africa.4 Even though pure online learning may not be equivalent to face-to-face learning for improving EBHC knowledge and skills, it has various benefits that are attractive to healthcare providers and policy makers,5 who might not have the time and resources to sit in a multiday face-to-face workshop.
Furthermore, tighter budgets and lower enrolment rates are changing traditional higher education, where academics and EBHC teachers are under increasing pressure to develop online course material.6 For those with limited technical expertise, the development or transition to an online course is often seen as an intimidating and formidable challenge. Simply migrating face-to-face content into an online space is a common mistake as this approach often leads to failure. In the context of supporting EBHC thinking and decision making in Africa, with a specific emphasis on SRs, we undertook the challenge of transitioning a 4-day, face-to-face Primer in SR workshop into a 6-week online course.
The original face-to-face Primer workshop aims to increase the capacity of health staff, researchers and policy-makers to use SRs by equipping participants with the knowledge and skills to find, understand, appraise and use SRs of the effects of interventions. The workshop has been implemented and evaluated in various settings in Africa for 8 years (figure 1). It was first held in Tanzania in 2012, with a group of malaria researchers at Ifakara Health Institute. Other workshops have included, among others, tuberculosis (TB) specialists in Chennai in 2015, neglected tropical diseases policy and programme staff in Ghana and Cameroon in 2016/2017, public health policy specialists working for the Department for International Development in the UK in 2017, and researchers at the KEMRI Wellcome Trust in Kilifi, Kenya in 2018.
Geographical distribution of face-to-face Primer in Africa and distribution of online only participants (2014–2019).
The face-to-face Primer’s format is participant—and interaction-centric, where presentation content and examples of Cochrane reviews for group work are tailored to the target audience. Presentations are punctuated by interactive and hands-on exercises using small group discussions. During the 4 days the participants work through various learning objectives including understanding how to phrase questions, search the literature, assess risk of bias (ROB) of included studies, assess ROB of reviews and how to interpret results and Grading of Recommendations Assessment,Development and Evaluation (GRADE) Summary of Findings tables. We encourage feedback from peers and include time for online engagement before and after the workshop. These are all key aspects to consider when teaching EBHC.7 For example, in Kilifi, Kenya, the majority of participants were infectious disease researchers. We thus used, infectious disease reviews in malaria,8 immunisation9 and TB10. Where each group was tasked with developing their own review question and searching for evidence, determining ROB, interpreting results and making judgements on certainty of evidence and GRADE linked to their example review.
However, despite the Primer’s reach in Africa, offering a face-to-face workshop in countries where the need is greatest is expensive, time-consuming and likely unsustainable. In this paper, we share our experiences and the lessons we learnt when migrating a face-to-face workshop to an online learning space by general academics, using our experience in the Primer in SRs as a case study.
Our approach to developing the online Primer
Our initial step was to set up a small team of EBHC teachers who had taught on the face-to-face Primer. We kept our team as small as possible, with only two members (AR and SD) and a core project lead (MM), to enable efficient workflow and decision making. This core working team reported their progress and provided feedback to senior colleagues, specifically for checking content accuracy and flow. Beyond developing the online Primer, the working group had primary responsibilities such as research and teaching commitments, a challenge for us, as developing an online course is time intensive. We thus found that providing at least one person in the working group with some protected time was ideal (1–2 days a week), supported by external funding, to spearhead development of online content. This worked well, with regular input from the rest of the working group and senior colleagues.
We then mapped out the objectives, flow and process of the face-to-face Primer, highlighting the unique face-to-face components such as group discussions, ice-breakers and group activities such as peer feedback. We spread the overarching objectives and content across a 6-week period, providing participants with approximately 1 week to work through at least two key concepts (eg, introducing evidence-based practice and phrasing questions) or one difficult concept (eg, interpreting SR results). We added a precourse introduction and learning platform orientation period followed by a postcourse period of about 3 weeks, allowing participants to catch up and discuss content (table 1). Since our target audience for the online Primer was busy clinicians and policy-makers, we thought that providing 1 week to navigate through 2–3 hours worth of content, including readings, at their own pace, was feasible.
Overview of the online Primer in Systematic Reviews (SR)
After content mapping, sections were divided across the three working group members, each taking a topic they were most familiar with, who then developed the online presentation content. We used the face-to-face presentations as a starting point and expanded on the content for an online audience. Our content was not text heavy, as we decided to use voice-overs for most slides. Each presenter wrote the script for the voice-over of their presentation, which was later recorded and developed using the authoring tool software Articulate Presenter and Quizmaker.11 We used a standard learning management system (Moodle) to host the online course, which was supported by our University. Through the e-learning platform, participants were able to track their progress, engage in synchronous or asynchronous discussion forums and complete the multiple-choice questions or assessments at the end of each week. Synchronous discussions were facilitated through the Moodle Chat Activity and Zoom meetings, allowing participants to engage with facilitators as a group or one-on-one during specified times. The welcome and orientation space on this platform included an announcements space, navigation tutorial, course overview and a guide on learning to learn online (figure 2). Content for the postcourse period included asynchronous forum discussions. Drawing from the face-to-face Primer, we linked essential readings and resources to each topic, including videos on YouTube with in-depth explanations about the content, Cochrane reviews and landmark publications.
Screenshot of course orientation week.
Each week also had a dedicated asynchronous forum discussion, which was the primary method of engagement. To spark conversation, each week we posted 2–3 questions related to the topic being covered. Across the three facilitators, the responsibility for posting and answering questions rotated weekly. This allowed us to better manage our time and to respond to questions timeously and in detail.
As much as possible, we tried to mirror the features of the face-to-face Primer in considering the key aspects linked to successful EBHC teaching and e-learning.7 We created ample space for synchronous and asynchronous peer-to-peer or peer-to-facilitator discussions via forums or Zoom meetings. We encouraged peer feedback on specific activities, for example, on the PICO questions generated, or judgements of ROB. We included relevant topics and examples as far as possible, and designed interactive presentations. Online feedback was informal and attempted to flatten perceived academic hierarchies.
The online Primer in SRs has been running since 2016 and has been offered six times with a total of 160 participants with an 86% completion rate. Below we incorporate some user feedback during pilot testing of the online Primer to improve usability, accessibility and content. We investigated this using a mixed-methods design (see online supplementary appendix 1). We present some key lessons learnt during the development and implementation of the course, for busy academics who might consider transitioning from a face-to-face programme to an online approach.
Supplemental material
User feedback and lessons learnt
Working group composition and expertise
We found that keeping the core working group as small as possible was advantageous, as setting up meetings and reviewing and developing content was far more efficient when actioned by a small dedicated team. If funding is available, it is helpful to have an information technologist (IT) as part of the team. This works well if she or he is paired with a content expert, who can then create and transfer content to the online platform in parallel. Our university had a dedicated IT expert who provided support on technical issues with the Moodle platform or Articulate. However, our experience was that an IT specialist is not essential if at least one team member is familiar enough with a particular software or e-learning platform.
Designing course content
In comparing the face-to-face to the online Primer, an important aspect that took a long time to get right was the ‘look and feel’ of the digital learning content. This was especially difficult as there were multiple contributors. During face-to-face workshops, this is a non-issue as the look and feel is expressed in body language and engagement. Online, however, it comes across with first impressions and has a lasting impact. We therefore spent significant time and effort aligning presentation styles, colours and text format across topics.
In feedback, participants did enjoy a variety of mediums within content, particularly videos, and that was an important consideration when deciding what face-to-face content was best suited for what kind of teaching medium (eg, graphs, text or videos).
‘A video gives me the impression that, you know I’m actually seeing people, it gives it a bit of life’ (novice researcher)
Plan for updates and changes
When planning the design of an online course, we found updating content a significant challenge that took a large amount of time and effort. This was especially true where we had to redo voice recordings, update videos or introduce new methods or assessments. We found it was much easier for the majority of content to be in a narrative form as opposed to self-contained videos or educational packages (eg, self-contained packages developed through software such as Articulate). We often wanted to make small tweaks in our content packages; however, due to the authoring tool software we were using, it required far more input than simply editing text and it quickly became an inefficient task.
Maintain online engagement and interactive learning spaces
Participants enjoyed engaging with each other online and creating a community of practice. We found that creating a dedicated space for participants to introduce themselves and describe what they were interested in and why they joined was a valuable icebreaker to promote a comfortable space for engagement.
‘Sharing in forums gives a classroom effect where you could actually communicate with other colleagues, that I found very helpful’ (senior researcher).
However, more often than not, without facilitators’ weekly input, there would be limited questions and forum activity from participants. We quickly realised that weekly questions to stimulate discussions were needed, with encouragement for feedback. While engaging online, we kept our language informal in forums as most of our participants were not native English speakers. End of week self-assessments were also a key aspect of engaging with participants by providing them with feedback and an opportunity to reflect on their progress and understanding.
Consider internet access restrictions
When designing course content, we had to be cognisant of the target audience’s internet access and quality. For the online Primer, high-bandwidth-dependent content such as high-resolution images or large files to download were kept to a minimum. Where possible, we found that allowing participants to read text that was self-explanatory was the most efficient way of keeping bandwidth demand to a minimum.
Online courses have the advantage of allowing participants to work through content at their own pace and where they are most comfortable, either at home, at work or on the phone. However, many of our senior participants were unable to engage with content during working hours, as business or institutional firewalls prevented access. This was an issue we were unable to resolve, except by forewarning participants who might want to access content from work.
‘…it was easier for me to log in when I was at home than when I was in the office, and that’s because they have the firewall, so they block many external links’ (experienced researcher).
Conclusion
We shared our approach and lessons learnt of transitioning a face-to-face workshop to an exclusive online learning space by drawing from our experiences from the Primer in SR workshop and online module. We applied best teaching EBHC practices such as creating interactive spaces in an online environment. EBHC teachers who are responsible for creating online learning content should consider working in small and efficient teams, striking a balance between content and IT expertise, planning ahead to enable easy updates and on the fly changes, provide an interactive online space through engagement, and consider internet and access restrictions when developing content for low-income and middle-income country audiences.
Ethics statements
Patient consent for publication
Footnotes
Twitter @MikeMcCaul3
Contributors MM and AR drafted the manuscript. SD, TK, PG and TY contributed to the manuscript and provided guidance. All authors read and approved the final manuscript.
Funding Cochrane’s Global Evidence Synthesis Initiative Pilot site (2013–2015). Effective Health Care Research Consortium which is funded by UK aid from the UK Government for the benefit of developing countries (grant: 5242).
Map disclaimer The depiction of boundaries on this map does not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. This map is provided without any warranty of any kind, either express or implied.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.