Wayne Palmer, Maxim Christmas
Division of Orthopaedic Surgery, Department of Surgery, Radiology, Anaesthesia and Intensive Care, The University Hospital of The West Indies, Mona, Kingston 7, Jamaica
Dr. Maxim Christmas
Division of Orthopaedic Surgery
Department of Surgery, Radiology, Anaesthesia and Intensive Care
The University Hospital of The West Indies
Mona, Kingston 7
Email: [email protected]
The Faculty of Medical Sciences at the University of the West Indies, Mona Campus, has been training medical doctors for over seventy years. The undergraduate programme comprises a clinical aspect that is divided into five- and ten-week clerkships in the final three years. The orthopaedic clerkship is a five-week programme which is offered in year four and introduces the students to musculoskeletal conditions. In this five-week clinical rotation, students are taught in the hospital clinics, wards and operating rooms across several hospitals.1
The key element is student to patient contact for the development of clinical skills. The students were assessed at the end of the rotation via a computer-generated multiple choice (MCQ) exam and a digital objective structured clinically exam (OSCE), consisting of videos and images with short answer questions to be completed within a fixed time. A procedure card was also completed which contributed to the final grade.
Appreciating the importance of student perception of their clinical rotations and the delivery of content at different sites, a previous study, looking at the comparison of the grades obtained at each site, showed that there were no significant statistical differences in end of clerkship grades amongst medical students who did the clerkship at different sites.2 While acknowledging possible explanations for this result, the author recommended standardization of content delivery at all sites with tutor training and small group clinical demonstrations.2 The final group of rotating students for semester 2 of the 2019/20 academic year, had their clerkship disrupted during the first week, due to the Covid-19 pandemic. A four-week break was then given by the university to prepare for total online delivery of all courses.
Preparation for online teaching
The lecturers in this course were encouraged to submit their university ID numbers to the faculty Information technology department to facilitate access to our Virtual Learning Environment (ourVLE) and Blackboard collaborate ultra (BBC), the university’s official video conferencing platform. Our Virtual Learning Environment (OurVLE) is the official online learning platform used by the University of the West Indies, (UWI) Mona. It is a Moodle based platform which has a portal which links to blackboard collaborate ultra (BBC), a videoconferencing tool geared for virtual teaching.
A meeting with lecturers was held prior to the restart of teaching to outline the new structure of the clerkship and to familiarize the lecturers with the blackboard platform.
Teaching sessions were posted in the calendar feature and as a document on our VLE (Appendix 1). A forum feature was created to allow students to communicate asynchronously with lecturers and a virtual office was created for synchronous meetings.
On the first day of teaching, the students were oriented to the new teaching paradigm and the different functions of the online platforms.
Process of delivery
Lectures were delivered via BBC and recorded for asynchronous access. PowerPoint lectures were also made available on our VLE (Appendix 2). Clinical teaching sessions were as follows: 1. virtual rounds with case presentations were done with students’ attendance via BBC but patient identifiers were withheld. 2. Clinical examination sessions were delivered with volunteers via a video conference platform (BBC) while eliciting feedback from students as the exam took place (Appendix 3).
The student assessment included a mid-rotation MCQ as well as a mock exam which consisted of a MCQ on our VLE, which was shuffled between questions and within question answers, and an oral structured clinically oriented exam (OSCE) held on BBC (Appendices 4 and 5). The grades were analyzed by grouping them according to the grade structure used in the Faculty of Medical Sciences, i.e. 50-54.9= C, 55-59.9 =C+ etc with a pass being scores >50. The OSCE and MCQ were give equal weighting.
The final exams were standardized, the examiners met the weekend before the exams, via video conferencing, to determine the exam questions and an expected answer sheet and a marking rubric generated. On exam day, a BBC waiting room was created where the students were given exam instructions 30 minutes before the start of the exams and an exam room time schedule posted.
Multiple exam rooms were created on BBC and each room had one examiner who examined one student at a time. The exam process was recorded for later review as needed. Once the student entered an exam room, the examiner identified himself using video and ID card verification. The examiner then requested that the student turn on his mic and video and show his ID card to verify his identity. The exam would then take place and the examiner documents any connectivity issues and the exam process. Students with connectivity issues were contacted by WhatsApp to complete the exam as needed or given additional time. The waiting room was monitored to prevent students from entering the exam rooms before they were required to do so.
Eleven stations were used for the online OSCE to avoid prolonged waiting times by students and prevented those examined from communicating with those not yet examined. Hence each examiner only had 2 students. There were 4 stations which each lasted 5 minutes for a total of 20 minutes per student.
Twenty 23 students were examined by this process. Extra time was needed in 3 (13%) exams due to audio issues and in one exam a WhatsApp call was necessary to complete the exam.
There was a 95.6% pass rate with only one student failing the clerkship and this student failed both parts of the exam (Figure 1).
The overall average score for the group was 61% (B score) with the average OSCE score being 59% and the average MCQ score 62%. Thirty percent of students scored 65% (B plus) or greater on the OSCE exam.
The examiners commented that the lack of clinical experience was evident compared to previous groups. There was also a sense of disconnect with the students during online teaching sessions and many felt that only few students actively participated in class compared to face to face teaching. A survey of the online class was done at the end of the rotation and 88% of students rated the online classes as good to excellent (Figure 2).
Figure 1. Line graph showing general distribution of final grade for clerkship (i.e. OSCE + MCQ). Failure indicated by red circle
Figure 2: Showing distribution of student perception of online classes
Students were generally satisfied with the teaching process and most preferred the structure of the online teaching but felt they lacked the experience and skill of participating in patient care
The COVID-19 pandemic, a global health crisis, has influenced many countries to close the doors on traditional classroom education at schools, colleges and universities. The pandemic has caused policymakers to reduce the contact between students in order to save lives. Medical students and other healthcare workers have been experiencing synchronous online teaching in recent years however, this significant and sudden disruption has catapulted online teaching to much larger health science populations over a relatively short period of time. There is evidence in the literature to suggest that online teaching is as effective as traditional classroom teaching with possible higher satisfaction levels.3,4,5
The current pandemic has forced the postponement and cancellation of students’ assessments and examinations. However, it has also influenced many institutions to replace traditional examinations with creative online assessment tools as the world shapes its new reality.
The authors believe that assessments of medical students at the end of each clerkship are critical to the learning process for several reasons. All students should be given the opportunity to demonstrate their understanding of the subject matter. Assessments also evaluate whether the educational objectives and standards of the material are being met. This allows for the collection of relevant information about student performance and progress in addition to improving the content and delivery of material.
The University of the West Indies (the UWI) has increased the number of students in the medical undergraduate programme over the years resulting in much larger groups being assigned to the orthopaedic rotation at any given time. With the inability to adequately accommodate the number of assigned students with respect to satisfactory delivery of content and assessments, additional staff at several hospitals were employed to deliver the necessary content to the students. However, the heavy clinical workload at the peripheral hospitals posed another challenge to face to face teachings and end-of clerkship examinations. In an attempt to alleviate the face to face assessments at the end of the rotations a decision was made to host the end-of-clerkship examination online. This was achieved through our VLE at the Medical Library’s computer laboratory. An electronic version of the course content was made available to the students and lecturers and a Virtual Course Room (VCR) via Blackboard Collaborate (BBC) was used to encourage online discussion, participation and learning by the students.6,7
Prior to the start of the pandemic, weekly online tutorials were held with all students to facilitate the delivery of the same course content to all students irrespective of hospital location. In addition, a student-related feedback system was established on our VLE to track and address any issues that may arise during the clerkship.1 The usual examination process, before the pandemic, consisted of MCQs, procedure cards, case presentations and an OSCE. Over the years of delivering these assessments to approximately thirty students per clerkship, an average score in the B range is usually obtained with 1-2 failures for each 5-week rotation. With this modified approach to include online teaching our results show that the students scored similarly to what obtained previously with the average score being 61% ( B range) and the OSCE score being 59% compared to 62% and 59% as reported by Palmer in a previous study of this clerkship.2
During the pandemic, attempts were made to standardize the experience with the sessions held for all participating lecturers to familiarize themselves with the teaching tools before the actual delivery of online lectures. All lecturers were experienced in teaching medical students in the traditional classroom setting. In addition, introductory sessions were also held for the medical students to create a common understanding of the online platform before actual online classes began. Delivery of content was satisfactory based on the feedback from the lecturers. However, less active participation by the students was noted by several lecturers. This may have appeared so because students were more likely to respond if they felt the lecture was seeing them and interacting with them directly or unfamiliarity with the technology.
Many students felt more comfortable without their video connections and responded to questions only by typing the answers. Direct questioning with naming of the students was effective in increasing active participation via the audio connection. The method of online teaching and assessment in this study was accepted by most of the students with good to excellent rating on the post rotation survey. However, the lack of hands on teaching negatively influenced the students’ confidence and general perception in their overall experience. The authors believe the assessment method at the end of the clerkship was an unbiased effective method of evaluation evidenced by the high success rate. We believe that this assessment method did not account solely for the single observed failure.
Total online delivery of this clerkship will never fully satisfy all the learning objectives especially those that require development of examination skills. However, this method of virtual clerkships can provide a good substitute in the absence of face to face encounters as we aid medical students in adapting to the new realities of the current pandemic. Long term comparative cohort studies will be necessary to assess the effects of virtual clerkships on medical graduates.
Conflict of Interest statement: the authors have nothing to disclose. There are no potential conflicts of interest for this paper.
Ethics Approval statement:The nature of the data reported in the manuscript is not of a sensitive nature and does not require ethics approval. This study was designed and interpreted around routine data collection during the pandemic period and does not include actual human or animal subjects. Data collection at this time is encouraged in order to objectively create standards and protocols for effectively and efficiently administering content to medical undergraduates and postgraduates during the pandemic and beyond.
Author contribution statement: WP has made a substantial contribution to the conception and design of the study, and data acquisition, analysis and interpretation. He was involved in drafting the manuscript and revising it. MC has made a substantial contribution to the design of the study, analysis and interpretation. He was involved in drafting the manuscript or revising it critically for intellectual content.
Final approval for submission was given by both authors.
Funding statement: There has been no funding granted to support this work
- Palmer W, Rose REC. Medical Education AP Lateral and Oblique Views. WIMJ 2018;67(5):506-7.
- Palmer W. Comparison of Medical Student Performance: The Effect of Rotations at Different Sites. WIMJ 2017;e231 doi:10.7727
- Kyaw BM, Posadzki P, Paddock S, Car J, Campbell J, Tudor Car L. Effectiveness of Digital Education on Communication Skills Among Medical Students: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019 Aug 27;21(8):e12967. doi: 10.2196/12967.
- George PP, Zhabenko O, Kyaw BM, Antoniou P, Posadzki P, Saxena N, Semwal M, Tudor Car L, Zary N, Lockwood C, Car J.J. Online Digital Education for Postregistration Training of Medical Doctors: Systematic Review by the Digital Health Education Collaboration. Med Internet Res. 2019 Feb 25;21(2):e13269. doi: 10.2196/13269.
- McCutcheon K, Lohan M, Traynor M, Martin D. A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. J Adv Nurs. 2015 Feb;71(2):255-70. doi: 10.1111/jan.12509. Epub 2014 Aug 19. PMID: 25134985.
- Branday JM, Carpenter RA. The Evolution of Undergraduate Medical Training at The University of the West Indies: 1948-2008 WIMJ 2008; 57: 530–6.
- Orthopaedic Course outline. Division of Orthopaedics Department Of surgery Radiology Anaesthesia and Intensive Care, University of the West indies, Mona, Jamaica.