Comparing Effectiveness of Online Text-based and Video-based Material in Anesthesia with Jet Ventilation and Microlaryngeal Surgery:

A Multicenter Randomized Trial

Sawita Kanavitoon, M.D.1, Saowapark Chumpathong, M.D.1, Arpa Chutipongtanate, M.D.2,3, Jutarat Tanasansuttiporn, M.D.4, Sirirat Rattana-arpa, M.Sc.1,*

1Department of Anesthesiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand, 2Division of Pediatric Cardiac Anesthesia,

Cincinnati Children’s Hospital Medical Center, 3Department of Anesthesiology, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand,

4Department of Anesthesiology, Songklanagarind Hospital, Prince of Songkla University, Thailand.

INTRODUCTION

E-learning has played an important role in medical education, especially since the COVID-19 pandemic. In contrast to conventional bedside teaching, it enables

“anytime-anywhere” access and is suitable for learners who want to learn the content of a training course at their own pace. Though e-learning cannot replace conventional clinical teaching, it provides a medium of knowledge

*Corresponding author: Sirirat Rattanaarpa E-mail: rattanaarpa.s@gmail.com

Received 18 January 2024 Revised 9 April 2024 Accepted 26 April 2024 ORCID ID:http://orcid.org/0000-0001-9771-4752 https://doi.org/10.33192/smj.v76i7.267386

All material is licensed under terms of the Creative Commons Attribution 4.0 International (CC-BY-NC-ND 4.0) license unless otherwise stated.

for many medical specialties, including anesthesia.1 The Royal College of Anaesthetists has developed a web-based resource, e-Learning Anaesthesia (e-LA), that provides knowledge and core concepts for trainees to assist in their preparation for examinations.2 In the case of clinical skills training, knowledge of each skill (the cognition stage) is required before dependent and independent skills practice (the integration and automation phases).2 Indications, contraindications, procedural steps, complications, and complication prevention strategies should be emphasized before learners undertake skills practice. In current training, providing learning objectives is not adequate for learners to gain clinical knowledge.3 Various learning modalities can be used to promote knowledge acquisition and students’ transition to a higher degree of competence.

Low-pressure low-frequency jet ventilation (LPLFJV) is a safe, tubeless airway technique. It is occasionally used for airway surgery as a rescue technique in “cannot intubate-cannot ventilate” situations.4,5 LPLFJV is usually applied at the supraglottic level, thereby providing proper airway support with a lower risk of airway fire and less hypercapnia than high-frequency jet ventilation (HFJV).5,6 The learning of LPLFJV by novices is challenging. This is because they have limited schema related to the topic, and there are many details on special equipment and monitoring, anesthesia choice, complications, and communication between surgeons and anesthesiologists. In Thailand, clinical knowledge and skills for anesthesia for jet ventilation and microlaryngeal surgery are taught simultaneously during patient encounters. However, varieties in patient pathologies and the time constraints imposed by clinical settings limit the knowledge that residents can gain.

Findings from previous studies support the notion that various types of learners achieve better training outcomes with video-based training than with conventional techniques or text-based online materials.7-11 Additionally, there is evidence that video-based training improves clinical skills and enhances short-term memory relative to text-based resources.12-14

This study compared the cognitive learning outcomes and satisfaction levels achieved with video-based and text-based online learning to identify which technique is superior and preferable for anesthesia residents. The specific research question was whether implementing online video-based training for jet ventilation and microlaryngeal surgery improves the cognitive domain learning outcomes of anesthesia residents of Siriraj Hospital, Ramathibodi Hospital, and Songklanagarind Hospital.

MATERIALS AND METHODS

Study design, setting, and population

This prospective randomized study recruited residents undertaking a 3-year anesthesia residency program. All residents enrolled in the program during the study period were eligible to participate. Their involvement was voluntary, and they had the right to withdraw at any stage. The residents signaled their informed consent to participate in the study by clicking on an “agree-to-proceed” button on the online site; this action automatically initiated a pretest. All data were de-identified, treated confidentially, and restricted to the researchers and research assistants involved in this study. Before this research began, its protocol was approved by the Institutional Review Boards of the Faculty of Medicine Siriraj Hospital (Si 655/2020), the Faculty of Medicine, Prince of Songkla University (40281/2020), and the Faculty of Medicine, Ramathibodi Hospital (1503/2020).

Sampling process and statistical analysis

A stratified randomization technique was used to form 2 groups: a “text group” and a “video group.” The randomization was based on each participant’s year in the anesthesia residency program (first, second, or third). For each program year, the same number of residents was assigned to each group. A target sample size of 102 was determined by estimating the effect size to be 0.5, which is a medium size for an educational study.11 The sample size was calculated for a 1-sided, independent- sample t-test using the following parameters: effect size

= 0.5, P = 0.05, type I error = 5%, and power = 80%. The calculation was performed in G*Power (version 3.1). After allowing for a dropout of 20%, the total sample size was determined to be 124 participants. In all, 126 residents were enrolled in the study (74 from Siriraj Hospital, 29 students from Ramathibodi Hospital, and 23 from Songklanagarind Hospital). However, 4 residents were later excluded from our analyses due to contamination of their learning material from the other group. In the end, there were 122 participants (Siriraj Hospital, 72; Ramathibodi Hospital, 27; and Songklanagarind Hospital,

23). Sixty participants were allocated to the text group, while 62 were assigned to the video group (Fig 1).

All statistical analyses were performed using SPSS Statistics version 21 for Windows (SPSS, Inc., Chicago, IL, USA). Continuous variables with normal distribution are presented as mean ± standard deviation, and non- normally distributed continuous variables are reported as median and interquartile range. Categorical data are shown as numbers and percentages. Data comparisons

Fig 1. Participant flow and randomization of the trial.

were performed using independent t-test, Mann-Whitney U test, or Pearson’s chi-squared test. A p-value less than

0.05 was considered statistically significant.

Research instruments

Video and text-based learning materials were developed by the research team and one senior anesthesiologist at Siriraj Hospital, Mahidol University. The video length is 21 minutes 47 seconds. The text-based material, consisting of 4 pictures, is 6 pages long. Content validity and comparability of video, text learning materials, and 40 multiple-choice questions (MCQs) were checked by other 3 senior anesthesiologists at Siriraj Hospital, Mahidol University. Each anesthesiologist had at least 10 years of experience with ear, nose, and throat procedures and was familiar with jet ventilation techniques. The MCQs assessed the remembering, comprehending, applying, analyzing, and synthesizing of the clinical skills and information provided in the video and text learning materials. The index of objective congruence (IOC) was used to validate the MCQs. Nineteen MCQs had an IOC = 1; the 21 other MCQs had an IOC = 0.67. The MCQs were used to form 3 sets of tests. Although each set had the same 40 MCQs, the questions were presented in a markedly different order in each set. Additionally, the test sets covered the same table of specifications used for knowledge in the video and text learning materials. No differences were found in the knowledge and learning points of the video- and text-group materials.

In addition, two 5-point Likert-scale questionnaires— one for the video group and the other for the text group— were developed to evaluate participants’ satisfaction with

the quality of the materials and their feelings toward the learning media. These satisfaction questionnaires were divided into 6 domains that gauged students’ perceptions of the following:

• the ease of understanding the content

• the degree of increase in confidence after learning

• a better technique than learning through clinical observation

• a better technique than learning in the operating room

• the likelihood of rereading or rewatching the material (as appropriate)

• overall satisfaction

Procedure and data analysis

The study was conducted from October 2020 to March 2021. The assigned learning materials, the 3 sets of MCQ tests, and the 2 versions of the satisfaction questionnaire were distributed online via the Siriraj E-Learning and Education Community (SELECx) website. The participants logged on with individually assigned usernames and passwords to access either the text- or video-based learning material (depending on whether they had been randomized to the text or video group). They also had to log on to perform the MCQ tests and complete the questionnaires. There were time limits between performing the online study and the MCQ tests. The first MCQ test (the “Pretest”) was performed before participants commenced online learning. The second MCQ test (“24-hour posttest”) was taken within 7 days of the Pretest and within 24 hours of completion of the online learning material. The last MCQ test (“3-month

posttest”) was taken 3 months after each participant had finished using the online material. The participants could review the learning material at any time during the study period; the date of each review by a participant was recorded (Fig 1).

At the same time as taking 3-month posttest, each participant completed the satisfaction questionnaire on the effectiveness of the video-based or text-based training material (as appropriate). The confidentiality of the participants’ responses to the 3 sets of MCQ tests and the satisfaction questionnaires was secured by restricting the data to the research team and information technology staff.

RESULTS

Demographic characteristics

In all, 122 residents enrolled in years 1, 2, and 3 of the anesthesia residency program participated in the study. Of these, 72 (59.0%) were students at Siriraj Hospital, 27 (22.1%) attended Ramathibodi Hospital, and 23 (18.9%) were from Songklanagarind Hospital. There were no significant differences in the sex, training year, mean age, mean grade point average (GPA), and time spent during the learning of the participants assigned to the video and text groups. The demographic data of the residents are summarized in Table 1.

Analysis of the learning outcomes of the text group and the video group

The mean ± SD scores achieved by the text-group participants for Pretest, 24-hour posttest, and 3-month posttest were 23.10 ± 7.17, 29.78 ± 4.79, and 27.24 ± 6.93, respectively (Table 2). The corresponding values for the video group were 23.08 ± 6.35, 29.72 ± 3.70, and 26.89

± 7.43. There were no significant differences between the 2 groups’ Pretest, 24-hour posttest, and 3-month posttest scores, with P = 0.987, 0.941, and 0.800 from the independent-samples t-test.

Analysis of the gain scores of the text group and the video group after learning

Two gain scores were employed to evaluate the degrees of learning the text and video group members achieved. One gain score was the difference between the 24-hour posttest and Pretest scores; the difference between the 3-month posttest and Pretest scores represented the other gain score. The gain scores for 24-hour posttest and the Pretest were 6.78 ± 6.59 for the text group and 5.77

± 3.74 for the video group, with no significant difference (P = 0.347). The gain scores for 3-month posttest and the Pretest were 4.69 ± 9.13 for the text group and 3.47 ±

9.07 for the video group, without a significant difference (P = 0.488; Table 3).

TABLE 1. Participant demographic data (N = 122).

Data are presented as n (percentage) and mean (standard deviation, SD).

*Center 1 = Siriraj Hospital, Mahidol University; Center 2 = Ramathibodi Hospital, Mahidol University; Center 3 = Songklanagarind Hospital, Prince of Songkla University.

** Learning time presents as median (IQR)

TABLE 2. Mean and standard deviation (SD) of MCQ test scores of the video group and the text group at 3 time points (Pretest, 24-hour posttest, and 3-month posttest).

TABLE 3. Comparison of gain scores after learning with text-based or video-based materials.

Analysis of the long-term memory results of the text group and the video group

The phase between 24-hour posttest and 3-month posttest was employed to assess the degree of conversion from working memory to long-term memory. Our study found a general decrease in the MCQ test scores between 3-month posttest and 24-hour posttest for individual students. The mean differences were -2.18 ± 6.53 for the text group and -3.10 ± 6.82 for the video group, with no significant difference between the groups (P = 0.495; Table 3).

Analysis of perceptions of the effectiveness of video- based and text-based learning

An independent-samples t-test was performed to find any significant differences between the groups’ mean scores for the 6 domains of the satisfaction questionnaires (Table 4). Fifty-two participants responded to the questionnaire of perception of effectiveness of learning material. The video-group students had significantly higher scores for ease of understanding the content (P = 0.001), a better technique than clinical observation

(P = 0.042), a better technique than learning in the operating room (P = 0.004), and overall satisfaction (P = 0.02). In contrast, the text-group students demonstrated a better mean score for the likelihood of rereading the text-based material than the video-group students had for rewatching the videos. However, the difference was nonsignificant (P = 0.285).

DISCUSSION

This study drew upon validated MCQ sets to evaluate the degree of knowledge acquired after training in a clinical skill using text-based and video-based online materials. The results revealed that video-based online learning was as effective as text-based learning. Regarding perceptions, the students were satisfied with both e-learning methods, rating each as better than traditional learning through clinical observation or in the operating room. The students rated video-based e-learning more highly than text-based e-learning in terms of perceived increases in confidence, ease of understanding the content, a better technique than clinical observation or operating-room learning, and overall satisfaction. In terms of memory

TABLE 4. Comparison of the participant perceptions of the effectiveness of video-based and text-based learning (N=52).

*A P-value < 0.05 indicates statistical significance

decay, time weakened the retention of memories in the text and video groups. Analysis of the gain scores for 3-month posttest (taken 3 months after each participant had finished using the online material) and the Pretest revealed the same memory decay rate for both groups. The finding of this study accords with a prior study that found no differences in students’ theoretical knowledge of the Dix–Hallpike test and blood pressure recording in theory assessments conducted via MCQs.15 However, the present investigation found different results from another study suggesting that higher-level tools that utilize video recordings with simple or complex animation can result in greater knowledge acquisition than lower-level modalities that provide text, audio, or a simple presentation.16 Other studies showed that while there were no differences in test scores for the theory parts of information presented via video- and text-based e-learning materials, scoring differences were found for procedurally related information.11 However, the current investigation assessed the “knows” and “knows how” via MCQ tests, while clinical performance was not

examined.

The finding related to perception can be explained by dual-coding theory, the video-based e-learning which holds that combining visual and auditory stimuli to present information can enhance understanding and promote assimilation of the learning topics.17

The finding from 3-month posttest is consistent with memory decay theory, which explains how time affects the retention of memories.18 Even the current investigation’s video-based e-learning, which promotes dual encoding, could not enable students to memorize

content completely. No previous study has compared the decay rates of text- and video-based e-learning. To promote long-term memory, content rehearsal for any form of e-learning should be emphasized in future research on other learner types (i.e., visual, aural, reading/ writing, and kinesthetic) to gain more information on the effectiveness of different e-learning modalities.19

This study has several limitations. First, there was no data on each participant’s preferred learning style, which might affect the learning outcomes achievable with different training techniques. Second, the MCQ tests only evaluated one aspect of cognition. The test results may not adequately reflect the practical skill and process elements of anesthesia training. Research using multi-aspect measurements for skill-based evaluations (eg, Objective Structured Clinical Examination) would provide more comprehensive data for assessing student learning. Third, this study evaluated the learning techniques by comparing the learning outcomes achieved with text- based and video-based e-learning resources for just 1 clinical skill. Therefore, it may not be possible to generalize our results to learning other skills. Lastly, there might be some maturation effects on the participants during the study period. Some residents might be exposed to the jet ventilation procedure in their assigned rotation. Further research on other skill types would provide more information on the effectiveness of different e-learning methods. Furthermore, assessments based on new technologies were recommended to improve simulation experiences and consequently better evaluate clinical performance.20

CONCLUSION

Satisfaction, clarity, and ease of understanding are the benefits of clinical learning through video-based online learning, superior to text-based online learning. Both learning materials provide improving gain scores without statistically significant differences. Tailor-made, learning type-based learning techniques may be useful for the 2024 learners.

ACKNOWLEDGMENTS

The authors would like to thank Miss Pavinee Sareenun for the SELECx administrative support, Miss Julaporn Pooliam for the statistical analysis.

Conflict of interest

The authors have no conflicts of interest to declare.

REFERENCES

1. Tan WW, Lin GSS. Dental Students’ Perspectives and Learning Experiences during the Covid-19 Outbreak: A Qualitative Study. Siriraj Med J. 2023;75(8):592-8.

2. e-Learning Anaesthesia | The Royal College of Anaesthetists [Internet]. www.rcoa.ac.uk. [cited 2024 Feb 24]. Available from: https://www.rcoa.ac.uk/e-learning-anaesthesia.

3. Pongratanakul R, Kanthanet W, Sungkharuk K, Srisoongnern

   S. Specific Learning Objectives in the Form of Self-assessment to Facilitate Rehabilitation Residency Training Competency: A Prospective Cohort Study. Siriraj Med J. 2023;75(4):275-81.

4. Apfelbaum JL, Hagberg CA, Connis RT, Abdelmalak BB, Agarkar M, Dutton RP, et al. 2022 American Society of Anesthesiologists Practice Guidelines for Management of the Difficult Airway. Anesthesiology. 2022;136(1):31-81.

5. Myint CW, Teng SE, Butler JJ, Griffeth JV, Fritz MA, Meiler SE, et al. Low Pressure Low Frequency Jet Ventilation: Techniques, Safety and Complications. Ann Otol Rhinol Laryngol. 2022: 34894211072630.

6. Vourc’h G, Fischler M, Michon F, Melchior JC, Seigneur F. Manual jet ventilation v. high frequency jet ventilation during laser resection of tracheo-bronchial stenosis. Br J Anaesth. 1983; 55(10):973-5.

7. Levitan RM, Goldman TS, Bryan DA, Shofer F, Herlich A. Training with video imaging improves the initial intubation success rates of paramedic trainees in an operating room setting. Ann Emerg Med. 2001;37(1):46-50.

8. Jang HW, Kim KJ. Use of online clinical videos for clinical

   skills training for medical students: benefits and challenges. BMC Med Educ. 2014;14:56.

9. Zhang D, Zhou L, Briggs RO, Nunamaker JF. Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness. Information & Management. 2006; 43(1):15-27.

10. Lee NJ, Chae SM, Kim H, Lee JH, Min HJ, Park DE. Mobile- Based Video Learning Outcomes in Clinical Nursing Skill Education: A Randomized Controlled Trial. Comput Inform Nurs. 2016;34(1):8-16.

11. Choi Y, Song E, Oh E. Effects of teaching communication skills using a video clip on a smart phone on communication competence and emotional intelligence in nursing students. Arch Psychiatr Nurs. 2015;29(2):90-5.

12. Soucisse ML, Boulva K, Sideris L, Drolet P, Morin M, Dube

    P. Video Coaching as an Efficient Teaching Method for Surgical Residents-A Randomized Controlled Trial. J Surg Educ. 2017;74(2):365-71.

13. Mishra K, Mathai M, Della Rocca RC, Reddy HS. Improving Resident Performance in Oculoplastic Surgery: A New Curriculum Using Surgical Wet Laboratory Videos. J Surg Educ. 2017;74(5): 837-42.

14. Ludwig S, Schuelper N, Brown J, Anders S, Raupach T. How can we teach medical students to choose wisely? A randomised controlled cross-over study of video- versus text-based case scenarios. BMC Med. 2018;16(1):107.

15. Buch SV, Treschow FP, Svendsen JB, Worm BS. Video- or text- based e-learning when teaching clinical procedures? A randomized controlled trial. Adv Med Educ Pract. 2014;5:257-62.

16. Worm BS, Jensen K. Does peer learning or higher levels of e-learning improve learning abilities? A randomized controlled trial. Med Educ Online. 2013;18:21877.

17. Junhasavasdikul D, Sukhato K, Srisangkaew S, Theera-Ampornpunt N, Anothaisintawee T, Dellow A; T-REX group. Cartoon versus traditional self-study handouts for medical students: CARTOON randomized controlled trial. Med Teach. 2017;39(8):836-843.

18. Ricker TJ, Vergauwe E, Cowan N. Decay theory of immediate memory: From Brown (1958) to today (2014). Q J Exp Psychol (Hove). 2016;69(10):1969-95.

19. Maprapho P, Sitticharoon C, Maikaew P, Lertsiripatarajit V, Keadkraichaiwat I, Charoenngam N. Changes in the VARK Learning Style from the First to the Second Preclinical Year of Medical Students: A Follow-up Cross-sectional Questionnaire Study in a Thai Medical School. Siriraj Med J. 2023;75(3): 181-90.

20. Thakker A, Devani P. Is there a role for virtual reality in objective structured clinical examinations (OSCEs)? MedEdPublish. 2019;8(180):180.