Look at Yourself First: Self-Assessment of Non-Technical Skills During a Simulated Anesthesia Crisis Situation
Keywords:
non-technical skills training, self- assessment, simulation assessmentAbstract
Background: Non-technical skills are an essential element of training for healthcare professions. However, the effectiveness of training has been questioned as a result of poor ability of learners to self-evaluate their non-technical skills. Objective: The goal of this study was to evaluate the correlation between self-assessments and expert assessments of non-technical skills in student registered nurse anesthetists (SRNAs) in a simulated crisis setting. Methods: Thirty-five SRNAs completed a non-technical skills workshop for anesthetists, and then participated in a crisis management simulation. A post-simulation self-assessment was performed immediately after the session. The simulation was recorded for expert assessment using the Anesthetists’ Non-Technical Skills framework. Results: The self-assessment test showed an acceptable level of internal consistency (Cronbach’s alpha = 0.80, 0.71, 0.84, and 0.85 for task management, teamwork, situation awareness, and decision-making, respectively). We found weak to moderate and inverse relationships in terms of the agreement between self-assessments and experts (Pearson correlation coefficient = -0.14, -0.09, -0.28, -0.45, and -0.33, P=0.45, 0.54, 0.10, 0.01 and 0.05 for task management, teamwork, situation awareness, decision-making, and the total score, respectively). Conclusion: This study showed no correlation of the self-assessment and performance assessment from the experts.
References
Flin RH, O’Connor P, Crichton M. Safety at the sharp end: a guide to non-technical skills. Burlington, NY: Ashgate Publishing 2008.
Boet S, Bould MD, Fung L, et al. Transfer of learning and patient outcome in simulated crisis resource management: a systematic review. Can J Anesth. 2014;61:571-82.
Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf. 2014;23:359-72.
Gordon M, Darbyshire D, Baker P. Non-technical skills training to enhance patient safety: a systematic review. Med Educ. 2012;46:1042-54.
Ardizzone LL, Enlow WM, Evanina EY, Schnall R, Currie L. Impact of a patient safety curriculum for nurse anesthesia students. J Nurs Educ. 2009;48:706-10.
Flynn FM, Sandaker K, Ballangrud R. Aiming for excellence - A simulation-based study on adapting and testing an instrument for developing non-technical skills in Norwegian student nurse anaesthetists. Nurse Educ Pract. 2017;22:37-46.
Wunder LL. Effect of a Nontechnical skills intervention on first-year student registered nurse anesthetists’ skills during crisis simulation. AANA J. 2016;84:46-51.
Garden AL, Le Fevre DM, Waddington HL, Weller JM. Debriefing after simulation-based non-technical skill training in healthcare: a systematic review of effective practice. Anaesth Intensive Care. 2015;43:300-8.
Boet S, Bould MD, Bruppacher HR, Desjardins F, Chandra DB, Naik VN. Looking in the mirror: self-debriefing versus instructor debriefing for simulated crises. Crit Care Med. 2011;39:1377-81.
Boet S, Bould MD, Sharma B, et al. Within-team debriefing versus instructor-led debriefing for simulation-based education: a randomized controlled trial. Ann Surg. 2013;258:53-8.
Cheng A, Hunt EA, Donoghue A, et al. Examining pediatric resuscitation education using simulation and scripted debriefing: a multicenter randomized trial. JAMA Pediatr. 2013;167:528-36.
Lee J, Lee H, Kim S, et al. Debriefing methods and learning outcomes in simulation nursing education: a systematic review and meta-analysis. Nurse Educ Today. 2020; 87:1043.
Roh YS, Kelly M, Ha EH. Comparison of instructor-led versus peer-led debriefing in nursing students. Nurs Health Sci. 2016;18:238-45.
Tannenbaum SI, Cerasoli CP. Do team and individual debriefs enhance performance? A meta-analysis. Hum Factors. 2013;55:231-45.
Bandura A. Social cognitive theory of self-regulation. Organ Behav Hum Decis Process. 1991;50:248-87.
Sa-artwai P. The development of nurse anesthetists’ competency assessment scale, Phramongkutklao Hospital. J Royal Thai Army Nurses. 2019;20:123-30.
Boet S, Larrigan S, Martin L, Liu H, Sullivan KJ, Etherington N. Measuring non-technical skills of anaesthesiologists in the operating room: a systematic review of assessment tools and their measurement properties. Br J Anaesth. 2018;121:1218-26.
Rosen MA, Salas E, Wu TS, et al. Promoting teamwork: an event-based approach to simulation-based teamwork training for emergency medicine residents. Acad Emerg Med. 2008;15:1190-8.
Johnson AP, Aggarwal R. Assessment of non-technical skills: why aren’t we there yet? BMJ Qual Saf. 2019;28:606-8.
Arora S, Miskovic D, Hull L, et al. Self vs expert assessment of technical and non-technical skills in high fidelity simulation. Am J Surg. 2011;202:500-6.
van Merrienboer JJ, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ. 2010;44:85-93.
Fraser KL, Ayres P, Sweller J. Cognitive load theory for the design of medical simulations. Simul Healthc. 2015;10:295-307.
Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ. 2003;37:830-7.
Lyk-Jensen HT, Jepsen RM, Spanager L, Dieckmann P, Ostergaard D. Assessing nurse anaesthetists’ non-technical skills in the operating room. Acta Anaesthesiol Scand. 2014;58:794-801.
Eva KW, Regehr G. Self-assessment in the health professions: a reformulation and research agenda. Acad Med. 2005;80(Suppl):S46-54.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright
Copyright of all articles published in the Thai Journal of Anesthesiology is held by the Thai Journal of Anesthesiology, the Royal College of Anesthesiologists of Thailand. Submission of a manuscript constitutes the authors' agreement that, upon acceptance for publication, copyright of the work is transferred to the Journal. The content and any opinions expressed in published articles are the sole responsibility of the authors and do not necessarily reflect the views of the editorial board or the Royal College of Anesthesiologists of Thailand.
Licensing
All articles are published open access under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0). Under this license, anyone may copy and redistribute published articles in any medium or format for non-commercial purposes, provided that appropriate credit is given to the authors and to the Thai Journal of Anesthesiology as the original source of publication, a link to the license is included, and the material is not modified in any way. Requests for commercial use or for adaptation of published material should be directed to the editorial office.
Authors' Rights
Although copyright is held by the Journal, authors retain the right to reuse their own published material in their future scholarly works, theses and dissertations, teaching, and academic presentations, and to deposit the final published version (PDF) in institutional or subject repositories or on personal and institutional websites, without embargo, provided that the Thai Journal of Anesthesiology is clearly acknowledged as the original place of publication.
