Development of a Workplace-based Assessment Scale for Student Anesthetist Nurses:

Parichad  Apidechakul; Namtip  Triyasunant; Ladda  Permpolprasert; Wiruntri  Punchuklang; Phongthara  Vichitvejpaisal

doi:10.33192/smj.v77i10.276186

Authors

Parichad Apidechakul Siriraj Health Science Education Excellence Center, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
Namtip Triyasunant Department of Anesthesiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
Ladda Permpolprasert Department of Anesthesiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
Wiruntri Punchuklang Department of Anesthesiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand
Phongthara Vichitvejpaisal Department of Anesthesiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok 10700, Thailand

DOI:

https://doi.org/10.33192/smj.v77i10.276186

Keywords:

Medical education, Scale, Student Anesthetist Nurses, Workplace-based Assessment

Abstract

Objective: To develop and validate a skills assessment tool for evaluating the workplace performance of student anesthetist nurses through a psychometric analysis of its reliability and validity.

Materials and Methods: The study was conducted in three sequential phases: Scale Development: A comprehensive literature review and expert consultations were undertaken to construct the initial assessment items. Content Validation: Three expert anesthetists reviewed the draft scale for content validity. Psychometric Testing: The final scale was evaluated using data from 40 student anesthetist nurse assessments in 2023-2024. Recruitment was conducted in the gynecology surgery rooms.

Results: The finalized workplace-based assessment scale consisted of 24 items spanning three key domains: patient and equipment preparation, technical skills, and non-technical skills. The psychometric analysis demonstrated stronginternal consistency (Cronbach’s α = 0.93), high inter-rater reliability (The Intraclass Correlation Coefficient (ICC) = 0.87), and moderate concurrent validity, as evidenced by Pearson’s correlation analysis (r=.511, p<.05, N=40).

Conclusion: The newly developed workplace-based assessment scale exhibits strong psychometric properties, providing a reliable and comprehensive tool for evaluating the clinical competencies of student anesthetist nurses in practice settings.

References

Antonsen KN, Hofstad JK. Simulating preoperative preparations with focus on non-technical skills in an OR nursing education program in Norway. In: Springer International Publishing; 2022. p. 37–52.

Ratanasuwan P, Sriraj W, Punjasawadwong Y, Choorat J, Charuluxananan S, Pravitharangul T. The Perioperative and Anesthetic Adverse Events in Thailand (PAAd Thai) study of anesthetic equipment malfunction or failure: An analysis of 2206 incident reports. J Med Assoc Thai. 2021;104:286–92.

Smedley P, Nicholas-Holley J. Recovery of patients from anaesthesia and surgery. In: Rodger D, Henshaw K, Rawling P, Miller S, editors. Fundamentals of Operating Department Practice. Cambridge: Cambridge University Press; 2022. p. 384–406.

Nucharoenkul S. Competencies and certification of knowledge and specialized expertise for advanced practice nurses. 1st ed. Bangkok: Siriyod Printing; 2010.

Matlhaba K. Essential clinical skills. In: Enhancing Clinical Competence of Graduate Nurses. Cham: Springer Nature Switzerland; 2025. p. 113–34.

Boker AM. Toward competency-based curriculum: Application of workplace-based assessment tools in the National Saudi Arabian Anesthesia Training Program. Saudi J Anaesth. 2016;10(4):417–22.

Khalafi A, Sharbatdar Y, Khajeali N, Haghighizadeh MH, Vaziri M. Improvement of the clinical skills of nurse anesthesia students using mini-clinical evaluation exercises in Iran: A randomized controlled study. J Educ Eval Health Prof. 2023;20:12.

Mortaz Hejri S, Jalili M, Masoomi R, Shirazi M, Nedjat S, Norcini J. The utility of mini-clinical evaluation exercise in undergraduate and postgraduate medical education: A BEME review: BEME Guide No. 59. Med Teach. 2020;42(2):125–42.

Kurdi MS, Hungund BR. Evaluation of mini-clinical evaluation exercise (mini-CEX) as a method for assessing clinical skills in anaesthesia postgraduate education. Indian J Anaesth. 2021;65(3):248–52.

Flin R, Fletcher G, Glavin R, Maran N, Patey R. Anaesthetists’ Non-Technical Skills (ANTS) System Handbook. Aberdeen: University of Aberdeen; 2003.

World Medical Association. World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA. 2013;310:2191–4.

Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav Res Methods. 2009;41(4):1149–60.

Kanjanawasee S. Classical test theory. 7th ed. Bangkok: Chulalongkorn University Press; 2013.

Saad S, Carter GW, Rothenberg M, Israelson E. Testing and assessment: An employer’s guide to good practices. Washington, DC: U.S. Department of Labor; 1999.

Taber KS. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Res Sci Educ. 2018;48:1273–96.

Popovich D. How to treat missing data in survey research. J Mark Theory Pract. 2025;33(1):43-59.

Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–63.

Senthil Kumar VS, Shahraz S. Intraclass correlation for reliability assessment: The introduction of a validated program in SAS (ICC6). Health Serv Outcomes Res Methodol. 2024;24(1):1–3.

Chin M, Pack R, Cristancho S. “A whole other competence story”: Exploring faculty perspectives on the process of workplace-based assessment of entrustable professional activities. Adv Health Sci Educ Theory Pract. 2023;28(2):369–85.

Slabbert B, Hoole C. A competency framework for coaches working in coaching development centres. SA J Ind Psychol. 2021;47(1):1–10.

Wingate TG, Bourdage JS, Steel P. Evaluating interview criterion-related validity for distinct constructs: A meta-analysis. Int J Sel Assess. 2025;33(1):1–21.

Gümüş MM, Kukul V. Developing a digital competence scale for teachers: Validity and reliability study. Educ Inf Technol. 2023;28(3):2747–65.

Maj M, van Os J, De Hert M, Gaebel W, Galderisi S, Green MF, et al. The clinical characterization of the patient with primary psychosis aimed at personalization of management. World J Psychiatry. 2021;20(1):4–33.

Delfino AE, Fuentes-López EA, de la Fuente RF, Altermatt FR. Cross-cultural adaptation and validation of the Spanish version of the Anesthetists' Non-Technical Skills (ANTS) assessment tool. J Clin Anesth. 2023;84:1-5.

Yeates P, Maluf A, Kinston R, Cope N, Cullen K, Cole A, et al. A realist evaluation of how, why and when objective structured clinical exams (OSCEs) are experienced as an authentic assessment of clinical preparedness. Med Teach. 2025;47(3):458–66.

Yi L, Shuai T, Zhou J, Cheng L, Jiménez-Herrera MF, Tian X. Development and validation of a machine learning-based predictive model for compassion fatigue in Chinese nursing interns: A cross-sectional study utilizing latent profile analysis. BMC Med Educ. 2024;24(1):1–20.

Martin EL, Martinez DA. The effect size in scientific publication. Educ. XXI.2023;26(1):9-17.

Gamborg ML, Mehlsen M, Paltved C, Vetter SS, Musaeus P. Clinical decision-making and adaptive expertise in residency: A think-aloud study. BMC Med Educ. 2023;23(1):1–20.

McIntosh TR, Liu T, Susnjak T, Watters P, Halgamuge MN. A reasoning and value alignment test to assess advanced GPT reasoning. ACM Trans Interact Intell Syst. 2024;14(3):1–37.

Chudapongse R, Sangpetngam B, Chankaew E, Aurboonyawat T, Churojana A, Withayasuk P. Comparative Evaluation of Imaging Modalities for Eligibility in Endovascular Treatment of Delayed Onset Acute Anterior Circulation Ischemic Stroke in Siriraj Hospital: A Retrospective Analysis. Siriraj Med J. 2024;76(9):573-80.