Development and Validation of an Objective Structured Teaching Encounter Evaluation (OSTE) Form Following Endotracheal Intubation Practice
Main Article Content
Abstract
Introduction: This study aimed to develop and evaluate the quality of an evaluation form for assessing faculty feedback following endotracheal intubation practice by medical students. Materials and methods: The form consisted of 10 items. The participants were 57 fifth-year medical students who evaluated feedback provided by six anesthesiology faculty members across 78 evaluation encounters. Results: The form was classified into three components based on Exploratory Factor Analysis (EFA) for construct validity: goal and objective setting, reflection and feedback provision, and developmental planning and monitoring. Content validity was assessed by eight experts, yielding an Index of Item–Objective Congruence (IOC) ranging from 0.67 to 1.00, Item-Content Validity Index (I-CVI) values ranging from 0.88 to 1.00, and all CVR values were 1.00. The Cronbach’s alpha coefficient was 0.915 for the overall form and 0.878, 0.866, and 0.827 for the three components, respectively. The mean evaluation scores of students who performed endotracheal intubation and those who served as observers were 18.875 ± 2.330 and 18.700 ± 3.456, respectively. The mean evaluation scores among the six faculty members ranged from 17.769 to 20.000. Conclusion: The evaluation form developed in this study demonstrated evidence supporting content validity, construct validity, and internal consistency reliability in the context of endotracheal intubation practice by medical students. Further evaluation of the instrument in different educational settings and populations is warranted before its broader application or use for assessing other practical skills.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Articles in this journal are copyrighted by the Royal Thai Army Medical Department and published under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) license.
may be read and used for academic purposes, such as teaching, research, or citation, with proper credit given to the author and the journal.
Use or modification of the articles is prohibited without permission.
Statements expressed in the articles are solely the opinions of the authors.
Authors are fully responsible for the content and accuracy of their articles.
Any other republication of the articles requires permission from the journal.
References
Medical Council of Thailand. [Medical Competency Assessment Criteria for National License 2012. 2nd ed., B.E. 2563 (2020)] [Internet]. Nonthaburi: Medical Council of Thailand; 2020. Available from: https://tmc.or.th/pdf/MCACNL2012-2-2563.pdf. Thai.
Tavakol M, Scammell BE, Wetzel AP. Feedback to support examiners’ understanding of the standard-setting process and the performance of students: AMEE Guide No. 145. Med Teach. 2022;44(6):582-95. doi:10.1080/0142159X.2021.1993164.
Boillat M, Bethune C, Ohle E, Razack S, Steinert Y. Twelve tips for using the objective structured teaching exercise for faculty development. Med Teach. 2012;34(4):269-73. doi:10.3109/0142159X.2011.599891.
Kay HG, Mahoney MR, Edwards RA. The objective structured teaching encounter (OSTE) in health professions education: a systematic review. Med Teach. 2023;45(8):893-905. doi:10.1080/0142159X.2023.2189539.
Julian KA, Appelle N, O’Sullivan P, Morrison EH, Wamsley M. The impact of an objective structured teaching evaluation on faculty teaching skills. Teach Learn Med. 2012;24(1):3-7. doi:10.1080/10401334.2012.641476.
Hambleton RK. Validating the test score. In: Berk RA, editor. A guide to criterion-referenced test construction. Baltimore (MD): Johns Hopkins University Press; 1984. p. 199-230.
Gupta S, Grover S, Menon V, Indu PV, Vidhukumar K, Chacko D. Item generation and establishing face and content validity of a rating scale: a primer. Indian J Psychiatry. 2025;67(8):816-22. doi:10.4103/indianjpsychiatry_750_25.
Ayre C, Scally AJ. Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Meas Eval Couns Dev. 2014;47(1):79-86. doi:10.1177/0748175613513808.
Lawshe CH. A quantitative approach to content validity. Pers Psychol. 1975;28(4):563-75. doi:10.1111/j.1744-6570.1975.tb01393.x.
Taber KS. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Res Sci Educ. 2018;48(6):1273-96. doi:10.1007/s11165-016-9602-2.
Kyriazos TA. Applied psychometrics: sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology. 2018;9(8):2207-30. doi:10.4236/psych.2018.98126.
Noosen P, Rangkrut S. [The needs of medical education development for medical teachers in Thailand]. Siriraj Med Bull. 2021;14(4):13-20. doi:10.33192/smb.v14i4.246921. Thai.
Li Y, Sun C, Wang Y, Chi H, Tang H, Ma M, et al. Bias in student evaluations of teaching in undergraduate medical education: a qualitative study from a medical school in Northern China. BMC Med Educ. 2025;25(1):784. doi:10.1186/s12909-025-07300-w.
Aburawi E, McLean M, Shaban S. Evaluation of faculty: are medical students and faculty on the same page? Sultan Qaboos Univ Med J. 2014;14(3):e361-8. doi:10.18295/2075-0528.1600.
