Development and Validation of the Anesthesia Critical- Thinking Test

Main Article Content

Taniga Kiatchai
Kasana Raksamani
Suwannee Suraseranivongse

Abstract

Objective: Despite critical thinking being a crucial skill in anesthesia practice, no specific tool exists to evaluate the skill in anesthesia training. The present study measured the validity, reliability and practicality of the Anesthesia Critical-Thinking Test (ACTT) for measuring the stages of critical thinking in anesthesia settings. Materials and methods: Part I Tool Development: We developed the ACTT to evaluate critical-thinking skills by ranking learners into six critical-thinking stages: 1) unreflective thinker, 2) beginning critical thinker, 3) practicing critical thinker, 4) advanced critical thinker, 5) accomplished critical thinker and 6) challenged thinker. Five educational experts assessed content validity. Part II Tool Evaluation: We retrospectively and randomly assessed 47 long-essay assignments written by 1st-3rd-year anesthesia residents in 2018. The assignments were subsequently blindly reviewed by two researchers to determine the critical-thinking stages. Two evaluators for tested concurrent validity, reliability, and practicality. Results: The ACTT’s items were adjusted for content validity until the item-objective congruence was >0.5. Concurrent validity between the scores and the critical-thinking stages showed moderate correlation by both evaluators (Evaluator 1 rs 0.574, P<0.001; Evaluator 2 rs 0.603, P<0.001). High-performance learners (stages 3, 4 and 5) had significantly higher scores than low-performance learners (stages 1 and 2), with P=0.001. Interrater reliability was strongly correlated (rs 0.866, P<0.001). Both evaluators were satisfied with the tool’s practicality. Conclusion: The ACTT provides a valid, reliable, and practical means of evaluating the critical-thinking skills of anesthesia residents.

Article Details

Section
Original articles

References

Gaba, DM. Dynamic decision-making in anesthesiology: cognitive models and training approaches. In: Evans DA, Patel VL, eds. Advanced models of cognition for medical training and practice. Berlin, Heidelberg: Springer Verlag. 1992;p123-47.

Croskerry P. The theory and practice of clinical decision-making. Can J Anesth. 2005;52:R1-8.

Papp KK, Huang GC, Clabo LML, et al. Milestones of critical thinking: a developmental model for medicine and nursing. Acad Med. 2014;89:715-20.

Huang GC, Newman LR, Schwartzstein RM. Critical thinking in health professions education: summary and consensus statements of the Millennium Conference 2011. Teach Learn Med. 2014;26:95-102.

Huang GC, Lindell D, Jaffe LE, Sullivan AM. A multi-site study of strategies to teach critical thinking: “Why do you think that?”. Med Educ. 2016;50:236-49.

Sharples JM, Oxman AD, Mahtani KR, et al. Critical thinking in healthcare and education. BMJ. 2017;357:j2234.

Victor-Chmil J. Critical thinking versus clinical reasoning versus clinical judgment: differential diagnosis. Nurse Educ. 2013;38:34-6.

Facione NC, Facione PA. Critical thinking and clinical judgement. In: Facione NC, Facione PA, eds. Critical thinking and clinical reasoning in the health sciences: a teaching anthology. Millbrae, CA: California Academic Press. 2008;p1-13.

Facione P. Critical thinking: what it is and why it counts. Insight assessment [monograph on the Internet]. California: Measure Reasons LLC, 2020. [cited 2021 May 26] Available from: https://www.insightassessment.com/wp-content/uploads/ia/pdf/whatwhy.pdf

Hayes MM, Chatterjee S, Schwartzstein RM. Critical thinking in critical care: five strategies to improve teaching and learning in the intensive care unit. Ann Am Thorac Soc. 2017;14:569-75.

Stiegler MP, Tung A. Cognitive processes in anesthesiology decision making. Anesthesiology. 2014;120:204-17.

Gaba DM, Fish KJ, Howard SK, Burden A. Crisis Management in Anesthesiology, 2ed. Philadelphia, PA: Elsevier Saunders, 2015.

Stiegler MP, Gaba DM. Decision-making and cognitive strategies. Simul Healthc. 2015;10:133-8.

Schober P, Boer C, Schwarte LA. Correlation coefficients: appropriate use and interpretation. Anesth Analg. 2018;126:1763-8.

Baker K, Sun H, Harman A, Poon KT, Rathmell JP. Clinical performance scores are independently associated with the American Board of Anesthesiology Certification Examination Scores. Anesth Analg. 2016;122:1992-9.

Carter AG, Creedy DK, Sidebotham M. Evaluation of tools used to measure critical thinking development in nursing and midwifery undergraduate students: a systematic review. Nurse Educ Today. 2015;35:864-74.

Perez EZ, Canut MTL, Pegueroles AF, Llobet MP, Arroyo CM, Merino JR. Critical thinking in nursing: scoping review of the literature. Int J Nurs Pract. 2015;21:820-30.

Carter AG, Creedy DK, Sidebotham M. Development and psychometric testing of the Carter Assessment of Critical Thinking in Midwifery (Preceptor/Mentor version). Midwifery. 2016;34:141-9.

Carter AG, Creedy DK, Sidebotham M. Critical thinking evaluation in reflective writing: Development and testing of Carter Assessment of Critical Thinking in Midwifery (Reflection). Midwifery. 2017;54:73-80.