Validity and Reliability of the Multiple-Mini Interview (MMI) in Medical Student Selection: A Comparison of Onsite and Online Platforms

Cherdsak  Iramaneerat; Punyapat Maprapho; Orawan Uakit

doi:10.33192/smj.v78i11.276829

Authors

Cherdsak Iramaneerat Department of Surgery, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
Punyapat Maprapho Siriraj Health Science Education Excellence Center, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
Orawan Uakit Undergraduate Education Unit, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand

DOI:

https://doi.org/10.33192/smj.v78i11.276829

Keywords:

Medical student selection, Multiple Mini-interview, Validity, Reliability

Abstract

Objective: The objective of this study was to compare the administration of Multiple-Mini Interview (MMI) between the online and onsite platforms, in terms of validity evidence.

Materials and Methods: We retrospectively reviewed records of past MMI scores, examination materials, and participant questionnaire responses over a six-year period at the Faculty of Medicine Siriraj Hospital, Mahidol University. A total of eight MMI administrations were included, four onsite and four online. Validity evidence was assessed based on three key sources: test content, response processes, and internal structure.

Results: Over six years, eight MMIs were conducted, with 237 out of 340 candidates admitted to a medical school. Content analysis of the test specification tables indicated that both onsite and online platforms adequately addressed similar objectives. Participant satisfaction ratings were comparable between onsite and online MMIs. Qualitative analysis revealed minor issues in a few stations regarding clarity of instructions and scoring criteria. Additionally, some online MMI stations showed discrepancies between task time requirements and allotted time, and more technical issues were also reported. Score analysis showed that both highest and average scores from online MMIs were slightly lower than those from onsite MMIs. However, both formats yielded moderately reliable test scores (Cronbach’s Alpha of 0.49 - 0.51).

Conclusion: The online MMI is a viable alternative to the traditional onsite MMI. Both platforms effectively covered the same assessment objectives, yielded comparable reliability and participant satisfaction.

References

Eva KW, Macala C, Fleming B. Twelve tips for constructing a multiple mini-interview. Med Teach. 2019;41(5):510-16.

Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ. 2004;38(3):314-26.

Rees EL, Hawarden AW, Dent G, Hays R, Bates Hassell AB. Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37. Med Teach. 2016;38(5):443-55.

Hecker K, Donnon T, Fuentealba C, Hall D, Illanes O, Morck DW, et al. Assessment of applicants to the veterinary curriculum using a multiple mini-interview method. J Vet Med Educ. 2009;36(2):166-73.

Uijtdehaage S, Doyle L, Parker N. Enhancing the reliability of the multiple mini-interview for selecting prospective health care leaders. Acad Med. 2011;86(8):1032-9.

Sebok SS, Luu K, Klinger DA. Psychometric properties of the multiple mini-interview used for medical admissions: findings from generalizability and Rasch analyses. Adv Health Sci Educ Theory Pract. 2014;19(1):71-84.

Eva KW, Reiter HI, Rosenfeld J, Trinh K, Wood TJ, Norman GR. Association between a medical school admission process using the multiple mini-interview and national licensing examination scores. JAMA. 2012;308(21):2233-40.

Eva KW, Reiter HI, Rosenfeld J, Norman GR. The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad Med. 2004;79(10 Suppl):S40-2.

Reiter HI, Eva KW, Rosenfeld J, Norman GR. Multiple mini-interviews predict clerkship and licensing examination performance. Med Educ. 2007;41(4):378-84.

Dowell J, Lynch B, Till H, Kumwenda B, Husbands A. The multiple mini-interview in the U.K. context: 3 years of experience at Dundee. Med Teach. 2012;34(4):297-304.

McAndrew R, Ellis J. An evaluation of the multiple mini-interview as a selection tool for dental students. Br Dent J. 2012;212(7):331-5.

Thamwiriyakul N, Thamissarakul S, Wannapaschaiyong P. Association between grit and burnout among clinical medical students. Siriraj Med J. 2025; 77(2):175–82.

Tiller D, O'Mara D, Rothnie I, Dunn S, Lee L, Roberts C. Internet-based multiple mini-interviews for candidate selection for graduate entry programmes. Med Educ. 2013;47(8):801-10.

American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association 2014.

Cizek GJ. Sources of validity evidence for educational and psychological tests: A follow-up study. Paper presented at the Annual meeting of the National Council on Measurement in Education, Denver, CO, May 2010.

PASW Statistics for Windows, version 18.0 [program]. Chicago: SPSS Inc., 2009.

Padilla JL, Benítez I. Validity evidence based on response processes. Psicothema. 2014;26(1):136-44.

Rios J, Wells C. Validity evidence based on internal structure. Psicothema. 2014;26(1):108-16.

Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld J, Norman GR. Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ. 2009;43(8):767-75.

Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The Multiple Mini-Interview (MMI) for student selection in health professions training - a systematic review. Med Teach. 2013;35(12):1027-41.