The Impact of Item Writer Training on Item Statistics of Multiple-Choice Items for Medical Student Examination

Abstract

Background: Multiple-choice question (MCQ) item writer training is an important step towards the production of high quality tests. Educators generally suggested faculty development programs of long duration, which are not practical for many clinical teachers. This study explored an alternative approach, using a series of short workshops.

Objective: This study examined (1) the perception of clinical teachers on item writer training workshops, and (2) the impact of training on item difficulty and discrimination.

Methods: A series of three short workshops on MCQ item development and item analysis were administered. Participating clinical teachers were asked to provide satisfactory ratings of the workshops. Items developed for comprehensive examina- tions of fifth-year medical students were analyzed, comparing item difficulty and item discrimination between those written by workshop participants and non-participants, both before and after the workshops.

Results: Participants were very satisfied with the workshops. The items developed by participants before the workshop tended to be inappropriately too easy or difficult than those developed by non-participants, but showed similar discriminating power with items of non-participants. After the workshops, participants developed test items that were similarly appropriate in the difficulty level as compared with items of non-participants. Post-workshop items of participants had higher discriminating power than those of non-participants.

Conclusion: A series of three short workshops on item writer training is an effective and acceptable program for clinical teachers. It has a positive impact on test statistics, improving item quality both in their difficulty and discrimination.