Developing a Framework and Demonstrating a Systematic Process for Generating Medical Test Items

• Author / Creator

  Lai, Hollis

• Automatic item generation (AIG) is a field of research dedicated to the production of test items using computer technology. Despite dramatic developments on AIG in the past decade, there is little documentation on a general methodology for creating the models needed to generate items. The purpose of my dissertation is to introduce a three stage framework to guide the process of item generation and to present a proof-of-concept application demonstrating how items can be generated using this framework. The generative framework involves individual stages of development: cognitive modeling, item modeling and item generation. My unique contribution to the literature is threefold. First, I present a modeling approach for extracting knowledge from content experts that can be used for item generation. Second, I present a template-based item generation technique named n-layer modeling to minimize text similarity between generated items from the same model. Third, I present a method for evaluating text similarity for generated items. These methods and procedures are demonstrated in the context of medical education, specifically in the area of surgery. Results generated test items in the domain of hernia and post-operative fever. The application of n-layer item modeling demonstrated a generation technique that can produce more test items, and test items with less text similarity. By integrating test development processes with technology, the generation framework proposed in this study can combine a systematic and iterative process with the humanistic task of providing content expert knowledge to produce test items en masse for meeting current educational testing demands.

   

• Subjects / Keywords

  • Educational Measurement
  • Test Development
  • Automatic Item Generation

• Graduation date

  Spring 2013

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/R3C93H

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.