Developing a Framework and Demonstrating a Systematic Process for Generating Medical Test Items

  • Author / Creator
    Lai, Hollis
  • Automatic item generation (AIG) is a field of research dedicated to the production of test items using computer technology. Despite dramatic developments on AIG in the past decade, there is little documentation on a general methodology for creating the models needed to generate items. The purpose of my dissertation is to introduce a three stage framework to guide the process of item generation and to present a proof-of-concept application demonstrating how items can be generated using this framework. The generative framework involves individual stages of development: cognitive modeling, item modeling and item generation. My unique contribution to the literature is threefold. First, I present a modeling approach for extracting knowledge from content experts that can be used for item generation. Second, I present a template-based item generation technique named n-layer modeling to minimize text similarity between generated items from the same model. Third, I present a method for evaluating text similarity for generated items. These methods and procedures are demonstrated in the context of medical education, specifically in the area of surgery. Results generated test items in the domain of hernia and post-operative fever. The application of n-layer item modeling demonstrated a generation technique that can produce more test items, and test items with less text similarity. By integrating test development processes with technology, the generation framework proposed in this study can combine a systematic and iterative process with the humanistic task of providing content expert knowledge to produce test items en masse for meeting current educational testing demands.  

  • Subjects / Keywords
  • Graduation date
  • Type of Item
  • Degree
    Doctor of Philosophy
  • DOI
  • License
    This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
  • Language
  • Institution
    University of Alberta
  • Degree level
  • Department
  • Specialization
    • Measurement, Evaluation and Cognition
  • Supervisor / co-supervisor and their department(s)
  • Examining committee members and their departments
    • Carbonaro, Michael (Educational Psychology)
    • Buck, George (Educational Psychology)
    • Leighton, Jacqueline (Educational Psychology)
    • Stroulia, Eleni (Computer Science)