Download the full-sized PDF of Developing a Framework and Demonstrating a Systematic Process for Generating Medical Test ItemsDownload the full-sized PDF



Permanent link (DOI):


Export to: EndNote  |  Zotero  |  Mendeley


This file is in the following communities:

Graduate Studies and Research, Faculty of


This file is in the following collections:

Theses and Dissertations

Developing a Framework and Demonstrating a Systematic Process for Generating Medical Test Items Open Access


Other title
Automatic Item Generation
Test Development
Educational Measurement
Type of item
Degree grantor
University of Alberta
Author or creator
Lai, Hollis
Supervisor and department
Gierl, Mark (Educational Psychology)
Examining committee member and department
Carbonaro, Michael (Educational Psychology)
Stroulia, Eleni (Computer Science)
Leighton, Jacqueline (Educational Psychology)
Buck, George (Educational Psychology)
Department of Educational Psychology
Measurement, Evaluation and Cognition
Date accepted
Graduation date
Doctor of Philosophy
Degree level
Automatic item generation (AIG) is a field of research dedicated to the production of test items using computer technology. Despite dramatic developments on AIG in the past decade, there is little documentation on a general methodology for creating the models needed to generate items. The purpose of my dissertation is to introduce a three stage framework to guide the process of item generation and to present a proof-of-concept application demonstrating how items can be generated using this framework. The generative framework involves individual stages of development: cognitive modeling, item modeling and item generation. My unique contribution to the literature is threefold. First, I present a modeling approach for extracting knowledge from content experts that can be used for item generation. Second, I present a template-based item generation technique named n-layer modeling to minimize text similarity between generated items from the same model. Third, I present a method for evaluating text similarity for generated items. These methods and procedures are demonstrated in the context of medical education, specifically in the area of surgery. Results generated test items in the domain of hernia and post-operative fever. The application of n-layer item modeling demonstrated a generation technique that can produce more test items, and test items with less text similarity. By integrating test development processes with technology, the generation framework proposed in this study can combine a systematic and iterative process with the humanistic task of providing content expert knowledge to produce test items en masse for meeting current educational testing demands.  
Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.
Citation for previous publication

File Details

Date Uploaded
Date Modified
Audit Status
Audits have not yet been run on this file.
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 1647852
Last modified: 2015:10:12 10:31:41-06:00
Filename: Lai_Hollis_Spring_2013.pdf
Original checksum: b6c6016ea521586b750e132df72b789d
Well formed: true
Valid: true
File author: Hollis Lai
Page count: 180
File language: en-CA
Activity of users you follow
User Activity Date