Using Automatic Item Generation to Create Content for Computerized Formative Assessment

• Author / Creator

  Zhang Xinxin

• Computerized formative assessments (CFA) are becoming popular in post-secondary education because they can offer immediate feedback and on-demand testing. This has created a pressing need for large numbers of content-specified items together with their feedback so that the items may be available continuously for formative assessment. To support CFA in post-secondary education, I present a modified generation framework to address the operational difficulties that arise when applying existing frameworks. I also evaluate the application of the modified framework for item generation within the context of higher education. The proposed framework incorporates a human-machine interactive approach and employs a tree structure for cognitive modeling as well as introduces a new mechanism to assemble elements and a validation tool. The new framework was applied within the university teacher education field. Specifically, two instructors of EDPY 303 (Educational Assessment) used HIM_AIG, which is a software program developed based on the modified framework, to implement generation tasks for quizzes used in the course. Six experts in higher education evaluated the generated content using the statements that have achieved the greatest consensus among published item-writing and feedback-writing guidelines/recommendations. One hundred and thirty-four students took the practice quizzes, which contained the generated items. The results suggest that this modified approach is feasible; the evaluative quality of the generated items and feedback is comparable with that of the parent ones; the psychometric quality of the generated items satisfies the standard. Implications of the study and limitations of this research are also presented.

• Subjects / Keywords

  • Automatic Item Generation

• Graduation date

  Fall 2019

• Type of Item

  Thesis

• Degree

  Doctor of Philosophy

• DOI

  https://doi.org/10.7939/r3-vbt3-6598

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.