- 108 views
- 186 downloads
Computerized Formative Assessment with the Item Digraph
-
A Graph Theory Approach to Computerized Formative Assessment
-
- Author / Creator
- Varga, Stephanie
-
This paper introduces and tests a new approach I have designed for computerized formative assessment in education. The assessment, called GRAPH-CAT, is developed in the programming language of Python and a simulation is performed to measure its effectiveness. GRAPH-CAT is a cognitively diagnostic computerized adaptive test (CD-CAT) that estimates mastery of an attribute hierarchy using a directed acyclic graph and traditional computer adaptive testing (CAT). GRAPH-CAT reports ability based on both attribute mastery and traditional item response theory (IRT) ability. In this study, a Monte Carlo simulation of student responses is performed using a simulated item bank. Previous CD-CATs have generally not relied on traditional CAT based on the IRT framework. Instead, a single performance measure, such as ๐, is usually replaced by classification into knowledge state as defined by mastered attributes. GRAPH-CAT provides an arguably more robust measure of performance as it is based on both attribute mastery and IRT ability. The introduction of the item digraph realizes the potential held between the connectivity of attributes to create an efficient test. A strength of GRAPH-CAT is that it departs from the CD-CAT reliance on stochastic item administration. It is found that GRAPH-CAT is able to estimate attribute mastery with 92% accuracy in twenty items with a standard error of 0.33 and achieves 83% accuracy in ten items with a standard error of 0.38. These results demonstrate how ordering items with an item digraph may help provide the needed structure for item administration in CD-CAT.
-
- Subjects / Keywords
-
- Graduation date
- Fall 2019
-
- Type of Item
- Thesis
-
- Degree
- Master of Education
-
- License
- Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.