Making Diagnostic Inferences about Student Performance on the Alberta Education Diagnostic Mathematics Project: An Application of the Attribute Hierarchy Method

  • Author / Creator
    Alves, Cecilia
  • Cognitive diagnostic assessments (CDA) is an approach where the psychology of learning is combined with methods and models in statistics for the purpose of making inferences about students’ specific knowledge structures and processing skills. This study used a four-step principled approach to test design characterized by: (1) the development of cognitive models, (2) the construction of test items according to the knowledge and skills specified in the cognitive model, (3) the use of a diagnostic psychometric analysis to assess the plausibility of the underlying cognitive model and to providestudents’ attribute probability estimates, and (4) the creation of detailed score reports that map examinees’ mastery levels to provide more detailed information about students’ problem-solving strengths and weaknesses.Being among the first applications of the AHM to non-retrofit data from an operational testing program, the findings of this study add substantially to our understanding of the necessity of a principled approach to assessment design, and also contribute to a growing body of literature on CDA. Results of this study revealed that cognitive models adequately fit the data for the total sample of students; however, the fit for the observed and expected response data differed for high and low ability students.The average attribute probability estimates were ordered, as expected, from least to most difficult. In addition, the ordering of the attributes did not differ as function of the performance level of the students and the correlational pattern of the probability estimates indicated both convergent and discriminant evidence supporting the hierarchical structure of attributes. Concerning the reliability of the models, all six attributes in Subtracting 2-digit numerals produced consistent interpretations about the mastery of attributes, whereas Comparing and ordering numbers, only the decisions made for Attribute 1 were found to be consistent. Limitations of the study and recommendations for future research were also discussed.

  • Subjects / Keywords
  • Graduation date
    Spring 2012
  • Type of Item
  • Degree
    Doctor of Philosophy
  • DOI
  • License
    This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
  • Language
  • Institution
    University of Alberta
  • Degree level
  • Department
  • Supervisor / co-supervisor and their department(s)
  • Examining committee members and their departments
    • Gokiert, Rebecca (Extension)
    • Frenette, Eric (Universite Laval)
    • Buck, George (Educational Psychology)
    • Leighton, Jacqueline (Educational Psychology)
    • Cui, Ying (Educational Psychology)