ERA

Download the full-sized PDF of Comparison of vertical scaling methods in the context of NCLBDownload the full-sized PDF

Analytics

Share

Permanent link (DOI): https://doi.org/10.7939/R3764M

Download

Export to: EndNote  |  Zotero  |  Mendeley

Communities

This file is in the following communities:

Graduate Studies and Research, Faculty of

Collections

This file is in the following collections:

Theses and Dissertations

Comparison of vertical scaling methods in the context of NCLB Open Access

Descriptions

Other title
Subject/Keyword
Item Response Theory
Decision Accuracy
Vertical Scaling
Decision Consistency
Type of item
Thesis
Degree grantor
University of Alberta
Author or creator
Gotzmann, Andrea Julie
Supervisor and department
Rogers, W. Todd (Educational Psychology)
Gierl, Mark J. (Educational Psychology)
Examining committee member and department
Cui, Ying (Educational Psychology)
Childs, Ruth (Human Development and Applied Psychology)
Abbott, Marilyn (Educational Psychology)
Hayduk, Les (Sociology)
Department
Department of Educational Psychology
Specialization

Date accepted
2011-08-15T17:26:47Z
Graduation date
2011-11
Degree
Doctor of Philosophy
Degree level
Doctoral
Abstract
Vertical scaling is the process of establishing a numerical test score scale across several age or grade levels. Given that the current literature does not indicate which of the different vertical scaling procedure works “best” for all situations. This study evaluated the performance of four vertical scaling procedures (concurrent calibration, fixed common item parameters, test characteristic curve, and hybrid characteristic curve), across two content areas (Reading and Mathematics), two score distribution types (normal and negatively skewed), and two sample sizes (1,500 and 3,000). Five outcome measures were used to evaluate the results: decision accuracy, decision consistency, conditional standard errors at each of two cut-scores, root-mean-squared-differences of the scale scores between scaling procedures, and correlations between scaling procedures’ final item parameters. The data used in this study was from a U.S. large scale testing program in Reading and Mathematics for grades 3 through 8. These data were used to simulate the type of score distribution and sample sizes considered with 100 replicates for these combinations. The largest differences among the four vertical scaling procedures for Reading were found at the lower and upper grade levels, particularly for decision accuracy. Differences were found between the normal and skewed distributions, for decision accuracy where a different pattern of results were found. The accuracy results decreased markedly as grades increased for the skewed distribution. For Mathematics the largest differences across all outcome measures occurred across grade levels rather than across vertical scaling procedures. Sample size for both Reading and Mathematics did not seem to have an effect. Practitioners should ensure high decision accuracy and consistency values across all grade levels, and that a particular scaling procedure does not result in undesirable results. If a state program allows different procedures for different content areas, then the hybrid characteristic curve procedure would be most appropriate for Reading and the test characteristic procedure most appropriate for Mathematics. However, if the procedure must be the same, then the hybrid characteristic curve procedure could be used for both Reading and Mathematics. Measurement specialists can use these results to guide their implementation of vertical scaling for their state assessment programs.
Language
English
DOI
doi:10.7939/R3764M
Rights
Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.
Citation for previous publication

File Details

Date Uploaded
Date Modified
2014-04-24T21:58:51.167+00:00
Audit Status
Audits have not yet been run on this file.
Characterization
File format: pdf (Portable Document Format)
Mime type: application/pdf
File size: 1077141
Last modified: 2015:10:12 11:36:48-06:00
Filename: Gotzmann_Andrea_Fall 2011.pdf
Original checksum: e37be484b5b347166fbdbb3f8ff2ff27
Well formed: true
Valid: false
Status message: Invalid page tree node offset=57442
Status message: Outlines contain recursive references.
File title: ABSTRACT
Activity of users you follow
User Activity Date