ERA Banner
Download Add to Cart Share
More Like This
  • http://hdl.handle.net/10402/era.29427
  • Evaluating the Performance of the Uncorrected and Corrected Reliability Alpha for Range Restriction and the Confidence Intervals in a Single and Meta-Analytic Study
  • Li, Johnson C. H.
  • English
  • Reliability
    Range restriction
    Meta-analysis
  • Nov 29, 2012 3:35 PM
  • Thesis
  • English
  • Adobe PDF
  • 1732639 bytes
  • Range restriction has long been a methodological problem in educational and psychological research (Hunter & Schmidt, 2004), and this usually leads to a downward-biased estimate of a statistic. Even though much research has examined the performance of Pearson’s correlation under range restriction in both single and meta-analytic studies (e.g., Li, Chan, & Cui, 2011a), the assessment of reliability coefficients (e.g., coefficient alpha) under range restriction is relatively limited. Regarding a single study, Fife, Mendoza, and Terry’s (2012) have recently examined the performance of the uncorrected and bias-corrected coefficient alpha; as an extension, the performance of the confidence intervals (CIs) and widths also need to be examined. Regarding a meta-analytic study, Rodriguez and Maeda (2006) have proposed a framework for conducting a meta-analysis of coefficient alpha; as an extension, the accuracy of the bias-corrected mean alpha as well as the associated CIs also need to be evaluated. In light of these unexamined issues, this dissertation sought to evaluate the performance of the uncorrected and bias-corrected alphas—as well as their CI—in both single and meta-analytic study research situations. This provides a comprehensive assessment of reliability under range restriction, thereby providing guidelines about the treatment of biases that come from range restriction. The Monte Carlo results showed that the uncorrected alpha suffered as a function of the selection ratio and the correlation between the test and the selection variable in both single and meta-analytic studies. By contrast, the bias-corrected alpha could adjust for the bias appropriately. Moreover, the bootstrap CIs constructed for the bias-adjusted alpha in both single and meta-analytic studies were generally accurate across different simulation conditions, including sample size, item number, etc. Application of the correction procedure and CI construction in a real study is demonstrated. Based on these results, conclusions, discussions, and recommendations are also presented.
  • Doctoral
  • Doctor of Philosophy
  • Department of Educational Psychology
  • Measurement, Evaluation and Cognition
  • Spring 2013
  • Cui, Ying (Educational Psychology)
  • Cui, Ying (Educational Psychology)
    Gierl, Mark (Educational Psychology)
    Poth, Cheryl (Educational Psychology)
    Abbott, Marilyn (Educational Psychology)
    Mou, Weimin (Psychology)
    Cheung, Mike (Psychology)