- 134 views
- 97 downloads
Scaling and Linking Relationships between an International Assessment and a Canadian National Assessment
-
- Author / Creator
- Fung, Karen
-
The education system in Canada is not unified and varies by the provinces/territories. National and International large-scale assessments such as the Pan-Canadian Assessment Program (PCAP) and the Programme for International Student Assessment (PISA), provide an idea on how students in Canada are performing in participating provinces, and Canada as a whole. Although both tests target at students of different age and grade levels, they assess the same core content domains. There are interest in understanding the relationships between PCAP and PISA, and the comparability of their test scores. This study serves as the first study in Canada to establish linking and scaling relationships between the two, by testing two methods of vertical scaling and concordance: concurrent calibration using IRT, and equipercentile with post-smoothing. The study design is also unique as there was no common item between the two tests and the student samples were not equivalent. Relationships between the two tests in regards factors such as frameworks and test specifications were examined, followed by the linking procedures. The contents of mathematics and science were the focus of this study. There were four conditions tested for each procedure based on the content and whether they were the major or a combination of major and minor domains for the two tests linked. Data from PCAP 2010, PCAP 2013, PISA 2012, and PISA 2015 were used to create the four conditions. On evaluations of linking results, means, correlations, and population invariances by language groups of English and French were conducted using graphical comparisons and the calculation of Root Expected Mean Square Difference (REMSD). Results have found that at certain degrees, the two tests were similar in terms of framework and test specification, however categorizing items by subdomains from both tests together as a group would be difficult due to their differences in subdomain definitions and categorizations. Linking results from equipercentile method have suggested different post-smoothing parameters for each condition, and for both linking methods, they tend to agree with each other on their resulting scales. For all conditions, population invariance of language did not hold. English group tended to be closer to the total group than the French group, and there were smaller deviation of the French group for science in comparison to mathematics. Although results of this study seemed inconclusive and should be interpreted with cautions, it established new grounds for future studies. A list of future studies were suggested including the use of propensity score matching to create equivalent groups for linking, and also the suggestion to further test other methods of linking as comparisons.
-
- Graduation date
- Spring 2020
-
- Type of Item
- Thesis
-
- Degree
- Doctor of Philosophy
-
- License
- Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.