Response Time as a Predictor of Test Performance: Assessing the Value of Examinees' Response Time Profiles

• Author / Creator

  Tan, Bin

• Response time data has gained extensive attention in recent years, thanks to the increasing use of computer-based assessments. Previous studies examined the relationship between response time and test performance, yielding inconsistent findings such as positive, negative, and no relationship. To comprehensively examine the complex relationship between examinees’ response times and test performance, this study employs profile analysis to analyze assessment data from the Problem Solving and Inquiry tasks (PSI tasks) in the Trends in International Mathematics and Science Study (TIMSS) 2019 for grade-four mathematics. In the assessment, there were 29 items distributed across 17 screens, with response time recorded for each screen. The data used included responses from 27,682 fourth-grade examinees from 36 countries, including six benchmarking participants. The results of this study show that examinees’ standardized response time varied throughout the test. Furthermore, the study shows the predictive power of screen response time for test performance, with varying strengths and directions of the regression coefficients for different screens. Notably, considering separate response times for individual screens and accounting for within-examinee variability in response time provide more accurate predictions of test scores than relying on total response time or the average response time per screen. Additionally, the analysis uncovers that the relationship between total or separate response times and test performance is also distinct across different achievement groups. Low achievers exhibit a stronger positive correlation between response time and performance, while advanced achievers show a negative correlation. Moreover, the study reveals the influence of item position and item difficulty on response time patterns and their relationship with test performance. These findings contribute to advancing our understanding of the relationship between response time and test performance, with implications for the design and administration of future educational assessments.

• Subjects / Keywords

  • Response time
  • Test Performance
  • PSI tasks
  • Profile analysis
  • Patten analysis

• Graduation date

  Spring 2024

• Type of Item

  Thesis

• Degree

  Master of Education

• DOI

  https://doi.org/10.7939/r3-008n-2137

• License

  This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.