The Effects of Cognitive Ability, Motivation and Perception of Interview on the Accuracy and Validity of Think Aloud Interview Data

• Author / Creator

  Lutsyk, Alina

• The objective of the present study was to examine the impact of examinee variables such as cognitive ability, motivation, and perception of the interview on the accuracy and validity evidence of the think aloud interview, a method widely used to collect information on participants’ response processes in assessment validation studies. Think-aloud interviews, which included a series of tasks and questionnaires, were conducted with a sample of 66 undergraduate students in the large research-intensive university. The following research questions were investigated: 1. Do participants’ cognitive ability and motivation levels influence the accuracy of problem solving during the think aloud interview? 2. Do levels of anxiety and comfort influence students’ perception of the think-aloud interview? 3. Do students with more skeptical perceptions of the think aloud interview take less time to solve tasks than students with less skeptical perceptions; and 4. Do students’ anxiety or discomfort as well as attitudes about mistakes influence the accuracy of problem-solving during think-aloud interviews? The results of the present study indicate that cognitive ability and motivation levels did not significantly influence the accuracy of task solutions. Level of anxiety did not have impact on the accuracy either. However, participants’ direct level of comfort and behavioral attitudes towards mistakes were significantly related to the accuracy of task solutions. Participants’ indirect perception of the effectiveness of the interview was also related to completion time and their indirect level of comfort with the think-aloud interview. Finally, participants’ direct perception of the effectiveness of the interview was also related to participants’ external motivation and their direct level of comfort with the think-aloud interview. Implications of the results are discussed for best practices in validation studies.

• Subjects / Keywords

  • validation
  • cognitive ability
  • think-aloud interview
  • motivation
  • perception

• Graduation date

  Spring 2019

• Type of Item

  Thesis

• Degree

  Master of Education

• DOI

  https://doi.org/10.7939/r3-dm53-p786

• License

  Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.