Automated Essay Scoring Framework for a Multilingual Medical Licensing Examination

  • Author / Creator
    Latifi, Syed Muhammad Fahad
  • Automated essay scoring (AES) is a technology that efficiently and economically score written responses by emulating intelligence of human scorer. Present study had employed open-source Natural Language Processing technologies for developing AES framework, to score multilingual medical licensing examination. English, French, and translated-French responses of constructed-response items were scored automatically, and the strength of multilingual automated scoring framework were evaluated in relation to human scoring. Machine-translation was also contextualized for raising AES performance, when restricted sample size counters the performance of AES software. Specific feature extraction and model building strategies resulted in high concordance between AES and human scoring, with average maximum human-machine accuracy of 95.7%, which was in almost perfect agreement with human markers. Results also revealed that the machine-translator had raised predictive consistency but negatively influenced the predictive accuracy. Implications of results for practice, as well as directions for future research are also presented.

  • Subjects / Keywords
  • Graduation date
  • Type of Item
  • Degree
    Master of Education
  • DOI
  • License
    This thesis is made available by the University of Alberta Libraries with permission of the copyright owner solely for non-commercial purposes. This thesis, or any portion thereof, may not otherwise be copied or reproduced without the written consent of the copyright owner, except to the extent permitted by Canadian copyright law.
  • Language
  • Institution
    University of Alberta
  • Degree level
  • Department
    • Department of Educational Psychology
  • Specialization
    • Measurement, Evaluation and Cognition
  • Supervisor / co-supervisor and their department(s)
    • Dr. Mark J. Gierl, Department of Educational Psychology
  • Examining committee members and their departments
    • Dr. Damien Cormier, Department of Educational Psychology
    • Dr. Hollis Lai, Undergraduate Medical Education, Faculty of Medicine & Dentistry