- 467 views
- 533 downloads
A Neural Network approach to Automated Essay Scoring: A Comparison with the Method of Integrating Deep Language Features using Coh-Metrix
-
- Author / Creator
- Shin, Eunjin
-
Automated essay scoring (AES) has emerged as a secondary or as a sole marker for many high-stakes educational assessments due to remarkable advances in feature engineering using natural language processing, machine learning, and deep learning algorithms. The purpose of the study was to compare the effectiveness and the performance of two AES frameworks, each based on machine learning with deep language features and deep learning algorithms. More specifically, support vector machines (SVMs) in conjunction with Coh-Metrix features were used for a traditional AES model development and the convolutional neural networks (CNNs) approach was used for deep learning model development. Then, the strengths and weaknesses of the models under different circumstances (e.g., types of scoring rubric, length of essay, and essay type) were tested. The results were evaluated using the Quadratic Weighted Kappa (QWK) score and compared with the agreement between the human raters. The results indicated that the CNNs model performs better, producing more comparable results to the human raters than the Coh-Metrix + SVMs model. Moreover, our best models could achieve state-of-the-art performance in most of the essay sets with a high average QWK score.
-
- Graduation date
- Fall 2018
-
- Type of Item
- Thesis
-
- Degree
- Master of Education
-
- License
- Permission is hereby granted to the University of Alberta Libraries to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. Where the thesis is converted to, or otherwise made available in digital form, the University of Alberta will advise potential users of the thesis of these terms. The author reserves all other publication and other rights in association with the copyright in the thesis and, except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatsoever without the author's prior written permission.