Generating and Evaluating AIG Items

• Author(s) / Creator(s)

  • Armoush, Nour.
  • Gorgun, Guher.
  • Wongvorachan, Tarid
  • Gierl, Mark.
  • Bulut, Okan.

• Computer-based testing (CBT) has been growing more popular alongside the advancement of educational technology. As the demand for CBT rises, it is vital that large item banks are also being created. The traditional method of producing items is expensive, time consuming, and lacks scalability. Automatic Item Generation (AIG) is an augmented intelligence approach that combines the expertise of subject matter experts with computational power to efficiently produce large numbers of items. This study demonstrates the application of large language model such as GPT-4 in the AIG process. The generated items were evaluated by content experts. As a result, 77/168 Items were accepted with no revisions required, while 53/168 of the items required minor revisions.

• Date created

  2024-04-20

• Subjects / Keywords

  • Psychometrics
  • Large Language Model
  • Natural Language Processing
  • Automatic Item Generation

• Type of Item

  Conference/Workshop Poster

• DOI

  https://doi.org/10.7939/r3-4wmb-wg60

• License

  Attribution-NonCommercial 4.0 International