AI-Marking Assistant

A Web-Based Application for Human-in-the-loop GAI Assisted Assessment Marking and Feedback

Abstract

This paper introduces AI-Marking Assistant (AI-MA), a prototype application that aims to improve the efficiency and consistency of grading and grading feedback by allowing educators to integrate Generative Artificial Intelligence (GAI) assistance into the grading process. While automated grading has the advantages over traditional human marking of being efficient, timely, consistent, scalable, and objective, there are also known limitations and potential issues associated with process. Grading and feedback can lack the nuance and context that would normally come from an expert marker. Results can also be biased by the training data and there are significant ethical and legal implications of allowing a machine to grade assignments without human oversight.  AI-MA aims to overcome these limitations by offering a human-in-the-loop GAI assisted interface that allows educators to leverage the power of GAI while keeping an active oversight and interactive role in the marking process. AI-MA allows human graders to manually mark assignments, or edit the output of a GAI model with results fed back to the model in order to improve its performance. A pilot study with the application demonstrates its potential to significantly improve grader performance while maintaining the quality of marking and grade feedback for students.