Artificial Intelligence and the Assessment of Learning
One focus for artificial intelligence (AI) in education has been the assessment of students. Significant investments were made in AI assessment tools and development, with US$166 million invested in 2022 alone — and assessment is the focus for more than 15% of EdTech companies. Here we summarize current developments.
Assessment at scale
AI has been used to support student learning and assessment for many years. Using papers first graded by instructors, AI grading systems learn how assessment rubrics are applied and can then apply these rubrics at scale. Just 25 marked papers can lead an AI system to assess 10,000 accurately. AI systems can do this for a variety of forms of assessments, from mathematical problem solving to short or long essays to multiple choice.
These tools can also assess the same assignment in a variety of languages without being biased by external factors such as handwriting, culture or the language the student chose. However, educators should understand that these systems do not actually “read” students’ work. Instead, they use algorithms to analyze syntax and other features of the text. As a result, shorter and simpler texts may be scored more accurately than longer and more complex ones, and any biases inherent in the algorithms are reflected in the assessment.
There are many such scoring systems. The challenge for educators is affordability, training in the use of these systems and the time required to train the system to grade appropriately.
Automated item generation
One challenge for instructors is to design an assessment instrument, whether it be a formative assessment intended to aid learning or a summative assessment intended to provide a grade for learning. AI can generate items to be used in tests, examinations and assessments. The instructor creates an item model, showing the system what an ideal test or set of tests looks like, and then the system generates samples that can be refined.
For example, if an assessment typically has ten items — four on key knowledge, three testing the knowledge in use and three on alternative approaches to problem-solving — then the AI system replicates this approach in each sample test it creates. Once the AI “samples” are confirmed or modified by the instructor, the AI system then generates thousands of versions of the test.
Because of the volume of equivalent tests created, students can move from periodic assessments to assessments on demand. Rather than sit an exam on a specific date, they can ask to be tested at any time. Their test differs from others in terms of the specific items, but the knowledge and understanding under review is identical in every case.
AI and peer-to-peer assessment
One aspect of assessment practices, that has grown significantly in the past decade, is the use of software to support peer-to-peer assessment. Services such as Kritic, Peerceptiv® and Teammates all make use of smart technologies and AI to facilitate fair and appropriate peer assessment.
They do so by:
- Offering guidelines and help during individual reviews to help assessors provide better feedback to fellow students.
- Integrating probabilistic and text analysis inference models to improve the accuracy of the assigned grades, removing bias and “trade-off” deals between students.
- Developing feedback on review strategies that enable peer assessors to review each other’s work.
- Employing a spot-checking mechanism to help instructors optimally oversee the peer assessment process for consistency across a group of assessors.
AI and project-based learning and assessment
Effective project-based learning demonstrates learning in action: applying knowledge, capabilities and skills to a project and using the project experience to enrich learning. Ideally, knowledge, skills and capabilities should be assessed before the project starts, during the project (several times) and at the end so learning gains can be mapped.
AI products such as ChatGPT or other forms of chatbot can be used to support the student experience — the system can provide suggestions on how to complete a task, locate skills development videos or tools and provide resources that can support project work 24/7.
AI can also be used to generate appropriate assessment tools. AI-supported systems such as Valid-8 can support the assessment of competencies, demonstrated by evidence submitted in a variety of formats: video, audio or text or some combination of these. By using AI to automate cross-referencing to competency statements and task-based evidence, Valid-8 can accelerate the completion of legally defensible competency tasks.
Supporting inclusion
A significant opportunity for AI-enabled supports for learning and assessment is to provide support for learners with exceptionalities.
- Partially sighted students can convert text to speech and speech to text, enabling them to undertake assessments created for sighted persons.
- Text or audio can be translated into any language. For a learner whose first language is not the same as the language of instruction, an assessment can be translated instantly from one language to another, and responses written in their own language can be assessed as if written in the language of instruction.
- Video captioning, generated automatically, can allow someone who is a deaf or hard of hearing to fully understand a Zoom session or face-to-face interaction.
- For students with speech impediments, Voiceitt picks up speakers’ unique speech patterns, recognizes any mispronunciations, and normalizes speech before creating an output of audio or text.
- Students who use sign language can now use an AI-supported skill on Amazon Alexa that can convert their signing to speech, enabling them to be understood by those who do not have signing skills.
This field is constantly evolving and there have been significant AI advancements to support students with exceptionalities. Many people in the AI development community are committed to equity and inclusion and to the full engagement of these students in their learning.
Checking for cheating
As AI reviews a student’s work, it can automatically check for plagiarism and other forms of cheating (e.g., two or more students in the same cohort providing identical answers). AI systems can also compare handwriting samples from a student’s past work with their examination submission to verify the student who took the exam is the same person who submitted assignments during the course.
AI proctoring systems such as Examonline, ProctorEdu or Examroom can also use biometrics — facial recognition and fingerprint recognition — as well as writing forensics to determine whether the student sitting an exam is in fact the person they are supposed to be. Such systems also monitor activity in the space the student is using to make sure they are not cheating.
What’s next?
Given the growth of the AI-enabled tools described here, we can expect to see more assessment-only credentials such as those offered by the University of Wisconsin, more deployment of assessment on demand and more use of competency-based assessment. Although some educators may be more concerned about academic misconduct, others will use the emergence of highly functioning AI systems to improve how students are assessed.
