Developing Student Skills to Assess AI as a Content Provider

Challenge

The genie is already out of the bottle when it comes to Generative Artificial Intelligence (GenAI) in education, according to Professor Lynn Kabaroff, Program Coordinator, Fitness and Health Promotion at Cambrian College in Sudbury, Ontario. Some students began using GenAI for assignments soon after it became available November 2023. Others were interested in its applications but weren’t sure how and when to use AI, while a third group hesitated due to concerns about permitted usage and potential cheating. As an educator, Professor Kabaroff wanted to equip her students with the skills to adopt AI responsibly and effectively in both their academic and professional lives. She also sought to mitigate the risks of inappropriate AI usage, particularly in the fitness and health promotion fields, where the adoption of inappropriate exercise programs could have negative consequences.

Experimentation

In a Resistance Training course, students previously worked in groups to create a one-week exercise program for beginners, using professional association guidelines and sample client scenarios. With the advent of AI-generated exercise programs, Professor Kabaroff recognized the need to familiarize students with these tools and develop their ability to critically assess AI outputs to ensure an appropriate match of exercises with individuals’ needs and capacities.

To address this, she restructured her initial assignment to incorporate AI. She encountered a wide range of student comfort with AI. To ease this transition and motivate students, she began by emphasizing the importance of AI literacy for their future careers and clarifying that the use of AI in this assignment was acceptable within Cambrian College guidelines.

The revised assignment required groups of students to input scenarios in AI, each describing a specific client, including age, gender, occupation and other characteristics. The group would add prompts to outline the expected results. Their next task was to analyze the program generated by AI. At the analysis stage, Professor Kabaroff encouraged the students to ask questions and request clarifications and enhancements to the suggested programs to get the best results. Several groups worked effectively with AI to make improvements. For example, they recognized weaknesses in recommended exercise programs, such as requiring five days a week of exercise for people in high-stress jobs, and so asked AI for more appropriate scheduling. Other students accepted the first draft without modification. In-class discussion highlighted the importance of not settling for the initial results and encouraged a more demanding approach.

A similar assignment in a Therapeutic Exercise course required student groups to use AI to create a suitable program for a client needing therapy. Again, some groups of students were completely satisfied with the initial AI results and had no questions or changes to suggest. When students then compared their results with established rehabilitation training plans created by industry professionals, it was obvious there were mistakes and gaps in the AI programs. During the in-class debriefing on the assignment, Professor Kabaroff stressed the necessity of questions and of more critical analysis of AI results. She subsequently required students to ask AI at least two questions when completing this assignment, fostering a more critical engagement with the content and the technology.

Results

As students progressed through the Resistance Training course, Professor Kabaroff expanded the exercise to include more advanced scenarios. She redistributed the first AI assignment of the one-week exercise program to use as a starting point. This time the students were to ask AI for a one-month exercise program for a client with six months of workout experience. Professor Kabaroff also provided a list of guiding questions to help the student critically evaluate and improve AI’s recommendations.

This iteration saw students engage with greater enthusiasm and awareness. Discussions revealed a deeper understanding of AI’s strengths and limitations, as well as the value of critical inquiry in achieving quality results.

Next Improvement Steps

Building on these experiences, Professor Kabaroff plans to have students examine the sources cited by AI to create the exercise programs, verifying that AI quoted the information from the source accurately and in full. This approach has multiple benefits:

Familiarize students with diverse exercise program sources
Highlight the importance of precise and complete exercise descriptions
Teach students to recognize AI’s limitations as a reliable source

Another professor who used the assignment in the Resistance Exercise course encountered AI’s tendency to produce similar exercise programs for varied client profiles. This limitation is not obvious to students as they do not see each other’s programs but has significant implications for quality applications of AI in the field. Professor Kabaroff is considering ways of illustrating this to students as a weakness in over-reliance on AI. Part of the challenge is finding time in the class schedule to introduce additional content.

Having already modified her AI assignments based on experience and feedback, she predicts she will continue to make changes three or four more times before she is satisfied with the result.

Potential

Professor Kabaroff is introducing AI in her teaching to respond to how AI impacts the need for memorization, a previously essential component of teaching and learning. The availability of AI and computer searches means that when a student is not sure of, for example, how many repetitions of an exercise are best in a particular situation, they can readily look it up. The focus is now on content management, with optimal requesting and assessment of AI results, finding reliable sources, critically evaluating their accuracy and applying them effectively. This approach balances recognizing AI’s potential with caution against uncritical acceptance.

The Vice-President, Academic, at Cambrian College took steps to address AI’s broader implications by establishing the Academic Integrity and Artificial Intelligence Working Group to address challenges and concerns about the use of GenAI in academic settings. The ad-hoc group generated actionable recommendations to help mitigate these concerns.

Their efforts have laid the groundwork for policy and educational development and resulted in the creation of a dedicated Academic Integrity Office to coordinate the institution’s approach to academic integrity.