Opportunity
Disenchanted with a textbook publisher’s online homework modules for first-year physics students, Ontario Tech University professor Dr. Joseph MacMillan set out to create his own. He received a teaching innovation grant and with the help of a colleague, Rupinder Brar, and some student assistants, the project began some seven years ago. Today, 12 interactive, multimedia modules covering all topics in his course PHY 1010 Physics I are available.
Dr. MacMillan was motivated to undertake this project as the publisher’s modules allowed students to click through them guessing the answers. He also found little correlation between student module grades and course final grades, which made their use questionable. Cost of the publisher’s modules was another factor. Students had to pay an ancillary course fee to use them, something the Ontario government discourages universities from charging. He avoided having to charge students by building the modules on an open-source platform freely available for anyone to download.
The new modules focus on mastery learning. This approach allows students sufficient time and support to learn concepts almost fully before advancing to new related concepts. It contrasts with traditional learning that limits the available time to learn new concepts, accepting the reality that some students will learn more than others during the allotted time.
Innovation
After researching various platforms to support the mastery learning modules, Dr. MacMillan settled on the open-source learning management and assessment system LON-CAPA. The system originated at Michigan State University and is now used by over 160 institutions worldwide, mainly in natural science courses. Among its many features is the ability to generate random personalized questions, create hints and feedback for questions, and allow the developer to enter questions containing mathematical and scientific formula and graphics. An added attraction of LON-CAPA is it has a large question bank written by other users.
Dr. Macmillan adapted the system to meet his own project requirements. Each of the 12 mastery modules has three levels of complexity. Level 1 has four questions on basic conceptual facts, Level 2 has three straightforward numerical questions, and Level 3 has two complex questions involving multiple concepts. Level 1 and 2 problems are presented as multiple-choice questions, whereas Level 3 questions require students to enter a numerical answer. Level 2 and 3 questions are illustrated with narrated video animations, which Dr. MacMillan believes enhance understanding. An illustration of the structure of a mastery module is below.
Students must answer all questions before discovering if they answered them correctly. If they get them all correct, they can advance to the next level. However, if they make a mistake on even one question, they are presented with another set of different questions. If they fail to answer all correctly after 10 attempts, they are given a score of zero and can move to the next level. Dr. MacMillan designed the system this way on the theory that delaying feedback and paying a high price for being incorrect will encourage students to be more thoughtful about their answers and avoid guessing. He reports very few students ever fail all 10 attempts.
Benefits and Outcomes
Thousands of students use the mastery modules each year and they were thoroughly tested and debugged at this point. They run on any web browser and posed no screen accessibility issues. The LON-CAPA system provides Dr. MacMillan with complete analytics about student responses and performance, which he monitors closely. For example, originally there was no limit on attempts students could make to reach the next level. However, he discovered some students attempting numerous times to generate identical questions so they could share answers. This led to him implementing the 10-attempt limit.
Dr. Macmillan achieved his goal to eliminate student costs. He also found students are spending more time on the modules than previously with the publisher’s exercises, however there is only a moderate correlation between student success on the modules and course grades.
According to Dr. MacMillan, students have mixed feelings about the mastery modules: some recognize right away they are beneficial, while others are less enthusiastic. Below are several sample student comments:
- Mastery modules were a significantly less stressful way of testing my knowledge on the subject. It helped ensure I fully understood the topic and forced me to try to resolve questions I got wrong.
- The mastery modules ensured I was applying myself every week.
- The weekly mastery module helped me get a deeper understanding of the course content.
- Other than the fast pace, there is nothing else I would complain about the course, except of course, the dreaded Mastery Modules. They surely posed a challenge, but when I finally solved them after having episodes of frustration and anxiety, you could just picture me leaping out of my seat and shouting for joy, "Yes! I am victorious."
Dr. MacMillan’s leadership in developing the physics mastery learning modules earned him the Ontario Tech University’s Learning Reimagined Award in 2020.
Challenges and Enhancements
Dr. MacMillan says the mastery modules are running smoothly and only minor tweaks will be necessary if issues arise. He sees the modules as one step in making all course resources, including the textbook, accessible to students at no cost.
Faculty who wish to make use of the mastery modules are welcome to contact Dr. MacMillan. He is willing to freely share his database, but faculty would first need to have LON-CAPA installed on a local server.
Further Information
Dr. Joseph MacMillan
Faculty of Science
Ontario Technical University
Oshawa, Ontario L1G 0C5
Canada
[email protected]
