Opportunity
Over a five-year period, Svend Andreas Horgen, an Assistant Professor in the Institute for Informatics and e-Learning at the Norwegian University of Science and Technology (NTNU) in Trondheim, Norway, attended numerous conferences and visited multiple schools to talk to teachers about how to use information and communications technology effectively in education and learning.
Building from this experience, he worked with colleagues to develop an online course on using technology for effective teaching but, as it was necessary to pay for registration, very few teachers enrolled. There are about 90,000 teachers in Norway, but only about 100 participated in the online course.
In order to reach a wider audience, it was decided to develop a Massive Open Online Course (MOOC), which would be free for all registrants. In 2013, NTNU received one million kroner (about $150,000 CAD) to create the MOOC, in addition to building knowledge, experience, and a model for MOOCs in Norway.
Innovation
The first decision of the development team working on the MOOC was to discard the materials developed for the original online course, but to take advantage of the knowledge and experience gained in its development and delivery.
Developing the MOOC: Concurrent e-Learning Design is used as the design method to take advantage of a wide range of expertise and participation (More information on this design method is available by reading Collaborative International Development of a Massive Open Online Course at the Norwegian University of Science and Technology). In concurrent design, experts work together on outlining all aspects of a course – pedagogy, content, administration, technology, delivery, assessment, and quality measures. Scalability was also an important feature for the MOOC design.
The team members worked over four months, with frequent online meetings, to develop a 40-page description of all components of the MOOC. The MOOC was designed for teachers in secondary schools, as well as higher education.
Building in Flexibility: Most courses, whether in-class or online, have a traditional structure with set enrolment, start, stop, and exam dates. This MOOC was designed to offer participants flexibility in those administrative, as well as many other, components.
Learning Materials: All the learning materials are immediately made available to students when they enrol, through the LMS Canvas, so they can work through them at their own speed. The materials include short lectures, videos, TED talks, readings, blogs, slides, a wide variety of exercises, and a series of short interviews in which professors and teachers describe their uses of particular technologies.
All the content provides theory and demonstrates options for technological applications to teaching and learning. For example, in the module on formative assessment, both pedagogical and online theory, in addition to a reflective exercise, are used to look at how new tools available online can influence how the MOOC participants do assessment in their classes. There are also a number of self-administered multiple choice assessments, so students can assess their own understanding of the module.
Design: The MOOC is divided into 15 modules, in three sets of five, with each followed by a final exam. Each module is divided into multiple segments, containing materials suited to the content. For example, Module 1, An Introduction to Information and Communications Technology, contains 13 parts, with lectures and demonstrations and interviews to convey the basics of “ICT in learning” and to motivate student participation. The LMS keeps track of how far participants have progressed in each module so they are returned to where they finished during their last session.
Integrating Student Content: In each module, below the text and videos, is a section for student discussion, integrated into the main content rather than separated into a discussion forum. The comments include those from former students (since 2014) as a way of building a sense of online community, even with students studying at different times. Students are able to inform the professors if they think one of the comments is inappropriate and the professors consider its deletion. This has not been an issue to this date. An introductory video outlines what types of comments are encouraged.
Formal Enrolment: After the participants complete the first three of five modules in the first section of the course, they are offered the opportunity to formally enrol as a student at NTNU. Formally registered participants can receive credits from NTNU. It was determined if they worked through the materials and assignments to that point, they could be considered a serious student. This continuous enrolment created a challenge for the university administration, but was solved by batching the participants and processing them in groups, regardless of registration dates as a background process invisible to the students.
Flexible Exams: Svend Andreas Horgen, working with his colleague Geir Maribu, agreed they “could grade exams every day if they did it cleverly”, thereby offering students flexibility and very quick response times. The final exam for each group of five modules requires the students to submit a five-minute video in response to questions distributed in advance. The students produce a video to talk about what they learned in the modules, discussing the theory and practical information and their influence on their own practice as teachers. To present this effectively in a five-minute video requires considerable preparation from the students. The professors can then use short break periods during their work days to view a video, and prepare two- to four-minute video responses. This video feedback strengthens relations between students and professors, and is designed to be motivating.
Peer Assessment: Many of the modules include reflective exercises the students can complete through writing, video, slides – whatever is best for them. Professors Horgen and Maribu do not have the time to assess each of these, so they set up a process for peer assessment, even though the participants are studying at their own pace. Each day, they check Canvas, the LMS, to see if two or three students submitted the same exercise. If so, they use Canvas to set up connections so the students review each others’ work according to criteria established by the professors.
When participants registered to receive credit, their work in completing the exercises in each module and in participating in the peer assessment is assessed by the professors as part of the final assessment.
Outcomes and Benefits
There are about 2,000 students currently registered in the MOOC, and Professor Horgen sees a steady increase in the level of activity. The institution’s interest is in completion as for each MOOC completion, NTNU receives a fee from the government; his main priority is learning, which also implies MOOC completion.
Student feedback is very positive – with 90% at levels 4 or 5 on the five-point satisfaction scale. Many commented that the MOOC transformed the way they teach. All aspects of the MOOC – delivery, discussions, variety of materials and assessment, and peer review - received positive comments.
Concurrent e-Learning Design involves experts with different competencies in a thorough analysis and design process interconnecting all aspects of the course. Although it takes extra time, the process results in a coherent and cohesive course.
The designers found Canvas was not ideal for the presentation of large amounts of learning materials. Instead of working within the limitations of one particular LMS, the designers determined what they wanted to do with the MOOC and then found the systems that would support those choices. As a result, they developed a web-based system for presenting learning material in a better way than Canvas can do. The downside of this approach is students are brought outside of Canvas as they work with the learning materials. However, students are used to navigating between different websites and web-based systems in their work and studies, so they have not encountered problems.
The video exams provide two key advantages - quick turnaround time and the establishment of a relationship between professors and participants through direct, individual responses. Exams after each five modules were set up as motivators, with a sense of accomplishment throughout the MOOC. The participants also get messages from the professors after each final exam to encourage them to complete the course.
The peer assessment expectations are outlined by professors and those receiving the comments from their fellow students are also asked to provide feedback on how useful it was and whether it has impacted their thinking or approach. Peer review offers student-to-student communication and the possibility of a community of practice, even in an asynchronous MOOC.
When participants registered to receive credit for the MOOC, the professors confirm they completed the peer reviews, as well as the exercises in each module.
Challenges and Enhancements
As is often the case, the greatest strength of the MOOC – its flexibility – can also be a disadvantage as students move at their own pace – often leaving months between the completion of modules. The lack of firm due dates can make the MOOC a low priority in busy lives. Individual e-mails are regularly sent to students to encourage their progress, but this is a manual and time consuming process due to lack of such functionality within the LMS.
The peer assessment approach was modified in the last year to be much more detailed in the guidance on what feedback should be provided – with questions like what did you learn from reading this and what could be improved. The peer review process is seen as an area requiring further enhancement. Professor Horgen and two other colleagues are currently analysing the quality of the peer feedback comments.
Potential
The MOOC represents new thinking for higher education in Norway – Professor Horgen describes it as designing “outside the box of traditional higher education to create a MOOC that exemplifies flexibility”. It is not offered as the model for every MOOC, as many cannot incorporate that level of flexibility.
The current area for research is the peer assessment function - the latest changes led to improvements, but further study is underway on the quality of comments and if and how students are learning from them. As they are teachers in both secondary schools and higher education, the students are often motivated to find they are part of a study and so are willing to participate in questionnaire and face-to-face research.
Making the flexible MOOC model more widely known and having people adopt and adapt the approach are also key goals.
For Further Information
Svend Andreas Horgen
Assistant Professor
Department for Informatics and e-Learning
Norwegian University of Science and Technology
Norway
