Quality in online learning is fundamentally about:<\/p>\n
<\/p>\n
<\/p>\n
\n<\/ol>\n
A variety of quality frameworks and standards have been used to establish a basis for quality assurance in online learning including:<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
\n<\/ul>\n
Most of these standards focus on the materials used for learning and on the nature of assessment. Increasingly, these standards have started to require an assessment of student engagement and interaction.<\/p>\n
The way in which quality assurance functions in colleges and universities is through what might be termed \u201cinquiry\u201d. Inquiry is the process of critical questions about a program of studies or course being answered to the satisfaction of learners, instructors, and those reviewing the experience and outcomes of the program or course. The answers to critical inquiry questions vary by course, by discipline and for different student groups. The quality assurance process requires that the questions have been asked and answered with rigour.<\/p>\n
The standards for quality from the various quality assurance bodies, which looked at quality for online learning (see Appendix One), generally focus on these key categories and questions:<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
\n<\/ul>\n
When you review the above quality categories, you will see that several refer to the nature of instructional design (e.g. the first five of the above). Quality is fundamentally about design and intention with respect to achieving learner outcomes: quality is designed into<\/em> programs and courses; not inspected in once they are available to the public. This is why peer review of courses before they are offered to learners \u2013 a core component of the Quality Matters\u2122 (QM) program \u2013 is such an important component of how quality has been assured for many online programs and courses. Some programs and courses are sometimes piloted with small groups of learners for credit before being offered more widely.<\/p>\n Quality assurance in face-to-face instruction focuses primarily on three things:<\/p>\n <\/p>\n <\/p>\n \n<\/ol>\n In online learning quality assurance, much more of a course is \u201cvisible\u201d to scrutiny by third parties. The course outline remains important, but so too are all of the course materials and how they are used. Since many components of a course are \u201ccaptured\u201d for subsequent use \u2013 e.g. videoconferences and audioconferences with learners are often stored so that students can review them \u2013 these too become available for peer review. Student and faculty interaction, as well as student and student interaction, are also captured for review.<\/p>\n Many involved in online learning would claim that quality review and assurance in online learning is much more comprehensive than traditional face-to-face instruction. Some faculty have seen the level of scrutiny, which is possible for their online courses, as \u201cintrusive\u201d, while others welcome the opportunity for peer review to improve quality of the design and development of their courses.<\/p>\n The key, from a faculty member\u2019s point of view, is to become increasingly curious about \u201cwhat works\u201d best for the learning materials and type of student who registers for the course and be welcoming of the involvement of others in design and development. It is also important to think about \u201cmodalities\u201d of learning \u2013 the different kinds of learning experiences which a student will encounter during a program or course. This is particularly important given that you may have an increasingly diverse cohort of students, bringing different experiences and expectations to the classroom. \u00a0<\/p>\n These modalities can include (but are not limited to) team projects, self-study, active research, games and simulations, student presentation, challenges and case studies. In classrooms, many of these can be offered \u201con the fly\u201d in response to issues or concerns raised by students. In online learning, they need to be anticipated and planned for.<\/p>\n The key here is simple: different kinds of learning outcomes require different learning experiences to achieve them. Quality guidelines focus on outcomes and engagement. The faculty member needs to determine how outcomes and engagement will be achieved through different modalities of learning.\u00a0<\/p>\n Colleagues can help shape the opportunities for the use of different modalities within a course. A research study (available here<\/a>) by Chao, Saj and Hamilton (2010) noted that most faculty appreciated the collaboration which quality assurance guidelines and processes encouraged.<\/p>\n While more time is needed for such collaboration, the end result is a high quality learning experience for the student which makes use of reliable technology, quality third party learning resources and skillfully designed activities. Rather than working alone, the faculty member can enhance their teaching and learning activities through collaboration.<\/p>\n The current focus for quality assurance in many colleges and universities is on these critical aspects of the list above:<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n \n<\/ol>\n While other issues arise from time to time, these six issues dominate current quality assurance for online learning discussions.<\/p>\n Program and Curriculum Design<\/strong><\/p>\n \t\u00a0<\/p>\n \tWhen a program quality assessment is undertaken by a body such as Postsecondary Education Quality Assessment Board in Ontario<\/a>, the review team will focus on the learning outcomes intended from the program and whether or not the pedagogy adopted is likely to achieve these outcomes, given who the students are and the experience and skills of the faculty. They will also compare the program with similar programs offered elsewhere and look in depth at how outcomes are to be assessed. The design of the program must demonstrate the link between the learning process and learning outcomes.<\/p>\n Increasingly, review teams are also asked to look at the ways in which technology is being used, whether the program is classroom-based or fully online. The question is: \u201cdoes this program make best use of the available resources for learning\u201d, given who the students taking this course are?<\/p>\n In a recent review, for example, significant concern was expressed that the course materials made available for an in-class program relied too heavily on a single textbook and made too little use of the rich online resources available to support the learning objectives of the program.<\/p>\n \n\t<\/li>\n <\/p>\n The Delivery System (including the learning management system) and Related Pedagogy<\/strong><\/p>\n Decisions about which learning management system (LMS) a college or university uses are generally made at an institutional level \u2013 the faculty member is then asked to leverage the system in place to meet the learning needs of the course and the students taking it. This is not the place to review the learning management system available (there are over 200 of them) or their functionalities, except to observe that all of the leading LMS systems offer similar functionalities (for a review see here<\/a>).<\/p>\n The challenge for the faculty member is whether what they would like to do, from a teaching and learning point of view, is possible within the limits of the LMS available to them. In practice, most LMS systems now enable most of the activities faculty members use most often \u2013 the challenge comes when the faculty member wants to do something \u201cnew\u201d or different. Then they may need to link to a web-based activity rather than embed the work within the LMS. For example, a simulation or animation is easier to link to than to \u201cbuild in\u201d.<\/p>\n How students achieve the learning outcomes intended varies by experience, levels of literacy and numeracy, motivation and knowledge. The challenge for a faculty member working online is that they know little about their learners. They have to anticipate different kinds of learners in their course and design the course so that the learning needs and styles of different learners are represented in the modalities of learning and they have to ensure that these different modalities \u2013 e.g. a simulation versus a research task, a video lesson versus a group project \u2013 are the best way of achieving the learning outcomes intended. This is not an easy task.<\/p>\n What many faculty members do is to develop a learning modalities grid. This grid should:<\/p>\n <\/p>\n <\/p>\n <\/p>\n \t\t\u00a0<\/li>\n \n\t<\/ol>\n \n\t<\/li>\n <\/p>\n Preparation and Skills of the Faculty Member<\/strong><\/p>\n Some institutions, such as the University of Phoenix, require all faculty who teach online to have successfully completed a \u201cHow to Teach Online<\/em>\u201d course before they begin this work. The Ontario Undergraduate Student Alliance sees this as an essential requirement for quality assurance[1]<\/a>.<\/p>\n Some institutions offer voluntary professional development and there are significant resources available to support the development of professional knowledge and skills for faculty members (see here<\/a> for such resources).<\/p>\n For professional development, a faculty member should consider an annual update of a teaching philosophy statement. The University of Toronto offers some guidelines as to how to develop such a statement, which they use as part of a teaching dossier required for tenure review and assessment. Many institutions have teaching toolkits, which help faculty members develop their knowledge and skills and provide access to best practices.<\/p>\n Unless professional development is mandatory, each faculty member should consider their professional obligation for development as an opportunity to enhance their skills, knowledge and understanding. There are a growing number of post-graduate qualifications in online learning which may also be attractive to some faculty members.<\/p>\n \n\t<\/li>\n <\/p>\n Assessment and Authentication<\/strong><\/p>\n There are a number of issues with assessment for online learning which concern faculty. Here we look at three:<\/p>\n <\/p>\n <\/p>\n \t\t\u00a0<\/li>\n \n\t<\/ol>\n The first is simple: in an online environment, how do we ensure that the person submitting an assignment or examination is in fact the learner registered for the course? What steps need to be taken to ensure the integrity of the assessment process?<\/p>\n A variety of measures can be taken to ensure the authenticity of the learner submitting assessments. These include:<\/p>\n <\/p>\n <\/p>\n <\/p>\n \t\t\u00a0<\/li>\n \n\t<\/ul>\n The use of many of these tools for authentication is usually a matter of institutional policy. When a faculty member is concerned about authenticity, they should consult their administrators. What is important to remember is that authenticity issues occur in face-to-face teaching environments, especially when proctored examinations are not used.<\/p>\n A second issue raised by faculty concerns the use of \u201cparticipation\u201d in a marking scheme for a course. What should the \u201cparticipation\u201d grade be based on and how much weight should it carry for the final grade for the course? Where there is lab work, field work or project work? How will this be assessed for participation when the course is online?<\/p>\n Practices vary between programs, courses and institutions. Some general guidelines are offered here:<\/p>\n <\/p>\n <\/p>\n \t\t\u00a0<\/li>\n \n\t<\/ul>\n Both at the program level and the course level, faculty need to give consideration to the overall \u201carchitecture\u201d of assessment. For example, over the course of a program of studies in a specific discipline, what kinds of assessment will the learner be asked to complete and are there a sufficient variety of assessment processes to assess learning outcomes appropriately? If every course in a program uses the same form of assessment (e.g. multiple choice or the long-form essay), are we seeing a sufficient range of abilities from the learner?<\/p>\n There are no \u201cstandard\u201d answers to this question, but it is one which faculty needs to consider \u2013 quality assurance reviews at the program level will look at this question. Some suggestions for faculty working on a course within a program are:<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n \n\t<\/ul>\n Faculty members should review the assessments used in related courses to the ones they are developing so as to ensure that learners experience both variety of forms of assessment and consistency in terms of the rigour of assessment. Quality assurance reviews of assessment will look at the rubrics, forms of assessment and rigour by sampling learner work and comparing that work with the rubrics established for the course.<\/p>\n Increasing use is being made of machine intelligent assessment systems, which permit not only rapid development of complex assignments (not just multiple choice, but short answer questions, essay type questions and other assessment forms) with automated marking, linking to remedial course materials. A learner who secures a low grade on one or more parts of the assessment is directed to appropriate remedial learning materials. It is called \u201cadaptive assessment\u201d.<\/p>\n Adaptive assessment can go further: based on learner responses, the computer program adjusts the difficulty of questions throughout the assessment. For example, a learner who answers a question correctly will receive a more challenging item, while an incorrect answer generates an easier question. By adapting to the student as the assessment is taking place, these assessments present an individually-tailored set of questions to each student and can quickly identify which skills students have mastered.<\/p>\n Emerging quality standards see such assessment practices as increasingly necessary, given the range of learner abilities present in our colleges and universities. Such adaptive assessment provides both grades and feedback, but also links the learner to appropriate online materials for remediation and additional study. Any alternative assessment methods need to demonstrate this same functionality.<\/p>\n \n\t<\/li>\n <\/p>\n Student Engagement<\/strong><\/p>\n Student engagement is regarded, given the research on learning outcomes, as the best predictor of student performance and success. This appears to be the case whether the learning is classroom-based, hybrid, blended or online.<\/p>\n The National Survey of Student Engagement, which many Ontario colleges and universities are participating in, measures learner engagement by focusing on these key characteristics:<\/p>\n <\/p>\n <\/p>\n <\/p>\n \n\t<\/ul>\n A range of best practice examples and supportive learning materials are available to faculty in support of this framework for engagement (see here<\/a> for these resources). Also available are comparative data showing how a variety of institutions and programs are doing against the same criteria, though not all of these data relates to online learning.<\/p>\n To ensure engagement, a systematic review of the course before it is offered, and after the first offering, is undertaken of the key points in the learning experience students are expected to be engaged, the variety of kinds of engagement opportunities afforded and the \u201cvalue\u201d of these engagement activities, given the intentions the faculty member has for the course. \u00a0<\/p>\n Once the course is offered, a systematic approach to evaluating whether these \u201cpoints of engagement\u201d worked in the way intended is usually undertaken and adjustments made to continuously improve the effectiveness of learner engagement as designed into the course.<\/p>\n \n\t<\/li>\n <\/p>\n Use of Open Education Resources<\/strong><\/p>\n One of the remarkable developments over the last six to eight years has been the growth and development of open education resources (OERs).<\/p>\n These are course materials \u2013 entire courses, course components, simulations, games, activities, animations, learning designs \u2013 which are available for use under an \u201copen source\u201d licence. Faculty members developing a course are able to use these materials as part of their own course or as the course which they then support.\u00a0<\/p>\n You can find catalogues of these resources \u2013 individual course components or substantial parts of courses \u2013 at Merlot II (here<\/a>), Open University (here<\/a>), MIT (here<\/a>), the OER Commons (here<\/a>) or look through a compendium of such resource sites at the Commonwealth of Learning (here<\/a>).<\/p>\n The quality assurance question which needs to be asked about OERs, and other third party materials, is how well they are integrated into a designed experience for the learner: just pointing to the URL for the materials and saying \u201clook at these\u201d is not a way of ensuring the effective use of OERS or third party materials. Learners need to know why they are reviewing or using them, what they are supposed to do when they use them and what learning is intended to be associated with them.\u00a0\u00a0<\/p>\n There are two \u201cstreams\u201d of quality-related considerations which faculty need to consider before using OER materials in their course design and development.<\/p>\n The first stream refers to basic quality. These questions apply:<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n \t\t\u00a0<\/p>\n \t\tThese are very practical questions, but need to be addressed before the faculty member turns to the second stream of quality questions, which are:<\/li>\n <\/p>\n <\/p>\n <\/p>\n \t\tThese questions focus more on integration of the OER materials into a course design and on sustaining their value over time.<\/li>\n \n\t<\/ul>\n \n\t<\/li>\n \n<\/ol>\n Those who develop policy and practice for quality assurance in post-secondary education are engaged in a shift from an \u201cinput\u201d view of quality (\u201cit is about the design of the course\u201d<\/em>) to an \u201cinput and process\u201d view of quality (\u201cit is about the course and the experience of learning and the competencies it produces\u201d<\/em>).<\/p>\n This shift is adding a stronger focus for faculty on student assessment and engagement, both as part of the design of the program or course, and as part of the practice of teaching and learning, which is why assessment and engagement have been a strong focus in this introductory material.<\/p>\n The key message for faculty is that design is the key to quality. How a course is designed – both in terms of the knowledge and skills it seeks to secure for learners, but also how it engages those learners \u2013 will determine whether or not the course is a \u201cquality course\u201d.<\/p>\n To ensure the course does what it says it will do in terms of knowledge and skills, peer review is increasingly seen as the approach needed. A working draft of the finished course is sent to independent reviewers for comment and feedback on both the content and the learning design. Student feedback from past versions of the course or from a review of the draft in preparation by select students can be most valuable.<\/p>\n Quality is not an abstract construct \u2013 it is a pragmatic one. Quality in online learning is about the design, the experience and the outcomes.<\/p>\n Chao, I.T., Saj, T. and Hamilton, D (2010) Using Collaborative Course Development to Achieve Online Course Quality Standards. International Review of Research in Open and Distance Learning, Volume 11 (3) available at http:\/\/www.irrodl.org\/index.php\/irrodl\/article\/view\/912\/1644<\/a> (Recovered January 22, 2014).<\/p>\n Appendix One: Quality Standards, Frameworks and Resources<\/p>\n Quality Scheme<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n Date <\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n Sponsoring Organization <\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n Comments<\/strong><\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n The Quality Matters Rubric for Higher Education<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2011<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Quality Matters<\/p>\n A scheme in wide use in North America.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n European Universities Quality in e-Learning<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2011<\/p>\n \n\t\t\t<\/td>\n <\/p>\n European Foundation for Quality in E-Learning<\/p>\n Updated continuously<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Extensively used in Europe \u2013 includes a \u201cbadge\u201d to indicate that a program or course has been quality assured.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Commonwealth of Learning Quality Toolkit<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2009<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Commonwealth of Learning<\/p>\n With continuing revision www.col.org<\/a><\/p>\n \n\t\t\t<\/td>\n <\/p>\n In extensive use throughout the Commonwealth in higher education.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Effective Practice in a Digital Age<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2009<\/p>\n \n\t\t\t<\/td>\n <\/p>\n JISC (UK)<\/p>\n JISC is the vehicle used in the UK to support the use of technology in higher education. This resource has been widely used by faculty members and course teams in the UK.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Quality on the Line \u2013 Benchmarks for Success in Internet Based Distance Education<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2000<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Institute for Higher Education Policy<\/p>\n https:\/\/eric.ed.gov\/?id=ED444407<\/a><\/p>\n \n\t\t\t<\/td>\n <\/p>\n Seven categories of benchmarks (24 benchmarks in all). Also in use in North America.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Quality Scorecard for the Administration of Online Education Programs<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2010<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Sloan Consortium<\/p>\n http:\/\/sloanconsortium.org<\/a><\/p>\n \n\t\t\t<\/td>\n <\/p>\n Used in North America and Latin America. Third party evaluation of the administration of distance learning online.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Guidelines to Good Practices: Open and Distance Learning (GGP-ODL)<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2011<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Malaysian Qualifications Agency<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Used extensively in Asia in support of online learning.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Asian Association of Open Universities<\/strong><\/p>\n AAOU Quality Assurance Framework<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2010<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Asian Association of Universities<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Best practice driven guidelines, continuously improved.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Rubrics for Online Instruction<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2003<\/p>\n \n\t\t\t<\/td>\n <\/p>\n California State University<\/p>\n http:\/\/www.csuchico.edu\/roi\/<\/a><\/p>\n \n\t\t\t<\/td>\n <\/p>\n Aimed at faculty members and intended to address the question \u201cWhat does a high quality online course look like?\u201d.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Capacity to Deliver Online Degree Programming <\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2011<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Postsecondary Education Quality Assessment Board<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Quality assurance framework in extensive use in Ontario post-secondary institutions.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Quality Framework<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2007<\/p>\n \n\t\t\t<\/td>\n <\/p>\n African Virtual University<\/p>\n \n\t\t\t<\/td>\n <\/p>\n A basic framework for quality assurance in online teacher education using 10 criteria.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n <\/p>\n Quality Assessment Standards for Programs Delivered in Blended, Distributed or Distance Modes<\/strong><\/p>\n \n\t\t\t<\/td>\n <\/p>\n 2008<\/p>\n (revised 2011)<\/p>\n \n\t\t\t<\/td>\n <\/p>\n Campus Alberta Quality Council<\/p>\n These are standards added to the quality assurance framework for all post-secondary programs offered in Alberta.<\/p>\n \n\t\t\t<\/td>\n \n\t\t<\/tr>\n \n\t<\/tbody>\n \n<\/table>\nDifferences between Quality Assurance for Face-to-Face Instruction and Online Learning Quality Assurance<\/h2>\n
<\/p>\n
Transitioning from Face-to-Face to Online in Terms of Quality<\/h2>\n
Six Key Quality Assurance Issues<\/h2>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
Authentication<\/em><\/strong><\/h4>\n
<\/p>\n
Participation<\/em><\/strong><\/h4>\n
<\/p>\n
The Design of Assessment<\/em><\/strong><\/h4>\n
<\/p>\n
Developments in Online Assessment<\/em><\/strong><\/h4>\n
<\/p>\n
<\/p>\n
Shifting Quality Assumptions<\/h2>\n
Design of the Learning Process and Assessment is the Key to Quality<\/h2>\n
References<\/h2>\n
<\/p>\n<\/p>\n
<\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n <\/p>\n