Making Sense of Blended Learning: Treasuring an Older Tradition or Finding a Better Future?

Making Sense of Blended Learning: Treasuring an Older Tradition or Finding a Better Future

Sir John Daniel, O.C.

Contact North | Contact Nord Research Associate 

"And gladly wolde he lerne and gladly teche"

Geoffrey Chaucer, Prologue to the Canterbury Tales, describing the Clerk of Oxenford

The Opportunity
 

A consensus is emerging that blended learning, a term that embraces various combinations of classroom presence and online study, will become the most common approach to teaching and learning in higher education. Does this consensus simply aim to safeguard the tradition of face-to-face teaching against an invasion of fully online learning - or can blended learning raise higher education to new levels of effectiveness and quality? We attempt an answer to this question.

Introduction

The MOOCs frenzy that was sparked by a few elite US institutions in 2012 alerted universities worldwide to the opportunities and threats of online learning (Daniel, 2012). As higher education faces up to this new reality, 'blended learning' has become the most common term for institutional strategies to address it. 'Blended' is a conveniently flexible word that can be applied any mixture of classroom activity and online instruction, although 'blended teaching' would be a more accurate descriptor of the evolving institutional approaches to course delivery. How students really learn is more mysterious (Israelite, 2015).

This essay examines what blended learning is and how it might best be used. A key question is whether blending some classroom activity with online study is merely a stopgap measure on the path to a largely online future. Are institutions simply trying to maintain age-old instructional habits against a rising tide of online learning opportunities or does face-to-face interaction really add value? If so, what is that value? Going further, might we reconfigure the whole teaching-learning process to something more effective than anything yet seen?

Anyone writing about online learning today owes a huge debt to Professor Tony Bates for his magisterial work Teaching in a Digital Age: Guidelines for Designing Teaching and Learning (Bates, 2015). It is a pleasure for me to acknowledge this debt upfront, as well as the inspiration I have gained from Professor Bates' prolific output of books and articles on educational media over many years. I refer those who want to explore further the past, present and likely future developments in online learning to this excellent book.

What is Blended Learning?

Although I have suggested that 'blended teaching' is a more appropriate term for institutional strategies than 'blended learning', I shall use the latter term in this paper because it is in common use. What do people mean by blended learning? Here are three contributions to a definition:

1. For Wikipedia (2015a), 'blended learning is a formal education program in which a student learns at least in part through delivery of content and instruction via digital and online media with some element of student control over time, place, path, or pace'.
    
2. The Christensen Institute (2015) adds that the student 'also learns at least in part in a supervised brick-and-mortar location away from home and the modalities along each student's learning path within a course or subject are connected to provide an integrated learning experience'.
    
3. Bates (2015, p. 309) notes that blended learning embraces a wide variety of designs, amongst which he distinguishes:
   • technology used as classroom aids (e.g. PowerPoint slides, clickers);
      
   • using a learning management system to support classroom teaching (e.g. for storing learning materials or for online discussions);
      
   • using lecture capture for flipped classrooms;
      
   • sequencing semesters of residential study with semesters studying online;
      
   • short periods on campus for hands-on experience or training followed by concentrated time studying online;
      
   • hybrid or flexible learning requiring the redesign of teaching to enable students to do the majority of their learning online, coming to campus only for specific in-person sessions (e.g. laboratories) that cannot readily be done online.

Most writers and institutions use 'blended learning' and 'hybrid learning' as synonyms, but considering hybrid learning as a particular manifestation of blended learning is more helpful. Bates (2015, p. 311) posits a continuum of learning situations ranging from face-to-face instruction with no technology to fully online distance teaching. He applies the term blended learning to all combinations of technology with presence in a classroom between these extremes.

In this context, Bates uses the term hybrid learning to designate situations where, instead of using technology in a serendipitous and opportunistic way, the whole teaching-learning system is redesigned to create optimum synergy between the in-person sessions and learning online.

The two ends of the Bates continuum deserve comment. While there has been a steady increase in fully online learning around the world since the beginning of this century we do not have statistics to compare the numbers of students in courses taught in classrooms to those who study fully online. Classroom teaching is likely the majority option at present although, since the numbers studying online are growing much more rapidly where a choice is available, we can predict a crossover at some point in the future.

What is clear, however, is that most academics and institutions still judge classroom instruction to be a higher quality option than fully online teaching so they treat it as the default mode when making choices about the use of technology. Is this judgement correct?

Does technology-based learning work? What does the research say?

Online learning is a contemporary expression of distance education, which has been around for many years. In that time there have been hundreds of comparative studies of the impact of different teaching technologies on student learning and performance. Such studies are difficult to conduct and interpret for several reasons. Much of the research conducted on campus classes, where different groups of students are taught in different ways, is based on very small samples. Conversely, the distance-teaching universities, which operate at scale and collect data from large samples, cannot readily compare their results with face-to-face teaching in other institutions since the distance students tend to be older and often more motivated than the on-campus students.

The tone of the findings of many such studies was set nearly half a century ago by Dubin & Taveggia (1968) (see also Neuendorf, 2013) who concluded: 'we have reported the results of a reanalysis of the data from 91 comparative studies of college teaching technologies conducted between 1924 and 1965. These data demonstrate clearly and unequivocally that there is no measurable difference among truly distinctive methods of college instruction when evaluated by student performance on final examinations' (p. 35).

Later researchers have refined this type of study by conducting meta-analyses, using sophisticated statistical tools to bring together the results of many separate investigations. Two such meta-analyses of more recent research are particularly noteworthy.

Bernard et al. (2004) compared distance education with classroom instruction for a variety of learners, from young children to adults, on measures of achievement, attitudes, and course completion, by examining 232 studies published from 1985 to 2001. They found an overall effect size close to zero for student achievement. However, asynchronous distance education had a small but significant positive effect on student achievement, whereas synchronous distance education had a small but significant negative effect. These authors also found also that a substantial proportion of the variability in effect sizes for student achievement and attitude outcomes was accounted for by the studies’ research methodology.

Means et al. (2013) compared blended learning with face-to-face teaching on the basis of articles published between 1996 and 2008 that focused on web-based courses where more than 25% of the instruction - but not all - was delivered online. To avoid the problems of methodology encountered in the Bernard study, these authors were very rigorous in selecting research for inclusion in their meta-analysis, winnowing a pool of over 500 papers down to a final cut of 45. They found that, on average, students in online learning conditions performed modestly better than those receiving face-to-face instruction. The advantage over face-to-face classes was significant in those studies contrasting blended learning with traditional face-to-face instruction but not in those studies contrasting purely online with face-to-face conditions.

In their conclusions these authors quote Clark (1983), who cautioned against interpreting studies of instruction in different media as demonstrating an effect for the medium itself, inasmuch as conditions may vary with respect to a whole set of instructor and content variables. They apply this caveat to their own meta-analyses in a passage that is worth quoting in full:

'(Our findings) should not be construed as demonstrating that online learning is superior as a medium. Rather, it is the combination of elements in the treatment conditions, especially the inclusion of different kinds of learning activities that has proved effective across studies. Studies using blended learning tended also to involve more learning time, additional instructional resources, and course elements that encourage interactions among learners. This confounding leaves open the possibility that one or all of these other practice variables, rather than the blending of online and offline media per se, accounts for the particularly positive outcomes for blended learning in the studies included in the meta-analysis. From a practical standpoint, however, a major reason for using blended learning approaches is to increase the amount of time that students spend engaging with the instructional materials. (Our) findings do not support simply putting an existing course online, but they do support redesigning instruction to incorporate additional learning opportunities online while retaining elements of face-to-face instruction. The positive findings with respect to blended learning approaches... provide justification for the investment in the development of blended courses' (Means et. al. 2013, p. 36).

We can summarise the implications of these two major meta-analyses as follows:

• Both studies show that face-to-face instruction is not superior to asynchronous distance or online teaching.
   
• The Bernard result that synchronous distance education had a small negative effect on student achievement means that institutions should not waste money on trying to re-create the impression of live, face-to-face instruction by investment in expensive remote classroom systems for synchronous instruction.
   
• Since most online learning is asynchronous, the earlier Bernard result showing that asynchronous distance education achieves better results than face-to-face teaching resonates with the results of the Means study.
   
• The superiority of blended learning probably lies not in the online medium itself but in the combination of elements that it brings into play, resulting in deeper student engagement with the instructional system.
   
• If we adopt Bates' distinction between blended learning as any mix of technology with face-to-face teaching, contrasted to hybrid learning as the redesign of the whole teaching-learning system, the research argues for investment in hybrid learning. In other words, the task is to 'create optimum synergy between the in-person sessions and learning online' (Bates, 2015, p. 310).

These results present us with a challenge. Elsewhere Bates (2015, p. 315) observes that there is very little evidence or theory to guide decisions about what is best done online and what is best done in person in blended learning, or indeed when fully online learning is a better option than classroom teaching. Curiously, perhaps, there is 'very little evidence-based theory about what makes face-to-face teaching so special'.

Bates argues that we should therefore posit what he calls the law of equal substitution, namely the assumption that academically, most courses can be taught equally well online or face-to-face. From this starting point other factors, 'such as cost, convenience for teachers, social networking, the skills and knowledge of the instructor, the type of students, or the context of the campus, will be stronger determinants of whether to teach a course online or on campus than the academic demands of the subject matter. These are all perfectly justifiable reasons for privileging the campus experience. (Also) there are likely to be some critical areas where there is a strong academic rationale for students to learn in a face-to-face or hands-on context. In other words, we need to identify the exceptions to the law of equal substitution. These unique pedagogical characteristics of campus-based teaching need to be researched more carefully... We need to turn the question on its head: what are the academic or pedagogical justifications for the campus, when students can learn most things online?' (Bates 2015, p. 329).

In the absence of useful academic research we shall approach this question from two directions. First, we look briefly at the history and evolution of teaching methods in higher education. Second, since student choice has largely driven the rise of online learning, we should give importance to their views about how they prefer to learn.

Teaching in higher education: what can we learn from history?

Even a short history of the evolution of teaching methods in higher education would require another essay. Here I shall simply comment on relevant developments in an eclectic manner, touching on some personal experiences.

Socrates and Plato

The fundamental elements of blended learning were present in the alliance between Socrates and Plato. Socrates believed in stimulating critical thinking and illuminating ideas through face-to-face inquiry: discussion between individuals based on asking and answering questions. Although this Socratic method is more often praised than practiced in contemporary higher education, it is held up as ideal method of teaching in some disciplines despite some iconoclastic commentary (e.g. Above the Law, 2015).

Socrates opposed the written word, which is today the main medium for distance and online learning - indeed for higher education generally. He considered that putting things in writing both weakened our capacity for memorisation and also spread untruths because the author's hypotheses could not be readily challenged through immediate dialogue. Fortunately for us, Plato recorded Socrates' hostility to writing, rather than adopting it himself, and captured some of his dialogues for posterity. Without Plato's writings we would have no basis to consider Socrates an icon for oral teaching!

The Lecture

These two key media, the oral and the written, come together in the lecture. The Latin (and French) meaning of 'lecture' is that which is read. In medieval universities, books being scarce, the instructor would read from an original source and the students would take notes. As books - and textbooks - became more available the instructor's oral discourse took the form of glosses on the original from the lecturer's own notes. From the 16th century onwards a lecture came to mean 'oral discourse on a given subject before an audience for the purpose of instruction' (Wikipedia, 2015b).

Although it is fashionable today to criticise lectures, universities still use them for most of their teaching: 'they have survived in academia as a quick, cheap and efficient way of introducing large numbers of students to a field of study' (ibid.). Moreover, lectures can have an element of prestige. The conferment of an academic honour is usually accompanied by an invitation to give an inaugural or keynote lecture. This oral tradition of communication of knowledge has been widely adopted in MOOCs (Massive Open Online Courses), where short video lectures, usually lasting between five and fifteen minutes, often provide the backbone of the instruction. Earlier research had showed that students' attention to a lecture - and their note taking - declined after 25 minutes.

The Tutorial

Another form of teaching with a venerable tradition is the tutorial. Oxford University claims that 'tutorial teaching is a unique aspect of the educational experience that Oxford offers its students' (Oxford Learning Institute, 2015). The usual elements of tutorials are that:

• students meet weekly, either individually or in very small groups, with the tutor of the discipline they are studying;
   
• each student spends an average of 13 hours reading independently and preparing written work for the tutorial (Oxford University Commission of Enquiry, 1997);
   
• students discuss their written work with the tutor 'thus honing their oral communication skills and giving them an opportunity to receive constant feedback'.

There are also extensive programmes of lectures at Oxford, but they are regarded as less important than tutorials and attendance is optional. As an Oxford student in the 1960s I was invited to sample the lectures offered in my subject areas each term and to persevere with any that I found useful. The work of researching and writing the weekly essay was the fundamental vehicle for learning at Oxford. My own experience, studying the physical sciences, was that the tutorials added little, although they seemed to be a richer experience for my fellow students in the humanities and social sciences. Later I came to realise that because the Oxford system was based on several years of largely independent study to prepare for a set of final exams, it had served as my introduction to distance learning!

Les Travaux dirigés

After Oxford I did a research degree at the University of Paris where I observed my professors and French doctoral student colleagues running travaux dirigés (TD) for undergraduates. These seemed to me to be more effective than the Oxford tutorial, at least for the physical and natural sciences. The students had to do some prior preparation, as for Oxford tutorials, but the TD, which brought together a somewhat larger group, was conducted more in the style of an apprenticeship. Teachers coached students to apply the theory they had learned and broadened the focus of discussion. The groups were, however, small enough for the instructor to give time to individual students where necessary.

Using the term apprenticeship in universities requires a word of explanation, because the term is more often applied to trades training. But, deriving from the French apprendre, it simply means that a student learns from a master in the field. It describes learning by doing under supervision, or modelling ways of being. Most teachers in higher education learn how to teach by apprenticeship - by watching their own professors teach and imitating them when given instructional duties themselves. The French students in the travaux dirigés were not only learning about the topic of the session, but also about how teaching is done in French universities. The same is true of graduate seminars in North America, which are apprenticeships in both the subject matter and academic culture.

We shall argue later that instructors today require rather more formal apprenticeships than the type just described because teaching through blended learning is a multi-dimensional process. 

Correspondence education

Commercial correspondence education started when universal postal services were introduced in the mid-19th century. Then in 1858 the University of London created an External Programme that allowed people worldwide obtain degrees by challenging its examinations. Before the end of that century public universities in Australia, Canada and the US were offering correspondence courses. These all offered correspondence tuition but with a few exceptions (e.g. Australia's University of Armidale) did not offer opportunities for students to meet face-to-face.

By the mid-20th century correspondence education had become a topic of research, led in particular by Borje Holmberg (1983). His influential theory of guided didactic conversation' viewed correspondence education as the 'conversation-like interaction between the student on the one hand and the tutor/counsellor of the supporting organization administering the study on the other'. At the time he formulated the theory these were mostly written conversations conducted through the post, but this changed rapidly in the last third of the 20th century, notably with the launch of the UK Open University (UKOU) in 1971.

Supported Open Learning

The UKOU sought to modernise correspondence education by using a variety of media and to improve it by providing fuller support to students. Following the Oxbridge pedagogical tradition that the UK most admires, the UKOU's teaching and learning system combined carefully assessed independent student work, interactive tutorials and centrally organised examinations and came to be called 'supported open learning'.

'Open' means that there are no academic prerequisites for undergraduate admission and that, subject to deadlines for assignment submission, students are free to organise their independent work on the multi-media course materials to fit their own schedules. 'Supported' means that for each course the student has a tutor, whose task is to mark and comment thoroughly on their assignments and to hold optional local tutorial sessions every few weeks.

Because it has no academic prerequisites for undergraduate entry the UKOU attracts many students who are well motivated but whose formal educational backgrounds are weak. The University provides extra support to these students, leading many to catch up quickly and achieve their degrees.

Experience over the years showed that, broadly speaking, one-third of all students attended these tutorial sessions regularly, one-third occasionally, and one-third rarely, if at all. In a quite different context the Whitney International University System of private institutions in Latin America, which offers students a choice between classroom courses and online learning, finds that students new to distance learning make regular use of opportunities for face-to-face interaction is the early parts of their courses, but attendance tails off rapidly once they become accustomed to independent study (Best, 2015).

Today 'supported open learning', which integrates independent study of multi-media materials with opportunities for interaction, would be called blended learning - with the difference that much of the independent study is now online. But the challenges of blending these two components in pedagogically effective and economically efficient ways have not changed. A paper that we wrote at Québec's Télé-université addressed the trade-offs among cost, learning effectiveness and student convenience in the pre-Internet era (Daniel & Marquis, 1979).

Supported Networked Learning?

The Internet became integral to the UKOU's teaching and learning system from the late 1980s onward, causing the numbers of students online to jump from 5,000 in 1995 to 110,000 in 2000. In a recent paper to the UKOU Council the current vice-chancellor wonders whether 'supported networked learning' might not be a better name for the University's teaching system today given the much higher intensity of interaction within the UKOU community that the Internet makes possible (Horrocks, 2015).

Students' independent work: are they doing enough?

An obvious conclusion from this commentary on various approaches to teaching in higher education is that real learning requires students to supplement the instruction they receive with independent work. Private study after lectures is essential for embedding learning and understanding, whilst the effectiveness of tutorials or interactive sessions depends largely on the essays or assignments that students prepare beforehand and on whether teachers assess these fully and constructively.

Current research indicates, however, that at least in the US, students are spending less and less time on independent work outside class. The Washington Post asks: 'is college too easy? As study time falls, debate rises'. It reported that, 'Over the past half-century, the amount of time college students actually study — read, write and otherwise prepare for class — has dwindled from 24 hours a week to about 15...' (de Vise, 2012).

The Huffington Post (Gorski, 2011) reported that '45% of students don't learn much in college', adding that ' A new study provides disturbing answers to questions about how much students actually learn in college – for many, not much – and has inflamed a debate about the value of an American higher education. The research of more than 2,300 undergraduates found 45 percent of students show no significant improvement in the key measures of critical thinking, complex reasoning and writing by the end of their sophomore years. One problem is that students just aren't asked to do much, according to findings in a new book, Academically Adrift: Limited Learning on College Campuses' (Arum & Roksa, 2011).

A review of this book (Jaschik, 2011) cites the finding that '36 percent of students did not demonstrate any significant improvement in learning over four years of college'. It continues: ' the main culprit... is a lack of rigor... 32 percent of students each semester do not take any courses with more than 40 pages of reading assigned a week, and half don't take a single course in which they must write more than 20 pages over the course of a semester... Students spend, on average, only about 12-14 hours a week studying, and much of this time is studying in groups'.

Some of the correlations that emerged from this research are important for our discussion of blended learning, for example:

• Students who study by themselves for more hours each week gain more knowledge -- while those who spend more time studying in peer groups learn less;
   
• Students whose classes reflect high expectations (more than 40 pages of reading a week and more than 20 pages of writing a semester) gained more than other students, especially if faculty assessed their written assignments thoroughly;
   
• Students who spend more time in fraternities and sororities show smaller gains than other students.

In another review of the book Cain (2011) finds its results troubling but warns against nostalgia for a supposedly better past. He concludes: ' Perhaps the authors are correct that we are not now in a golden age of promoting students’ critical thinking, but that does not mean that there has ever been one'.

This brief review of the history of university teaching shows that the quality of students' learning from oral teaching depends considerably on how much they follow up on lectures with further study and prepare material carefully for critique in tutorials. This is not surprising. First, we learn more by doing things ourselves rather than hearing others talk about them. But, second, our efforts need to be assessed and critiqued to uncover misunderstandings and dubious assumptions. I suspect that one of the reasons that the UKOU, with its supported open learning, performs highly in nation-wide polls of student satisfaction in Britain every year is that the University expects its tutors to comment extensively and mostly in writing on the work of individual students.

Earlier we quoted the conclusion of Means et al. (2013) that 'a major reason for using blended learning approaches is to increase the amount of time that students spend engaging with the instructional materials'. Can blended learning help us address the problem that students are not doing enough independent work to learn much?

What do students think of blended and online learning?

If they are prepared to shop around a bit students already have a wide choice of study options that represent many points on the continuum from face-to-face to online teaching. The blends of learning methods that institutions offer will inevitably evolve to meet student preferences. What do students look for and what do they find? We shall give three examples of student views: some reluctant to engage with online learning, some with experience of blended learning, and some who have taken courses purely online.

Wong (2015) reports on surveys conducted in Hong Kong conducted over several years. These showed that online learning is not nearly as popular with students as anticipated. Traditional face-to-face learning remains the preferred mode of study despite the fact that Hong Kong has high availability of personal computers and high penetration of broadband access. Wong investigated these negative attitudes, which contradict the common expectations of Hong Kong's government officials, educators and the media and do not match findings in similar jurisdictions.

The most important barrier identified was a lack of self-discipline and self-motivation in learners. This is tied to a teacher-centred and utilitarian learning culture with a tradition of rote learning that is the opposite of the self-directedness and student-centredness that online study requires. A second barrier is that much online learning is done in written form, which would need to be in English in Hong Kong, although students prefer Chinese as a social language and Chinese characters are more difficult to input anyway. Since Hong Kong institutions have done little to plan for online learning, the students' utilitarian aim of getting through the course with minimal work is often shared by teachers, who seek to instruct with least effort.

Across the Pacific in Canada attitudes are very different. A study of blended learning at York University found high-achieving students to be particularly enthusiastic (Owston, York & Murtha, 2013). They report: 'a remarkably strong relationship was found between perceptions and grades. Compared with low achieving students, high achievers were the most satisfied with their blended course, would take one again, and preferred the blended format more than fully face-to-face or online. High achievers also found blended courses more convenient, more engaging, and they felt that they learn key course concepts better than in other traditional face-to-face courses they have taken. An implication of the study is that low achievers may not be able to cope with the blended environment as well their high achieving peers. Therefore, when scaling up blended learning, institutions may want to consider offering students a choice of whether to enrol in blended or fully face-to-face course sections where feasible, especially in subject areas that students find difficult'.

The Learning and Teaching Office at Ryerson University did a literature survey on online learning for the benefit of its faculty (Schwartz, 2013). Her research found that online learning is only as good as the pedagogy underlying it and that course structure has the greatest influence on student perceptions. The instructor's role is also a crucial factor in student satisfaction with online learning. Students 'want their faculty to be partners in the learning process by providing content expertise, scaffolding learning experiences, helping students make connections, and providing prompt feedback… they expect to have a professor' (Barcelona, 2009). Schwartz notes other studies where a majority of students found the online course more challenging than a traditional course and a better learning opportunity in which they were more likely to do their assigned readings. She found evidence that older students and women had somewhat more favourable views of online learning than younger students and men, which is consistent with the experience of the open universities and earlier generations of multi-media distance education. Finally, she found that people who have already experienced an online course were more likely to take another one successfully.

To summarise, this brief roundup of students' views show that four factors in particular influence their attitudes to technology-based learning.

First, a cultural tradition where students are used to learning by rote and reproducing the knowledge thus acquired through conventional tests is not a good environment for introducing online learning - or at least not without significant planning and preparation by institutions and faculty.

Second, high achievers take to blended learning more readily than low achievers - which is probably true of almost any pedagogical innovation. The more students experience blended learning the better they perform.

Third, both blended and online offerings stimulate students to work harder and engage more fully with the course.

Fourth, sound pedagogy, especially very clear signposting of what students are expected to do, is essential to the success of online teaching. Also, where technical standards are not met, students tend to have a very negative experience of the course (Uvalić-Trumbić & Daniel, 2013).

Knowledge and skills for life and work in the 21st century

So far we have looked at two intrinsic features of the environment in which blended learning is being introduced, the traditions of university teaching and student preferences. Equally important, however, are extrinsic factors: notably the skills and knowledge that graduates will require for life and work in the 21st century. What should the focus of students' work in higher education be?

Skills

There is a lively debate about the knowledge and skills that people need for life and work in today's world, often summarised in the term '21st century skills'. For example, the Economist Intelligence Unit - EIU (2015) asked employers to name the most important skills they sought in their employees. The top five responses were problem solving, team working, critical thinking, creativity and leadership. Basic skills like literacy and numeracy came lower down the list, possibly because the employers surveyed took those skills for granted. They may also have assumed that graduates will have the subject knowledge that they need to make a start in their jobs, even if they require further training for the specific professional tasks they will be required to perform. Where should institutions find the balance between fostering skills and teaching content and what type of content do graduates need?

A particular virtue of Bates' e-book Teaching in a Digital Age: Guidelines for Designing Teaching and Learning (Bates, 2015) is that it starts by reflecting at some length on the skills and knowledge that people will need for living and working in today's and tomorrow's worlds. What are his conclusions?

He agrees (2015, p. 16) that certain skills have acquired greater importance in a knowledge society but his analysis goes deeper than the EIU report. Adapting work on the topic by the Conference Board of Canada (2014) he emphasises the importance of the following skills:

• communication skills (including the use of social media)
   
• the ability to learn independently
   
• ethics and responsibility
   
• teamwork and flexibility
   
• thinking skills
   
• digital skills
   
• knowledge management (which he calls 'perhaps the most over-arching of all the skills', adding that the skill of how to find, evaluate, analyse and disseminate information within a particular context is a skill that graduates will need to employ throughout their careers).

Bates also makes the important point that because these skills mostly need to be embedded within a knowledge domain, developing them is often context specific. Writing mainly with higher education institutions in mind, he stresses that content and skills are tightly related and that as much attention needs to be devoted to skills development as to content acquisition. This imposes constraints since 'although content can be transmitted equally effectively through a wide range of media, skills development is much more tied to specific teaching approaches and technologies'.

Academic knowledge

In a particularly significant section on the acquisition of content, Bates (2015, p. 59) swims against the stream by attacking the view, often heard outside higher education, that academic knowledge is less relevant in the Internet age. Building on the work of Laurillard (2001) he emphasises the difference between academic knowledge and knowledge or beliefs based on direct personal experience.

Without denying the importance of the experiential component of study, Bates argues persuasively that academic knowledge, which is 'a second-order form of knowledge that seeks abstractions and generalisations based on reasoning and evidence', is likely to be more future-proof than much experiential learning. He stresses that the concept of academic knowledge is equally applicable to both pure and applied knowledge. Both have the four fundamental components of academic knowledge: transparency, codification, reproduction and communicability.

Academic knowledge also applies to all levels of higher education, to community colleges as well as to universities. One reason why some institutions are moving more material online in professional and vocational programs is because the cognitive learning element in many professions and trades has rapidly increased. For example, trades now require more academic learning, such as increased ability in mathematics, electrical engineering and electronics. 

It is not an accident that the societies that have prospered in both the industrial era and the knowledge age attach importance to rigour, abstraction, evidence-based generalisation, rationalism and academic independence. Depreciating the importance of academic knowledge is tantamount to cutting off the branch on which we are sitting.

What does this imply for how we blend the means of learning? The key point, according to Laurillard (2001), is that university teaching must mediate between students' experience and its symbolic representation. We cannot expect students to construct academic knowledge simply through independent study or discussion with their peers. The teacher's role is to help them master the conventions and rules for acquiring and validating knowledge in that subject within a dialectical environment, in which argument and discussion within the rules and criteria of the subject discipline are encouraged and developed by teacher. Conversation and discussion are critical if this is to be achieved. Bates (2015, p. 78).

The classic example used to make this distinction between experiential and academic knowledge more concrete is Newton's Third Law of Motion, which states that for every action there is an equal and opposite reaction. Two decades ago Howard Gardner showed that even MIT physics students adopt the practices of naive elementary students when studying Newton's Laws of Motion (Gardner, 1991; Brown, 1992). The website of the American Psychological Association (2015) gives other examples of 'Common Alternative Conceptions (Misconceptions)' in Science, Mathematics and Language Arts' that show where students have difficulty grasping academic knowledge.

This summary of the skills and knowledge that graduates will need for the 21st century has clear implications for designing effective blended or hybrid learning, revealing two rather different areas in which faculty should invest special effort. The first is to give more attention to developing skills while embedding them in the appropriate context. The second is to ensure that students grasp the academic knowledge that underpins their areas of study.

This sounds like a tall order for hard-pressed faculty on today's crowded campuses. How should they blend the resources available to enable students to learn effectively? Bates provides tremendous help by devoting several chapters of his book to this vital question (Bates, 2015).

Putting it all together: blended learning for improving higher education

We shall now use the findings of the previous sections as building blocks to suggest how blended learning can create a better future for higher education. Expressed succinctly, five key findings were:

• We should aim for hybrid learning, designing for optimum synergy between online learning and teaching interventions.
   
• Face-to-face teaching is not more effective than online learning. The principle of equal substitution suggests reversing current practice and treating online learning rather than classroom teaching as the default mode.
   
• Students engage more deeply with online learning and work harder than in classroom courses.
• The more independent study and work students do, the better they learn.
   
• The 21st century requires a blend of skills and knowledge. Interactive teaching is particularly important in helping students to develop certain skills in context and to grasp the academic knowledge of their discipline.

What are we trying to blend?

We need first to unpack our assumption that face-to-face teaching and online learning are the two elements that we seek to blend. What do these two components really represent? Almost 40 years ago my Télé-université colleague Clément Marquis and I wrote a paper entitled Independence and Interaction: Getting the Mixture Right (Daniel & Marquis, 1979). The article examined 'the difficult synthesis which distance learning systems have to effect between those activities in which the student works alone and those which bring the student in contact with other people'.

It is helpful, although they are not a perfect match, to equate interaction with the face-to-face component of blended learning and independence with the online component. In particular, considering the nature of the interactions needed between teachers and students frees us from assuming that interaction must occur in a particular way, as is implied in the three other descriptors we commonly use: face-to-face teaching, in person contact, and classroom instruction. Holmberg's term 'guided didactic conversation' is more general and suggestive than any of them.

Interactive activities

In that earlier paper, we used interaction to cover only those activities where 'the student is in two-way contact with another person (or persons) in such a way as to elicit reactions and responses which are specific to his/her own requests or contributions'. Such contact need not imply face-to-face meetings, nor do the reactions have to be immediate (e.g. e-mail exchanges).

Our findings suggest that an effective blended learning course in higher education should include two interactive elements as a priority:

• Apprenticeship-style sessions, similar to the French travaux dirigés discussed earlier, used for two purposes: 1) developing skills in the knowledge domain that do not lend themselves readily to online learning, and 2) giving students an opportunity to strengthen their grasp of academic knowledge in the domain by having them work through applications and examples. Using the term apprenticeship, which is in less common use in universities, simply emphasises that the purpose is learning by doing.
   
• Setting assignments for students and treating the marking of these as an important teaching tool.

The apprenticeship sessions require little commentary because, in the sense that we are using the term, they are already commonplace. The use of assignments as a major teaching tool is less common. Indeed, 'marking' is usually one of the least popular tasks for faculty. Various universities have issued guides for this function and they stress that careful and conscientious marking is important both to help students to progress and to underpin the credibility of institutional awards. An older guide (University of Edinburgh, 1995) written in the pre-Internet age is still considered by that University as fully relevant today. More recent institutional documents pick up many of the same themes (Nipissing University, 2015; University of Southern California, 2015), which also apply to marking in schools (The Guardian, 2013).  

Independent activities

Here, as in our earlier paper, we use the term 'independent' activities to denote those learning activities where there is no interaction as defined above. These include elements of the course such as:

• study of written material;
   
• watching/listening to video/audio clips, podcasts, simulations, etc.;
   
• quizzes with automated marking;
   
• preparing assignments.

Placing assignment preparation in the 'independent' category requires a word of justification. Clearly the assignments are done with a view to later interaction with a teacher/marker, but we consider that the interactive phase begins with their response.

Our findings suggest that most of these independent activities should be in the form of online learning. This reflects Bates' 'law of equal substitution' and the evidence that academically most courses can be taught equally well online or in the classroom. Results cited earlier suggest that, in western countries at least, students would take to this well and engage increasingly deeply with the course content the more familiar they became with the online technology.

We saw earlier that students in Hong Kong resisted online study because it is more work and requires greater self-discipline (Wong, 2015). Nevertheless, we expect that such students would respond to a strong institutional lead on this matter, because the reward for learners who are ready to manage their time well and engage thoroughly with the course is greater convenience and flexibility as well as deeper learning.

But are some students less comfortable with the technology used in online learning? What should we make of the talk about 'digital natives' and the 'digital divide'? Several studies and literature surveys in Canada have shown that such a generational divide is largely a myth, although one that is deeply embedded much thinking about technology in higher education (Bullen et al., 2011; Smith, 2011; Gabriel et al., 2011).

Earlier research at the UKOU on thousands of adult students of all ages had also shown that there is no significant divide between older and younger people in their attitudes to studying with technology. It surveyed 7,000 students aged between 21 and 100, with 2,000 between ages 60 and 69; 1,000 aged 70 and over; and, for comparison, four 1,000-member groups of students in their twenties, thirties, forties and fifties respectively. The results showed that while there are differences in the use of digital technology with age, the change is gradual from group to group. There is no coherent ‘net generation’ and no clear break between two separate populations (Jones& Hosein, 2010).

This study also found a correlation, but not necessarily a causal relationship, between attitudes to technology and approaches to study that was independent of age: 'Those students who had more positive attitudes to technology were more likely to adopt a deep approach to studying, more likely to adopt a strategic approach to studying and less likely to adopt a surface approach to studying'.

Independence and interaction: getting the blend right. Some principles

The blend we have proposed above allocates a small - but vital - portion of courses to interactive activities and the rest to independent online study. How should we allocate learning activities that do not fall neatly into either component? There is, of course, is no blend of online and interactive teaching that will suit all requirements. Although a course can only be conceived in relation to the institutional and national context in which it is set, some principles may be helpful. We suggest four.

Focus on learning outcomes

A first principle is to distinguish between the purpose of interactive teaching and the quality of campus life. Attracting students to campus for social purposes is not a sufficient reason for including face-to-face teaching activities if the learning objectives could be achieved more effectively online. This is not to depreciate the value of campus life. I thoroughly enjoyed sport, clubs and partying at Oxford, although I note in retrospect that these activities had little relation to my studies, which were largely independent learning.

But Oxford University in the 1960s was very different from most universities today. Bates (2015, p. 27) observes that ' probably nothing has changed more in higher education over the last 50 years than the students themselves'. As examples he cites two of his local institutions, the University of British Columbia, 'where the mean age of all its graduate students is now 31, and more than one third of all students are over 24 years old' and the British Columbia Institute of Technology, 'which estimates that now more than half of its new enrolments each year already have a university degree'.

The key point is that the enormous diversity of today's students is expressed in a wide range of different tastes and needs. Older students, sometimes still labelled 'mature' or 'non-traditional' students, are not usually looking to their engagement with higher education for social life and cultural activities. Furthermore, there are now abundant online services that provide the 'opportunities for dating and finding future spouses' and having 'access to social contacts that can further your career' that are a feature of campus life (Bates 2015, p. 329). Personal observation during my years as vice-chancellor of the UK Open University suggested that contacts between its students, all distance learners, resulted in hundreds of romantic attachments!

However, having also served as president of Laurentian University, which then had four campuses in northern Ontario, I do not underestimate the challenge of adapting buildings, services and campus life in general to the needs and pressures of the Internet age. But trying to prevent the hollowing out of campuses by obliging students to attend unnecessary classes is probably futile. We suggest, therefore, that in optimising the blend of online and interactive experiences the focus should be on attaining the learning objectives of the courses/programmes and not on wider purposes, such as how to sustain the campus, important though such aims are.

Practical and laboratory work

A second principle is to address the challenges of practical and laboratory work head on. Evidence and experience shows that careful design can reduce substantially the time required for physical presence in the laboratory. Bates (2015, p. 324) works through the design of a hypothetical course in haematology and concludes that four of the six key components of content and skills could be done perfectly well online. Information technology (e.g. the development of a virtual microscope) has reduced the cost and increased the impact of simulated laboratory work.

Careful design can make laboratory work more effective than earlier practices. From its inception the UKOU was determined to teach degree-level science and technology with high standards to tens of thousands of students. Practical work is done by a combination of specially designed home experiment kits and laboratory sessions on the campuses of other UK universities. When the national quality assurance authority assessed the quality of teaching of General Engineering across Britain the UKOU was the only institution to score 100%.

Teamwork and division of labour

Principle three is the requirement for greater specialisation and division of labour in the teaching function of higher education - as is already the practice in research. Compared to most modern organisations, teaching in higher education is still a cottage industry where one individual is responsible for all stages of production and delivery. Effective hybrid learning will require division of labour and specialisation. This is already well under way for the development of its independent aspects, where most institutions have web designers, software programmers and media specialists who can help faculty design and develop courses. It will require a greater change of habits to implement this principle on the interactive side of equation. Yet as student numbers increase it becomes essential for the lead faculty member to have help with the vital functions of holding apprenticeship-style 'learning by doing'' sessions and marking assignments.

Instead of dividing large classes into smaller sections and having adjunct faculty teach each one semi-autonomously, it is more effective to make the course available online and use the adjunct faculty for the interactive components. This approach changes the demands on both students and faculty but in my experience they give greater satisfaction to both groups once they are familiar with them.

Keeping costs down and quality up

Principle four is to seek an optimal balance of cost and quality. In the earlier days of multi-media distance learning it was easy to distinguish between the economics of the independent and interactive elements of students' work. The general rule was that the independent activities enjoyed low marginal costs and therefore great economies of scale whereas costs for interactive activities rose steadily with student numbers and had to be managed carefully. More recent developments are mostly good news on the cost front. Online learning is bristling with innovation (Contact North | Contact Nord, 2015a) and Bates provides a good analysis of cost drivers (Bates, 2015: pp. 273-279).

First, the production and distribution costs of learning materials have dropped steeply for all media, although naturally there is a significant cost difference between studio-produced TV programmes with high production values and videos produced on an instructor's webcam. A second highly favourable development is the growing pool of Open Educational Resources (OER), which is freely available to teachers and students. Using some of the excellent OER that are available for simulations, experiments, interviews, etc. can considerably shorten the time that it takes a teacher to put together online course material and also improve its quality. Students for their part can find OER to help with difficult concepts or with academic knowledge that does not seem to accord with everyday experience. Bates (2015, pp. 434-438) also provides an excellent guide to the use of OER.

The only significant negative cost factor that has accompanied the growth of online learning is that, because it is now widely distributed among institutions, class sizes are usually smaller than in the open universities that were established in the era of multi-media distance education. The UKOU, for example, enrolled many thousands of students in each of its foundation courses for several years running and so what we have called independent activities benefited from great economies of scale. Institutions designing blended learning courses today have fewer opportunities for such economies and should therefore take advantage any that are available. This will mean offering as much of the course as possible online, as we have proposed, and making repeat offerings with only the minimal changes necessitated by new knowledge and feedback from students.

As regards quality, we simply note that the focus of assessments of quality by governments and their quality assurance agencies is moving steadily towards learning outcomes. This, for example, is the core criterion for the 'Quality Platform' being developed by the US Council for Higher Accreditation (CHEA) for non-institutional (post-traditional) higher education providers. This trend is a threat to older methods of judging quality by inputs such as library holdings and faculty qualifications, but is an opportunity for institutions that are redesigning their teaching in a holistic manner aimed at equipping graduates with 21st century skills and knowledge.

Conclusion: treasuring an older tradition or finding a better future?

I opened this paper with Geoffrey Chaucer's comment on a 14th century Oxford professor: 'and gladly would he learn and gladly teach'.

Today's world requires teachers to learn in a systematic way about the new context in which they must work. In his chapter on this vital topic Bates (2015, p. 415) observes that: 'moving to blended, hybrid and online learning requires a much higher standard of training for faculty and instructors... The use of technology needs to be combined with an understanding of how students learn, how skills are developed, how knowledge is represented through different media and then processed, and how learners use different senses for learning. It means examining different approaches to learning, such as the construction of knowledge compared with a transmission model of teaching, and how technology best works with either approach. Above all, it means linking the use of technology to the specific requirements of a particular knowledge domain or subject area'.

This sounds like another tall order, especially if the only reward for this investment of time and effort is to fight a more effective rearguard action against an online attack on treasured traditions of university teaching. I urge a more positive view and believe that once equipped with such knowledge and skills, faculty can take higher education into a new age of deep learning, effectiveness and efficiency. The notion of a previous golden age of higher learning is probably an illusion. Technology has enabled humankind both to improve quality and to cut cost in most of the products and services on which we rely. This can also be true of higher education. The large distance-learning systems of the open universities already show that it is possible.

What this new age requires is hybrid learning where the whole system is redesigned to create a happy blend of student-teacher conversations and online learning. This essay has highlighted, in particular, two important ways to make higher education more effective for the 21st century. First, students need to engage more fully with independent work. Online technology can help them do this (Contact North | Contact Nord, 2015b) and must be used intensively to free up time for students to prepare assignments and for teachers to use their interactions with students over their assignments as a prime vehicle for teaching. Second, teachers must help students, via apprenticeship-style sessions and commentary on their assignments, to develop skills and acquire academic knowledge.

We have purposely avoided exploring specific online learning technologies: there is no magic, all-purpose technology. Selecting media and delivery methods depends on a variety of factors. Bates has distilled years of research and experience on making such choices into his book Teaching in a Digital Age: Guidelines for Designing Teaching and Learning (Bates, 2015). Contact North | Contact Nord's portal (www.teachonline.ca) is a rich pool of resources on topics such as online innovation (Contact North | Contact Nord, 2015a), the pedagogy of engaged learning (Contact North | Contact Nord, 2015b) and a series on game changers in online learning (Contact North | Contact Nord, 2015c).

A future of hybrid learning is an opportunity, not a threat. If implemented sensitively and professionally it will lead to higher student performance and greater staff satisfaction than trying to revamp an older model of higher education that was simply not designed for the masses of diverse students seeking higher learning in today's technology-rich age. We cannot promise a golden age of learning but the opportunities for empowering humankind are enormous.

References

Above the Law (2015). Former Law Prof says, 'The Socratic Method is a Sh**ty Method of Teaching. http://abovethelaw.com/2014/06/former-law-prof-says-the-socratic-method-... Accessed 2015-07-18

American Psychological Association (2015). Some Common Alternative Conceptions (Misconceptions). http://www.apa.org/education/k12/alternative-conceptions.aspx Accessed 2015-08-08

Arum, R. & Roksa, J. (2011) Academically Adrift: Limited Learning on College Campuses By Chicago: University of Chicago Press.

Barcelona, R.J. (2009). Pressing the Online Learning Advantage: Commitment, Content, and Community. Journal of Continuing Higher Education. Vol. 57(3). http://www.tandfonline.com/doi/abs/10.1080/07377360903262218 Accessed 2015-08-20

Bates, A. W. (2015). Teaching in a Digital Age: Guidelines for Designing Teaching and Learning, BCcampus. http://opentextbc.ca/teachinginadigitalage/ Accessed 2015-07-16

Bernard, R. M., Abrami, P. C., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., Wallet, P. A., Fiset, M. & Huang, B. (2004). How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Review of Educational Research, 74(3), 379–439.

Best, R. (2015). Private communication.

Brown, R. (1992) Review of Gardner, H. (1981) The Unschooled Mind, Basic Books, New York. Association for Supervision and Curriculum Development. 

Bullen, M., Morgan, T. & Qayyum, A. (2011). Digital Learners in Higher Education: Generation is not the issue. Canadian Journal of Learning and Technology, Vol. 37(1). https://www.cjlt.ca/index.php/cjlt/article/view/26364

Cain, T. R. (2011) Thinking Critically About Undergraduate Education, Thought & Action, Fall 2011, National Education Association. http://www.elegantbrain.com/edu4/classes/readings/depository/TNS_560/out... Accessed 2015-08-01

CHEA (Council for Higher Education Accreditation) (2014). Exploring External Quality Assurance Review for Non-Institutional Providers.

Christensen Institute (2015). Blended Learning Definitions. http://www.christenseninstitute.org/blended-learning-definitions-and-mod... Accessed 2015-07-16

Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–449.

Conference Board of Canada (2014) Employability Skills 2000+ http://www.conferenceboard.ca/topics/education/learning-tools/employabil... Accessed 2015-08-04

Contact North | Contact Nord (2105a). 125 Online Learning Innovations.
http://teachonline.ca/sites/default/files/pdfs/pockets_of_innovation_the... Accessed 2015-08-11

Contact North | Contact Nord (2015b). Towards a New Pedagogy of Engaged Learning.
http://teachonline.ca/trends-directions/towards-new-pedagogy-engaged-lea... Accessed 2015-08-11

Contact North | Contact Nord (2015c). Game Changers in Online Learning: Series.
Accessed 2015-08-11

Daniel, J. S. & Marquis, C. (1979) Independence and Interaction: Getting the Mixture Right, Teaching at a Distance, 14: 29-44.

Daniel, J. S. (2012). Making Sense of MOOCs: Musings in a Maze of

Myth, Paradox and Possibility, Journal of Interactive Media in Education. http://jime.open.ac.uk/articles/10.5334/2012-18/ Accessed 2015-07-16

de Vise, D. (2012). Is college too easy? As study time falls, debate rises, Washington Post, May 21. http://www.washingtonpost.com/local/education/is-college-too-easy-as-stu... Accessed 2015-08-01

Dubin, R. & Taveggia, T. C. (1968). The teaching-learning paradox: a comparative analysis of college teaching methods, CASEA, Eugene, Oregon.

Economist Intelligence Unit (2015). Driving the Skills Agenda: Preparing Students for the Future. Accessed 2015-08-03

Gabriel, M. A., Campbell, B., Weibe, S., MacDonald, R. J. & McAuley, A. (2011). The Role of Digital Technologies in Learning Expectations of First-Year University Students. Canadian Journal of Learning and Technology, Vol. 38(1)https://www.researchgate.net/publication/260208264_The_Role_of_Digital_T...'apprentissage_les_attentes_des_etudiants_de_premiere_annee_univer

Gardner, H. (1991). The Unschooled Mind, Basic Books, New York

Gorski, E. (2011) 45% of students don't learn much in college. Huffington Post  2011-01-18

Holmberg, B. (1983). 'Guided didactic conversation in distance education', in Sewart, D., Keegan, D., and Holmberg, B. (eds.) Distance Education: International Perspectives. London: Croom Helm

Horrocks, P. (2015) Private Communication.

Israelite, L (2015). More Lies about Learning, Association for Talent Development.
https://www.td.org/Publications/Books/More-Lies-About-Learning Accessed 2015-07-16

Jaschik, S. (2011) 'Academically Adrift'. Inside Higher Ed. 2011-01-18. https://www.insidehighered.com/news/2011/01/18/study_finds_large_numbers... Accessed 2015-08-01

Jones, C. and Hosein, A. (2010). Profiling University Students’ Use of Technology: Where Is the Net Generation Divide? The International Journal of Technology Knowledge and Society Vol. 6 (3) pp. 43-58.

Laurillard, D. (2001). Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies NewYork/London: Routledge.

Means, B., Toyama, Y., Murphy, R. & Baki, M. (2013) The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature, Teachers College Record Vol. 115, 030303. 

Neuendorf, A. (2013) The Teaching-Learning Paradox Revisited,
http://andrewneuendorf.com/2013/12/18/the-teaching-learning-paradox-revi... Accessed 2015-08-10

Nipissing University (2015) Tips for Marking. https://www.nipissingu.ca/academics/school-graduate-studies/master-arts-...

Owston, R., York, D. & Murtha, S. (2013). Student perceptions and achievement in a university blended learning strategic initiative.
http://aifprojects.yorku.ca/files/2013/01/blended-learning-article-Smurt... Acessed 2015-08-20

Oxford Learning Institute (2105). Tutorial teaching. 

Oxford University Commission of Enquiry (1997). Chapter 9 - Teaching and learning: undergraduate education. https://governance.admin.ox.ac.uk/chapter-9-teaching-and-learning-underg...

Schwartz, M. (undated). Satisfaction of Students with Online Learning. Ryerson Learning & Teaching Office. 

Smith, E. (2011) The Digital Native Debate in Higher Education: A Comparative Analysis of Recent Literature. Canadian Journal of Learning and Technology Vol. 38(3) https://eric.ed.gov/?id=EJ999218

The Guardian (2013) Don't get ticked off: a beginner's guide to marking. Teacher Network.
http://www.theguardian.com/teacher-network/teacher-blog/2013/sep/12/teac... Accessed 2015-08-21

University of Edinburgh (1995). Tutoring and Demonstrating: A Handbook. Chapter 6: Marking and Commenting on Essays by Dai Hounsell.
http://www.docs.hss.ed.ac.uk/iad/learning_teaching/tutors/handbook/tutor...

University of Southern California (2015). Evaluating and Grading Student Work. Center for Excellence in Teaching.

Uvalić-Trumbić, S & Daniel, J. S. (eds.) (2013). A Guide to Quality in Online Learning, by Neil Butcher & Merridy Wilson-Strydom, Academic Partnerships, Dallas (also in Chinese).

Wikipedia (2015a). Blended Learning. https://en.wikipedia.org/wiki/Blended_learning Accessed 2015-07-16

Wikipedia (2015b). Lecture. https://en.wikipedia.org/wiki/Lecture Accessed 2015-07-28

Wikipedia (2015c). Travaux dirigés. https://fr.wikipedia.org/wiki/Travaux_dirig%C3%A9s Accessed 2015-07-28

Wong, A. L. S. (2015). What are the experts' views of the barriers to e-learning diffusion in Hong Kong. International Journal of Continuing Education and Lifelong Learning, Vol. 7(2) pp. 25-51