What's Next for AR/VR and Immersive Learning

Augmented and virtual reality now

Engaging learners by connecting them to a vivid, multisensory and powerful real-time virtual experience is an effective way to develop knowledge and skills. That is the promise of the immersive technologies known as augmented and virtual reality (AR/VR). 

The technology is delivering. Here are some examples:

• Engineering students immersed in a virtual construction project are asked to inspect a partially completed building, identify engineering issues and suggest corrections.
• Apprentice automotive technicians can work on a virtual engine, diagnose issues and undertake corrective action. 
• Aircraft maintenance crews in the Canadian Armed Forces use AR/VR and immersive learning to develop the skills and capabilities they need to repair and maintain aircraft. Studies show this form of training is significantly quicker than more conventional forms and produces similar results.
• Nurses in training can practice a virtual procedure with no danger of causing harm to a real patient. Similarly, radiologists working in oncology can develop a high level of skill without the need for significant supervision.
• A student in mining can walk through a virtual mine and examine the structures from a health and safety perspective to certify the mine as safe.
• A chemistry student can conduct a virtual experiment and detect changes in the chemical composition of materials without leaving their home or local online learning centre.
• Business students, working as a team, can engage with an AR/VR experience and not only “deal” with the significant issue at hand, but also develop the team and critical thinking skills that are so essential for managers and leaders.

For all these examples, students must have an appropriate AR/VR headset, an Internet-connected device, a safe place to engage with the simulation and a well-designed simulated learning experience that develops very specific skills and competencies.

Next developments in AR/VR

AR/VR and their combined use in immersive learning experiences will continue to grow, with market forecasts seeing the sale of equipment and software growing from $22.1 billion in 2020 to $161 billion by 2025, fuelled by new technologies, AI-related developments and better design.

Amongst the work in progress, we can see:

• Architecture and design – Using advanced AR/VR, designers and architects can better translate their thinking and ideas into a realistic experience for potential clients, permitting them the opportunity to “walk through” a building not yet constructed . Models of buildings, cars and aircraft built in AR/VR can also run as simulations to test assumptions about energy use, sustainability and security.
• Healthcare – The Spanish National Research Council is trialling the use of VR in treating patients with Parkinson’s disease.
• Training – Companies such as Boeing, UPS and others see significant gains in the deployment of immersive technologies to speed technical work. Boeing reports that engine installation can be accelerated by 25% simply through the use of AR.
• Retail – IKEA Place lets a customer see what an IKEA product would look like in their home. Other retailers are also experimenting with similar applications. For example, YouCam Make Up lets users use a selfie to explore what various make-up combinations would look like. 
• Onboarding new hires – Accenture uses AR/VR to help new hires understand a company, its work practices and their role, collaborating with Meta platforms to design these deep learning experiences.

Many of the new applications run on a smartphone or tablet, although headsets or smart glasses are an added value feature that significantly enhance the experience.

In education and training, we can see these developments emerging:

• Gamification – In early childhood education, teaching a child to read is a major goal. Using immersive books and gamified books (turning reading into a game) has been shown to accelerate learning, especially for those with mild autism. For example, Augmenta11y uses AR to help children with dyslexia read books using the camera on their phone.
• Virtual labs – These already exist but are becoming more varied and richer learning experiences, especially in chemistry, biology and physics. We can now dissect a frog without killing an animal or mix compounds and explore the consequences of doing so all in a simulated environment. At the degree level, Harvard’s Innovation Lab has established an AR/VR studio with many advanced lab experiences. One study showed that medical students trained in anatomy and surgery using AR/VR were 230% more efficient as surgeons than traditionally trained medical students.
• Virtual tours – Students can explore art galleries, caves, mines, landscapes and historic buildings without leaving home. 
• Technical learning - Students at colleges and universities can also hone their skills in immersive environments, “repairing” virtual models of complex equipment. They study the structure of devices and learn safety techniques before they start working with the real thing. Just a year ago, the European Union Aviation Safety Agency introduced a VR device for pilot training. It allows an individual to work out the riskiest maneuvers in an immersive environment. Pilots, instructors and test pilots practice on this equipment.
• Skills for workplace success – A variety of simulations have been built that enable learners to demonstrate their problem-solving, creative thinking and analytic skills alongside their social and team skills. For example, Walmart is using VR to train employees on how to behave during periods of heavy workloads, such as Black Friday. Long queues, numerous discounts and complaints from waiting customers provoke stressful situations even for experienced employees. New specialists learn to cope with non-standard working moments. With the help of VR headsets, they get into various situations and must choose the best behaviour strategy in a particular case. Colleagues watch a trainee through the screen and comment on their decision. 

In a US study, more than half the colleges and universities in the US are exploring some use of AR/VR, especially as it relates to their professional and skills-based programs. No parallel survey has been undertaken in Canada.

AR/VR and the skills agenda

Ontario has a major challenge: Employers can’t find the skilled employees they seek. Moreover, the skills gap will continue to grow as more employees retire, with an expected demand for half a million skilled professionals by 2030. Although immigration is intended to ease this problem, it takes time to determine whether a newcomer meets the knowledge, skills and capability requirement for various trades and professions. 

Apprenticeship numbers are down, and the number of employers supporting apprenticeships is stagnant. Just 18% provide places for apprentices, and completion rates are also low (about 34%).

One solution being explored in a variety of continuing education units and trades programs is to modularize apprenticeship and to offer both AR/VR training alongside work-based learning and support from a journeyman. This is being tested in the Employment Ontario Skills Development project at the National Electrical Trade Council, funded by the governments of Canada and Ontario. The project offers AR/VR modules for training electricians. Similar developments are occurring in other areas of apprenticeship, such as aerospace, IT and nursing. The deployment of AR/VR resources can both aid retention (it better reflects how young people learn) and accelerate completion. It can also reduce the time to completion.

The promise and the practice

The use of AR/VR has been evolving for the past 25 years. Coupled with artificial intelligence and blockchain, the skills required to develop a challenging, realistic simulation remain the biggest barrier to a more widespread deployment of AR/VR.  

One solution is to involve students in the co-design and co-creation of effective simulations. Such an approach has been used successfully in both construction engineering and architecture programs at the college and university level. A similar development has occurred in the teaching of design, especially interior and stage design, with students using video and an AR creation tool to replicate every theatre in a particular city where dramatic productions requiring set changes were staged. 

Despite the growing number of AR/VR deployments, many instructors are hesitant to commit to developing the needed virtual learning environment. There are three principal reasons for this:

1. Unfamiliarity with what, on the surface, seems like complicated software used to develop simulations. Various versions of available software development kits require the instructor to (a) identify the 3D environment they want to immerse their students in; (b) define the interactive components – just where in the 3D experience will students be required to interact?; (c) design the interfaces between the experience and the student; and (d) determining what instructional material will appear on-screen and when it will appear during the learning experience. Instructors see this as demanding design work in addition to being demanding technologically.
2. The need for more time for preparation for what is often a small part of the learning agenda for an entire course. A typical three-credit course involves 45 instructional hours. A typical AR/VR simulation is usually less than an hour. Many instructors question the return on their investment of time.
3. The need for more equipment for students, especially high-quality AR/VR headsets. Although there are low-cost versions of AR/VR headsets available, a quality headset capable of running advanced simulations (such as Meta-Quest 2 or Oculus Quest 2) cost more than $500 for each user. For a class of 20, that is a significant cost. For the same cost, the instructor could hire two teaching assistants.

To advance the deployment of AR/VR, some colleges and universities have created support teams, skunkworks or development sandboxes to support instructors wishing to deploy AR/VR in their teaching. At one university in Ontario, a community of practice group supports its members in developing new deployments and new instances of immersive learning across the campus. They have also established an extended reality (XR) open resource site with valuable support and learning materials freely available to all. 

Short courses and micro-credentials have also been developed that help anyone who wants to develop an immersive experience. A variety of MOOCs support the development of different skills required to design, develop and deploy a strong user experience on a variety of different AR/VR platforms. These courses range from beginner-level to more advanced applications.

The future: AR/VR enhances education

VR technology is enhancing education, providing affordable learning tools that will bring more benefits to all parties.:

• Students can delve into educational materials, regardless of their abilities. Teachers report that they face the problem of the “distracted generation,” in which students who are accustomed to creative experiences on screens find it difficult to receive traditional education. AR/VR is in front of the student's eyes, “forcing” them to interact with the environment and memorize the material. It better reflects how the current generation of students learn.
• It’s immersive technology that breaks down language barriers. English is not the first language of 25% of Canadians. It is especially difficult for newcomers to adapt and quickly learn the language. but a virtual classroom that also has a language translation function can enable a newcomer to learn quickly while at the same time supporting language learning.
• It breaks down distance barriers. Students anywhere in Ontario can work on complex equipment and in highly specialized laboratories without leaving home. A student in Bissett can work on the installation of a small modular nuclear reactor in Timmins without leaving home. A student in Fraserdale can repair a BMW engine even though the car in his garage is a Ford F150.
• It’s inclusive. AR/VR models are easy and adaptable for learners with disabilities. For example, some immersive environments change colour contrast or font size, or add audio commentary. Special gloves, such as SignAloud’s, are connected to other virtual classrooms. They help translate the gestures of people with hearing and speech impairments into speech and vice versa. The quality of these technologies is improving all the time.
• It’s adaptable for each participant. When there are 30 or more people in a class, it is difficult to organize personal training. VR provides a chance to measure the productivity of each individual student and adapt the method, teaching style and pace. It is truly a personalized form of learning.
• It establishes learning through practice. Theory without practice is easily forgotten. VR supports experiential learning and students’ interest in a subject. Numerous studies confirm that VR reduces cognitive load and is easier for learners to memorize complex, abstract topics.

In short, AR/VR has a growing presence in post-secondary education. As more applications emerge, more opportunities to learn both in an educational setting and at work will help the workforce upskill and accelerate learning. This technology is not going away. It’s just going to get better.