Why We Need New Indicators of Quality in Post-Secondary Education

Why We Need New Indicators of Quality in Post-Secondary Education

Post-secondary education faces a major new quality challenge.

Not only do colleges and universities need to demonstrate compliance with long-established quality principles, they also need to satisfy employers that their graduates have the skills and capabilities they need to succeed on the job. New hires must be able to hit the ground running — without additional training and skills development.

Why the Current Quality Regime Doesn’t Work?

There is a fundamental mismatch between graduate expectations of what their credential will bring, employer expectations, and what the quality regimes at colleges and universities require of each institution whose graduates are our future workers, citizens and community members.

Existing quality regimes focus on:

1. Alignment between course outlines with existing practices in higher education — what we call “fit with the system”;

2. Teaching standards as judged by a review of what teachers teach, what students produce and how this aligns with what the institution said it would do;

3. Internal processes: how decisions about students, programs and academic awards are made; and

4. The qualities of the staff and sustainability of the institution.

Although the various quality assurance agencies differ subtly, there is a great deal of similarity in their approaches, which gives rise to five major quality issues in post-secondary education.

1. Inhibits innovation.

Administration complain that the quality regimes they operate within make it difficult to innovate and respond to local, regional or national needs. The dominance of a specific quality ideology, the reluctance to accept different forms of teaching, assessment or learning and the hesitancy about new public/private partnerships are all examples of this issue.

2. Gives little weight to evidence of the value of a particular program for employers.

Employers prefer graduates from some institutions over others and, in some cases, will not hire graduates from certain programs if they don’t believe they have the skills and capabilities they require from a new hire. Yet few quality assurance agencies require or seek input from employers about the value they attach to a program or seek their views on new program proposals.

3. Gives little weight to the evidence associated with student engagement.

It is well established that the best predictor of learning outcomes is the level and depth of student engagement. Yet few quality assurance agencies require data from reliable measures of student engagement from related programs or the program under review[1]. Nor do they make significant use of the growing volume of predictive analytics many institutions now use to support students in their learning journey.

4. Takes a long time.

Quality assurance processes can take a long time and are expensive. Accreditation from the Association to Advance Collegiate Schools of Business (AACSB) takes several years and can cost between $200,000 and $800,000 in academic staff time, direct costs and compliance costs. Other forms of quality assurance processes for a new program can take between 12 and 36 months, and rarely cost less than $50,000 in time and direct costs.

5. Ignores the elephant in the room.

Many jurisdictions think they have borders for learning and that their quality assurance regimes ensure that qualifications earned within their borders are subject to their jurisdiction. This is pre-Internet thinking. Education is global. In 2020, for example, more than 660,000 Canadians studied one or more MOOCs with edX, Coursera or FutureLearn — none of which are Canadian-based distributors. Many of these Canadian students were studying for one of 67 degrees available that year (there are now more) or for one of the 1,180 micro-credentials (there are now more). And students wishing to study for an MBA can now do so online from Canada at one of more than 100 top-ranked schools. These programs are only subject to quality assurance if they teach face-to-face in a location within a Canadian jurisdiction.

Given the need to offer some programs quickly in response to market needs — such as stackable micro-credentials that can be transferred to a degree or “build your own degree” — traditional approaches to quality assurance need to be reimagined and redesigned.

What a New Quality Regime Needs to Look Like

1. The shift needed for a 21^st century quality assurance regime needs to focus on building trust in relationships and risk assessment, rather than “preventative” risk-aversion and compliance.

2. A new quality assurance approach needs to identify institutions that have appropriate quality regimes in place, with a low incidence of isolated academic risk-taking and sufficient capacity to sustain a program over time. Rather than assess every new program or change in program, such a regime would certify an institution as having appropriate quality assurance policies and practices in place, few incidences of poor judgement and risk-taking based on risk analytics data, and appropriate measures for dealing with these rare events[2].

3. There must be a strong focus on indicators of student engagement and satisfaction with both their learning experience and with the institution itself. Adopting and adapting the National Survey of Student Engagement (NESSE) and reviews of specific analytics would enable this to be a process of continuous monitoring by the institution — a way of assessing risk.

4. A new regime must also look at stakeholder satisfaction with the outputs of the institution, especially its graduates. This requires the systematic collection of evidence of the value of the learning experience such as satisfaction with graduates by employers, and graduate satisfaction some years after graduation.

5. The final feature needed is an indicator of innovation. What new programs, new approaches to teaching and learning, new public/private partnerships, new work-based learning experiences and new forms of assessment are being deployed and with what success? These indicators are essential to assess the agility and responsiveness of the institution to changes in stakeholder expectations, technologies and our understanding of effective pedagogy. Doing what has always done and expecting different results is not a hallmark of a quality organization.

Learning Must Adapt to a Changing Economy

Governments around the world are looking to colleges, polytechnics and universities to help drive social and economic post-pandemic growth. With targeted funding for specific competency-based programs that are linked to fast-growing sections of the economy, a new emphasis on short-form learning (micro-credentials, for example) and outcome-based funding heavily geared to completion rates and graduate employment, the stage is set for post-secondary education to demonstrate its value.

At the heart of this is an emphasis on demonstrable competencies and capabilities. Employers are now less interested in qualifications on paper than in what the graduate can do. Indeed, many employers, including Google, Costco, Apple, Whole Foods, Penguin Random House, Hilton, Public, Starbucks, Home Depot, IBM no longer include candidates’ academic qualifications or where they studied in a review of applicants for jobs.

Why Now?

As colleges, polytechnics and universities begin a return to campus post-pandemic, there is an opportunity to reimagine the support they require for their work and to rethink key business processes. Time, money and energy invested in compliance and regulatory processes must be reimagined to enable greater flexibility, responsiveness and innovation in and across our institutions. 

Institutions need to move quickly to respond to the current situation. They must meet the demand for innovative, creative responses in terms of what they offer and how they offer it. They must explore new hybrid models of teaching and learning and service delivery, new approaches to assessment, new combinations of micro-credentials and longer forms of learning, new programs to meet the demand for upskilling and reskilling being made by employers and governments. They need a quality assurance regime that supports innovation, flexibility and speed to market.