Colleges and universities are showing a significant interest in data analytics – the use of their available, complex and comprehensive data systems – to improve recruitment, retention, completion, and student engagement. Policy makers and administrators look to analytics to help with planning, student retention and satisfaction strategies and to support their strategic planning work. Faculty and instructors seeking to improve student engagement also look at developments in analytics and ask “can this help me see where my students are and what they are struggling with?”, particularly in online learning. A growing number of faculty and instructors are researching analytics in higher education and exploring the potential and the risks of using these technologies.
Many open source and commercial solutions are available to integrate the data, develop machine intelligent models of student behaviour and then use these models to anticipate what students might do so they can design appropriate interventions. As analytic systems are getting increasingly sophisticated, administrators, faculty and instructors want to explore the ways in which they can use analytic systems to support their work.
How the United Kingdom Open University Identifies Students at Risk?
The United Kingdom Open University provides an excellent example of how a major online university uses analytics. A team at the Open University, under the leadership of Professor John Domingue in the Knowledge Media Institute (KMI), worked in partnership with the Institute of Educational Technology (IET), to develop a predictive analytic system.
Students at risk are identified based on their interactions with the university’s learning management system, the time they are taking to submit assignments, patterns of interaction with learning materials and the grades they obtain on both formative and summative assessments. The Open University’s support system then acts to support students at risk of dropping out or who are seen to be underperforming – tutors can connect with students and help with any difficulties or challenges they are facing.
The Open University’s system can also influence course design and development, showing the course team what the students are struggling with or whether the learning gap between one course and another within a discipline is just too great (for example, the difference between an introductory mathematics course and a mathematics course just one level above). The value of the analysis is it can lead to action –rewriting these courses and possibly introducing more materials into the introductory course to make the next course easier.
Predictive Analytics Can Pay for Itself Very Quickly
Speaking about the development of analytics for colleges and universities in Canada’s The Globe and Mail newspaper in September 2016, Mark Milliron, co-founder of Civitas Learning Inc., one of the largest companies offering such services in the United States, suggests predictive analytics can make a significant difference in drop-out and completion rates. For example, by more systematic linking of the admissions profile of an individual to profiles of successful students, an institution can identify potential “at risk” students and offer early support and intervention opportunities. Many colleges and universities, who implemented predictive analytics, agree – the investment can pay for itself very quickly.
Five Key Ways to Effectively use Predictive Analytics
- The ability to connect all available data about students and their behaviour – their demography, their behaviour in relation to the learning management system, their performance on assessments (both formative and summative), the kind of questions they struggle with in formative and summative assessments and any other observational data about their interactions with the college or university.
The more comprehensive the database, the better the predictive quality of the analytics. This sounds straightforward, but is often more difficult than it at first appears. Many different data sources and formats across a college or university need to be connected and this is a very substantial and significant task – from social media pages related to a course, assessment data, and data relating to student use of a variety of learning media (for example, do they use video more than text, text more than audio and so on).
- While many predictive models have common elements across colleges and universities, the way in which these are displayed varies subtly within each – one cannot simply import the exact same parameters and analytic models; they need to be adapted for the student profile of the college or university and the range of programs and courses it offers.
For example, when a program has a high percentage of Indigenous students, the predictive models are different from a program which does not. The same applies when there is a high percentage of international students, whose first language is not the language of instruction or students new to the field of study. The predictive models need context and meaning at the level of the college or university.
- The strategy for action – what happens once a prediction is made – must be value driven. A faculty member or instructor cannot simply call a student to their office and say “we predict you are going to fail this course, so now we need to help you do better….”
The strategy for intervention needs to reflect the values, culture and supports available from the college or university. Any college or university adopting learning analytics needs to invest in the professional development of its staff to enable them to develop institution-specific best practices, based on what is known to work elsewhere.
- The strategy for action needs to be based on a theory of change, both in terms of how the college or university can be an enabler of student behaviour change and how the evidence the analytics produce supports changes within the college or university.
For example, if the evidence shows a particular program or course is far too likely to produce low grades or high drop-out and failure rates, what does college or university do to change the program or the basis for admission to the program? Committing to an investment in predictive analytics is also a commitment to a process of systematic, evidence-based change and development across the college or university. The old saying “the data doesn’t speak for itself” is correct – it provides a starting point for change, which must be aligned with the strategic intentions of the college or university.
- While big data can produce insights into what needs to happen to improve outcomes, small data are also critical. The term “small data” refers to observations which suggest that patterns of behaviour or activities of groups are showing signs of real change.
For example, the way in which students use smartphones shows continuous change in what they use them for. Catching these subtle changes as they occur can lead to the college or university making better use of mobile learning. Looking at the emergence of new ways of assessing students may lead to changes in how the student experiences a course, not at all related to big data and analytics.
The use of predictive analytics is an example of the way in which faculty, instructors and administrators are learning how to “dance with machine intelligent systems” – using our intelligence and compassion together with the predictive power of analytics can make a real difference to students and their learning.
