How Both are Helping Improve Learning
We are becoming accustomed to big data and analytics. Colleges and universities around the world are using big data analytics to help market their programs and services. Others are going further, using analytic models to predict when students are going to be in difficulty or are likely to drop out. Analytic and statistical models have been developed to predict students’ interest levels, travel and mobility ranges, likelihood for successfully passing courses, financial need levels, and likelihood of persisting to graduation. Big data has become big business.
Its growing use will continue in higher education as new models emerge, based on much bigger data sets.
Speaking of these developments, Michael King, IBM’s Vice President for Global Education Industry, said recently:
"The right set of information is everything. I think that, looking at lifelong learning and using data to help provide clearer pathways to students for a multi-institutional education plan, using tools like Watson, is an important goal. We want to show how to put more tools in their hands for broad data. We can give prescriptive data to save time intervening for individual students."
Big data has limitations
Big data is here to say. But it has limitations. While it may help provide insights as to who may be struggling with a component of a course or to complete an assignment, it does not suggest ways these students can best be helped. This is why we need small data.
The idea of small data is not new – it is how professions gain insights into practices that make a difference to outcomes. What is happening now is that there is an emerging discipline for small data capture – using observation, insights and clues to identify opportunities to change or improve practice. Learning is about relationships – the relationship between a student and their instructor, between a student and their peers, and the student and the body of knowledge or practice. Small data collection focuses on the ways in which these relationships develop, fail or succeed and seeks to understand patterns of activity within these relationships.
Small data “connects people with timely, meaningful insights organized and packaged – often visually – to be accessible, understandable, and actionable for everyday tasks”. This is how Allan Bonde of Actuate Corporation, which uses Small Data, sees it. He and his colleagues observe how people do things, store things, analyze things and share things, and use their observations to improve products and services. For example, many aspects of the physical design of tools we use each day are based not on big data analytics, but on small data. The gardening tool where the handle is shaped to reduce stress on the wrist is based on hours of observation of how gardeners use tools. The specially weighted knives, forks and spoons for those with Parkinson’s disease came from a similar set of detailed observations. Just as a forensic scientist investigates a crime scene looking for small clues, so those of us interested in improving learning outcomes, student success or course design need to do the same.
Small Data is Making a Big Difference
Here are five examples where small data have made a big difference:
- How changing language can change behaviour
A college used to use the language of “students at risk” and encouraged faculty to identify students who were at risk. Few faculty did so. But when they changed this ask to “identify students with promise whose promise is “unfilled”, not only did they get a much larger response, they also received a great many suggestions from faculty about what an appropriate response to the student’s need might be.
- How changing context can change outcomes
When an instructor, either in an online course or in a classroom, gives a context to some task or challenge, then students see the task or challenge in that context. Changing the context – for example, rather than being a health care context it becomes the context for a new video game – can change how the learner approaches this task. A legal education colleague did this – asked his class to create the rules for a new battle game between conflicting parties and then showed them how their rules for the battle were like the rules for a legal process.
- How observing peer groups can produce better group activities
A faculty member sent three groups different versions of the same online task – each group had a different component of the same problem and the “solution” required all three groups to realize they needed to share information between each other if any one of them was to be successful in solving the problem. Though she did not intend to formally teach problem-solving skills, she used the experience of the group work to do so with the result that the next time she undertook group activities all her groups performed better, faster, and smarter.
- How watching an individual student try to master a complex problem can help identify problem solving skills which can be share
The faculty member sat with a student who was struggling with some basic chemistry. Rather than explain the chemistry, the faculty member explored how the student was thinking about the work and what kind of processes they were using to “solve” the problem. This generated several insights into why this and other students were struggling with thinking like a chemist and led to significant changes in the design and delivery of these courses.
- How close observation leads to innovation
In designing a new approach to the design of a graduate degree, a small team spent time discussing the hopes and ambitions or potential of students and realized two things: (a) the students were looking for greater flexibility and choice than existed in any other program available to them – they were looking to “mix and match” their own degree; and (b) they wanted the opportunity to be flexible in how they studied (some in-class, some online, some through intense but short courses, some through projects). This led to a unique design for a graduate Masters degree which, now that the team can observe student behaviours within it, changes frequently to meet the faculty’s emerging understanding of what matters most to students.
- How close review of instructor online participation can change instructor behaviour
In one course, each instructor is assigned to a mentor. The mentor reviews not just the frequency of interactions with groups of students and individual students, but also the quality of these interactions. Careful observation can frequently lead to improvements in instructor interaction – more frequent, more varied, more humorous – so that the quality of relationship between instructor and students is both more authentic and more valuable.
These examples did not choose between small and big data. Where appropriate, big data helped confirm or suggest new avenues for small data exploration. But in these examples, it was the small data which led to the identification of patterns that then led to change.
Why Small Data?
Small data is what faculty and program level leaders use to make most decisions in post-secondary institutions. While some see the examples given above as anecdotes, repeated stories that combine to suggest patterns of distinct behaviours or opportunities lead to insight which in turn leads to change.
For example, Lego observed that one of the most frustrating things for parents is stepping on Lego blocks in bare feet. This led them to develop a slipper which is “Lego proof”. More importantly, Lego’s turnaround was triggered not by big data but by ethnography – the chance observation in the home of a German 11-year-old transformed their thinking about their market and their products (for this story, see here). Observations can change some organizations’ thinking in the same way a small clue at a murder scene can change how the investigators understand the crime.
What is so interesting about small data?
- Big data is hard
Doing it at scale and waiting for trickle down benefits can take time. Many in the college or university do not understand the analytic models and the complexities of the data and they are also suspicious of correlational data being used to suggest causation – a common mistake. Small data, in contrast, is a story or understanding which can be shared quickly and effectively and is easily related to - Small data is all around us
Social channels are rich with small data that is ready to be collected to inform learning design and educational decisions. At a personal level, we are constantly creating this small data each time we teach a class, mark assignments, log in, browse, post etc. Understanding patterns from these observations can trigger change. - Small data is at the center of the new understanding of student behaviour
Small data is the key to building rich profiles of our students. Not just who they are, but how they think, work, share, engage and work with others. Understanding these behaviours and ways of thinking should lead to significant improvements in pedagogy and the design of learning, especially online learning. - Data-driven learning is the next wave
Big and small data-driven learning design has the potential to revolutionize the way faculty interact with students and knowledge, transforming how students interact with each other and how students utilize knowledge resources for learning. This work will also transform assessment. - Platform and Tool vendors are starting to pay attention
New tools are emerging which enable faculty, students and others to capture and share small data so we can identify patterns quickly. Collaborative software with pre-designed capture tools are growing in use and several small data supports (Kanban systems, quick video capture) are emerging for this purpose. - Small data in education is about the learner
Small data is about the learner: what they need, and how they can act, what supports work best for them, how they use learning resources, what they know and don’t know about the technology they use. Focus on the learner first, and a lot of our decisions about teaching, learning design, technology supports will soon become clearer. - Simple
Small data is the right data. Some small data will start life as two or three faculty members making the same observations about how their students approach a learning challenge. Soon big data can be used to look at the behaviour on a larger scale, but you shouldn't need to be a data scientist to understand or apply small data for everyday tasks. Less is more and simple and small is good.
Three Keys for Small Data Collection and Use
Underlying the work with small data is the idea of collaborative professional autonomy. While each faculty member has a degree of autonomy in both what they teach and how they do so, outcomes from this work can be greatly improved through a deepening understanding of who learners are, how they learn and what other faculty members are doing which is making a difference to learning outcomes. Collaborative sharing of insights, observations and practices – sharing small data – can lead to major changes in what we teach and how we do so. This is true whether the program or course is in-class, online or blended. This is the first key to the effective use of small data: collaboration is key to understanding and interpreting small data.
The second key is to move beyond anecdotes and look for patterns. Anthropologists do this well – they study the behaviour of groups and then discern patterns in this behaviour – rituals and routines - and then observe deviance. This is what faculty members need to do to better analyze and interpret their small data observations. How do we interpret the thirty examples we have from a single classroom or online course in such a way as to discern a pattern which could change the way we design and deliver that course?
The final key is to do what kindergarten children do when given a new challenge. They work collaboratively, problem solving as they go, do lots of prototyping until they find a workable solution and then go for that solution with gusto. This is how young children use small data to discern patterns and proto-type solutions, each of which yields its own small data, so they continuously improve their understanding and approach and get it right. Adults, faced with the same task, often plan in depth the “winner take-all, one shot” solution and it either works or it doesn’t. They generally do not use small data to prototype and find solutions through trial and error and small data observations; they go for the big bang. We could learn a lot from watching how very young children solve complex problems.
Big + Small Data = More Opportunities for Teaching and Learning
There is no suggestion here that big data is not helpful. It can be. But it can also be misleading. Big and Small data will reveal far more opportunities for change which will reach into classrooms and affect teaching and learning. Small data can identify the nuances which will make all the difference to learning outcomes.
Suggested Reading
Lindstrom, M. and Heath, C. (2016) Small Data: The Tiny Clues That Uncover Huge Trends. New York: St. Martin's Press
Boje, D. M. (2014) Storytelling Organizational Practices: Managing in the Quantum Age. London: Routledge.
