The Misinformation Dilemma and the Role AI Can Play

Estimated time to read this post: 12 minutes

One major complaint about the use of AI to generate articles and answers to real-world questions is that it “hallucinates” — that is, it makes up facts that aren’t true and creates references to articles that don’t exist.

Although AI models like ChatGPT have improved in performing basic mathematics, they can still make errors, especially with more complex calculations. And they sometimes provide information that is imprecise or outdated when answering questions about the world.

A big part of the reason for this lies in the data being used to train AI services. AI companies depend on services like Common Crawl, which in turn creates a huge database of content scraped from the Internet. That’s more than 250 billion pages over the last 17 years from places like Twitter, Reddit, blogs, news articles and everything in between.

If you depend on this as your source of truth you’re bound to hallucinate. That’s why AI doesn’t know that hands (normally) have four fingers and a thumb, or that you can’t use glue in pizza.

AI, in short, faces the same problem understanding the world that we humans do. The more we depend on digital media, the harder it seems to distinguish fact from fiction. Did Kamala Harris’ large crowd really exist? Was there really a place called Doggerland? Are humans really changing the climate?

Misinformation is a major problem in today’s information-filled world. Fake news proliferates on social media, AI-based content farms fill search engines with dubious content, fake science journals obscure research and “pink slime” news sites undercut journalism. It’s almost too much for most people to manage on their own.

But it is a manageable problem.

Ancient philosophers would alter manuscripts and even create entirely new works attributed to ancient authors (which is how we got the Pseudo-Dionysius). We are still discovering how some of the most famous photographs from history were doctored, like the 19th-century image of Abraham Lincoln created using John Calhoun’s body.

The problem today isn’t that we can’t depend on computers. In fact, we can’t depend on any media without a system of fact-checking, critical thinking and trust. The problem is that, up to now, we have been able to depend on computers. And now this trust has been broken.

The real question is, how will we get that trust back again?

Single source of truth

AI will eventually be trustworthy. It will be so trustworthy that it’ll be humans who struggle to maintain credibility when they disagree with the machine (this, indeed, is one of the great fears people have of an AI-dominated future).

To understand how this trust will be developed we can look at two related subjects: data management theory and the scientific method.

In data management theory, the theory of database “normalization” is of great importance. This is a set of formal standards applied to databases in order to decrease redundancy and increase reliability. Theoretically, the objective is to ensure that each piece of data appears in a database once and only once. There should be no way, for example, that a person’s address could be located in two different places in the database.

For all practical purposes, though, data will be repeated in different places, to save time and resources. What should happen is that if you change that piece of data in one place, it should be changed in all the other places. That one place, then, becomes the “single source of truth.”

In a data network, managing the single source of truth becomes a lot more difficult. Network clients can depend on government registries for things like addresses and postal codes, or on organizations like ICANN for domain name resolution. But what about empirical facts, like the number of fingers in a hand, or the atomic number of calcium?

For this we rely on the scientific method. The core idea is that when empirical claims are made, they are made in such a way that they can be independently verified by multiple observers. Core claims are stored in reference documents like the CRC Handbook of Chemistry and Physics, The World Almanac and Book of Facts or LexisNexis. More ephemeral claims are confirmed through processes of experimentation and fact checking.

Sites like Wikipedia are using AI to look up and verify claims made using reliable external sources, which in turn helps other AI use Wikipedia to verify their own claims. Journalists are using AI for verification and fact checking. Note how complex this task really is, involving processes that range from gathering and harvesting information to verifying claims and integrating data from multiple sources.

Multimodal AI

Fact checkers are already using AI to support their work. Journalists are using AI to detect forged images and videos. Given access to these resources, and a mechanism for differentiating between what is trusted and what is not, AI becomes a powerful tool against misinformation.

But meaning and truth are based much more on one’s experience than text.

For example, suppose we’re building a weather prediction tool. We could use the system to harvest weather reports from around the Internet and derive some sort of prediction from that information. But why not plug the AI into the weather data itself? Hobbyists can already do this, connecting custom AI models into real-time weather datasets.

Much of today’s diagnostic AI already uses sensors; applications range from medicine to traffic control to economics. AIs are employed to recognize patterns and spot trends. And they are not merely passive observers; AI-assisted sensory networks can detect signal anomalies and adapt to changing conditions.

Combined with the Internet of Things (IoT) we will find AI playing an increasing role in the home and workplace: predicting, for example, when an elderly person will fall before they actually fall, or providing real-time information about a fire to help people escape, or create molecular dynamics simulations to help cure diseases like COVID. Combined with networks of security cameras, traffic sensors and other social data, AI will make increasingly accurate statements about public events.

None of these examples may appear to have very much to do with the use of AI in education, but at heart they speak to whether we can trust AI to create learning resources or conduct learning activities. And as we witness AI performing more and more remarkable tasks where fact and accuracy are absolutely vital, we will begin to trust it more.

Ultimately, AI knowledge will need to be based on what is called ground truth: the fact of the matter on the ground and in the world, as identifiable and measured by trusted and trustworthy observers, instrumentation and theory.

It will take a blend of social and computational process to recover from the seed of distrust. Industry standards, such as the Checklist for Artificial Intelligence in Medical Imaging (CLAIM), will be required.

None of this creates itself. But because so much depends on it, there is good reason to believe it will be created. The implications will be broad.

You can’t fool the algorithm

Humans are easy to fool. They fall victim to cults, cons and conspiracies. Their own senses fail them, as René Descartes famously observed. They make leaps of logic called “fallacies” and are subject to cognitive biases. They can also be subject to cognitive overload and information paralysis. They hallucinate, sometimes deliberately, they daydream and they report mirages as fact.

That’s why it’s so important that AI be trustworthy and fact-based. Were our AI systems to start gaslighting us, we would be almost defenseless. Our AI must be as steady as a clock, as reliable as a calculator.

With the modalities and fail-safes described above built in, AI will be much harder to fool than a human. Using tools such as First-Order-Logic-Guided Knowledge-Grounded (FOLK), AI will be able to spot inconsistencies well beyond the ken of the average human. AI will spot patterns in data humans cannot even name, let alone recognize.

Right now, it’s relatively easy to fool an AI, and some people have turned it into an art, putting bricks on the hood of a self-driving car or fooling image recognition systems with stickers. But that’s now. It will be very difficult to get misinformation past an AI that has access to scientific databases, sensor data, satellite photographs, historical records, and the weather outside the window.

It’s hard to overstate the importance of this. So much of human commerce and society depends on less than perfect information. We expect in day-to-day life to get away with little untruths, minor infractions of the law, shortcuts and sleights of hand. And pushing back will be like arguing with a speed camera; sure, you were only going 61, but you still see that strobe light flashing, incriminating you.

The same is true in education. As students, we expect to get away with things: handing in an assignment a bit late, collaborating a bit too much with colleagues, copying small snippets of other people’s text, using words without knowing fully what they mean, crafting woodwork to more or less flexible tolerances. People aren’t perfect, and we’ve learned to give each other enough flexibility to allow for that.

Can we trust trustworthy AI?

If there are risks in untrustworthy AI, there are probably greater risks in trustworthy AI. We may lose our direct attachment to truth and falsity through the senses. We may lose our common sense — flawed as it is — about what is credible and what is not. As with any mass media, should our AI systems be manipulated, it may not be possible to spot the misrepresentation.

This effect has already been demonstrated with calculators. Researchers deliberately programmed calculators to “lie” by changing the display. Students with lower numeracy skills were less likely to be suspicious of the results.

One might argue that the same effect was generated in people who grew up depending on television for the news; they became unable to critically evaluate whether the news they were watching was true. Others argue people who consume news through social media are suffering the same effect.

As we develop powerful AI tutors, we must emphasize the need to develop the human, not just the knowledge. It will be enormously helpful to ensure we are not learning about the world through misinformation, but our AI tutors need to understand how humans develop skills, instincts and common sense.

This is not simply to say, as so many have, that AI should merely augment the human, not replace the human. What good is it to have human oversight if the human is so much less capable than the computer? Humans need to hone and develop their intuition, enabling them to see when the computer is somehow getting it wrong.

What sort of intuitions should be at play? We can look at what humans bring to the world: things that aren’t facts, that are rooted in the human condition, that are based on our embodiment and our relation to the world. And we will need to learn these things not by acquiring them as “facts” to be remembered, but as based on our own lives and experiences.

For example, even as our lives become more technology-based, it will become even more important to work with our hands and be outside in nature. It will be more important to create art and culture, to look into ourselves and to see what counts as good, right and true.

In any AI world, we will need to become the AI’s ground truth. How we manage this, and how we learn to do it, will be the core challenge for education through the rest of the 2000s.

***

Fact check

Please see below for a list of sources used to write this article.

Common crawl - https://commoncrawl.org/

Hands - https://www.buzzfeednews.com/article/pranavdixit/ai-generated-art-hands-fingers-messed-up

Glue - https://www.theverge.com/2024/5/23/24162896/google-ai-overview-hallucinations-glue-in-pizza

Kamala Harris crowd size - https://apnews.com/article/trump-harris-detroit-crowd-size-photo-ff54a66d8e3197c90068ba94847297cf

Doggerland - https://en.wikipedia.org/wiki/Doggerland

Human impact on climate change - https://www.nrdc.org/stories/what-are-causes-climate-change

Fake science journals - https://languagelog.ldc.upenn.edu/nll/?p=64041

Pink slime - https://www.poynter.org/fact-checking/media-literacy/2023/pink-slime-journalism-local-news-deserts/

Pseudo-Dionysius - https://en.wikipedia.org/wiki/Pseudo-Dionysius_the_Areopagite