Every month, over 500 million people trust Gemini and ChatGPT to keep them informed about everything from pasta. Sex or homework. But if AI tells you to cook your pasta in gasoline, you probably shouldn’t be taking its advice on birth control or algebra either.
At the World Economic Forum in January, OpenAI CEO Sam Altman assured the obvious: “I can’t understand in your head why you’re thinking what you’re thinking. to explain and decide whether it makes sense to me … I think our AI systems will be able to explain the steps from A to B as well. And we can decide whether we think they’re good. steps“
Knowledge requires justification.
It is no wonder that Altman wants us to believe that large language models (LLMs) like ChatGPT can provide a transparent explanation of everything they say: without a good justification, humans Neither believes in anything nor ever suspects it to be true. Why not? Well, think about when you feel comfortable saying that you positively know something. Most likely, this is when you feel complete confidence in your belief because it is supported—by evidence, arguments, or the testimony of trusted authorities.
LLMs stand for credentialed authorities; Trusted purveyors of information. But unless they can explain their reasoning, we cannot know whether their claims meet our criteria of justification. For example, suppose you tell me that today’s Tennessee haze is caused by wildfires in western Canada. I can take you at your word. But suppose yesterday you solemnly swore to me that a snake fight was a regular part of a thesis. Defense. Then I know you’re not entirely trustworthy. So may I ask why you think the smog is caused by Canadian forest fires? For my belief to be valid, it is important that I know that your report is reliable.
The trouble is that today’s AI systems can’t earn our trust by sharing the reasoning behind what they say, because there is no such reasoning. LLMs are also not remotely designed. To Because instead, models are trained on vast amounts of human writing to detect complex patterns in language, then predict or extrapolate. When a user enters a text prompt, the response is simply the algorithm’s projection of how the pattern will continue. These results replicate (most) faithfully what a conscious human would say. But the underlying process has nothing to do with whether the output is valid, let alone true. As Hicks, Humphreys and Slater put it “Chat GPT is bullshit.“‘LLMs’ are designed to produce texts that appear to fit the truth without any real concern for truth.”
So, if AI-generated content isn’t the artificial equivalent of human knowledge, what is? Hicks, Humphreys and Slater are right to call it bullshit. Still, what LLMs spit out is true. When these “false” machines actually produce correct results, they produce what philosophers call Gettier cases (After the philosopher Edmund Gettier). These cases are interesting because they strangely combine true beliefs with ignorance of the justification for those beliefs.
AI outputs can be mirage-like.
Consider this example, from Writings Eighth-century Indian Buddhist philosopher Dharmavatara: Imagine we are searching for water on a hot day. We suddenly see water, or so we think. Actually we are not seeing water but a mirage but when we reach the spot we are lucky and there we find water under a rock. Can we say that we had real knowledge of water?
People largely agree. That whatever knowledge there is, the passengers in this example do not have. Instead, they were lucky to find water precisely where they had no good reason to believe they would find it.
The point is that whenever we think that we know something that we have learned from LLM, we keep ourselves as passengers of Dharmavatara. If the LLM was trained on a standard data set, it is quite likely that its assertions will be correct. These claims can be likened to mirages. And the evidence and arguments that can justify his claims are probably somewhere in his data set—just like water seeping under a rock. But the justifying evidence and arguments that probably exist did not play a role in producing the LLM—just as the existence of water played no role in creating the illusion that supported the travelers’ belief. That they can find him there.
Altman’s assurances, therefore, are deeply misleading. If you ask an LLM to justify its outputs, what will it do? It won’t give you any real justification. This is going to give you Getter justification: a natural language pattern that convincingly mimics a justification. A concept of justification. As Hicks et al said, this is a poor justification. Which, as we all know, has no justification.
Right now AI systems regularly mess up, or “hallucinate“In ways that continue to slip the mask. But as the illusion of justification becomes more convincing, one of two things will happen.
For those who believe that real AI content is a big Gettier case, LLM’s patently false claim to explain his reasoning will undermine his credibility. We will know that AI is being deliberately designed and trained to systematically deceive.
And those of us who aren’t aware that AI spits out gettier justifications — fake justifications? Well, we’ll just be fooled. To the extent that we rely on LLMs we will be living in a kind of quasi-matrix, unable to sort fact from fiction and unaware that there may be a difference.
Every production must be justified.
When weighing the importance of this difficulty, it is important to keep in mind that there is nothing wrong with the way LLMs operate. They are incredible, powerful tools. And those who think AI systems spit out gettier cases rather than (artificial) knowledge already use LLM in a way that takes this into account. Programmers use LLMs to draft code, then use their coding skills to modify it to their own standards and goals. Professors use LLMs to prepare paper prompts and then revise them according to their academic goals. During this election cycle any speechwriter worth the name will check the hack out of any draft AI composes before letting their candidate walk the stage. Etc. Etc.
But most people turn to AI exactly where we lack expertise. Think of teenagers researching algebra… or prophylactics. Or seniors seeking diet — or investment — advice. If LLMs are going to mediate the public’s access to this kind of critical information, at least we need to know whether we can trust them. And trust would require knowing what LLMs cannot tell us: if and how each output is legitimate.
Fortunately, you probably know that olive oil works better than gasoline for cooking spaghetti. But what dangerous recipes for reality have you swallowed whole, without tasting justification?
Hunter Kale is a PhD student in philosophy at the University of Tennessee.
Christina Gehrman, PhD, is an associate professor of philosophy at the University of Tennessee.
Credit : venturebeat.com
