LLM's might lie sometimes

It seems there’s things which after you learn they are dangerous, you grow to avoid them for ever. The burning effect fire has on physical contact for instance. You only have to get burnt once. The memory will for ever be effective. However, for some others, even after multiple learning lessons, we keep returning to put our hands in the metaphorical fire. Since, the fire is not consistently what earlier lessons say it is.

That’s LLMs. LLMs makes mistakes. The chat clients consistently flash that ‘information might be wrong’ warning. But is this warning really meant to protect the user, or as some form of legal shield against what the LLMs companies know will go wrong?

Back to the fire metaphor.

If an LLM is fire that we are warned against, that is with “this LLM might spew out lies sometimes”, it is an interestingly different kind of fire. LLMs are the kind of fire that is burning hot now and perceptibly warm the next time you touch it.

Emphasis on perceptibly. They are a fire that will burn you like you are a leper. Till it gets to a part that still got some nerves: your domain of expertise, or any topic you are knowledgeable about.

But sadly, that’s only a very small fraction of the surface area it has to burn. You won’t be sure its still burning till it returns to burn another one of those few topics you know something about.

Such warnings as “This LLM sometimes makes mistakes.” are utterly ineffective for getting humans to not trust AI. But its just how humans function. You can’t be skeptical 24/7. Its a lot of work. You’d rather use a tool that you are sure you have more chances to catch if its lying, even on topics about which you ain’t an expert.

And its gotten worse with the forced integration of AI with the browser search interface. SEO driven top results weren’t always right. But I’d say, people interacted more with the primary sources and for that, for those that were skeptical, there was more incentive to confirm if a source was speaking truth or not.

With AI summaries however, that incentive has been destroyed. The first result is a summary that’s covered in what looks like correct citations. This screams credibility. I mean, the AI scrapped “THE WHOLE INTERNET” the information is already well synthesized. Why bother scrolling down to do the boring useless work of selecting sources you have already vetted.

The path of least resistance is to, even for one that would have been more skeptical otherwise, take it as face value that what’s presented is truth.

Just a few days back I searched for my pseudonym on Google and the Gemini summary of who I was read like some Wikipedia entry on an established Computer professional. It sourced its summary from two articles, half written articles I’d published on probability and blew things out of proportion. But here’s what was worse. The source links, the credibility markers, led to some random LinkedIn posts that had nothing to do with the summary it had presented me with.

Now am wondering, what reaction would I have had if I’d instead searched for someone I did not know anything about?

I’d most likely have confidently shared my well cited findings. Now think about how many people are falling for this. For how so unpredictably inaccurate these results can be.

Talking to someone about this, tthey in their defense for AI said it wasn't as dangerous since got things right most of the times. This was an interesting response. However, this someone was just helping me make my point. And it made me rethink if the warning , however ineffective, was as effective as needed for its authors.

How many can intuitively tell that if something is predictably wrong it's a alternative safer than something what is perceptibly more than often accurate yet unpredictably wrong? The warnings are actually as well stated as they can be. They do not claim “The LLM is always wrong”. They rather say “The LLM is sometimes wrong”. But who knows or cares to constantly be on the watch for when it is? And why is this sold as an adequate warning? For a tool as seductive as an LLM.

Interesting.

Also, there's a fascinating kind of addiction that arises out of a tool's reward system being unpredictable. The same reason for why slot machines and chat platforms designed like Omegle are addictive. Articles have been written about this. Those I've read however only talk about LLM's giving a near accurate answer so the user can keep prompting for the perfect that never comes. I think this as well contributes to the same kind of addiction. Ain't no psychologist but the right key words to find articles on this are 'intermittent reinforcement'.

A solution? I have none. I just wanted to rant about it. I use LLM's for key word generation though. They seem to be good at that. For everything else, I think its safest to pass.

So the question, is the message meant to protect the user or protect the LLM company? I'd say it does a good job of protecting the LLM company. I doubt the number gets to 100 for those that change their minds about how they choose to interact with the service on exposure to that message.

That's it for this rant… i guess that's off my chest now.