Did you know cats were on the moon? That it’s safe to stare into the sun for 15 minutes or longer if you have dark skin? Or that in order to stay healthy, you should eat one small stone a day?
These are some of the latest pearls of wisdom that Google is dishing out to its US users (here in the UK we’re not so lucky yet). “Let Google do the searching for you,” the search giant promised when it unveiled a feature called AI Insights earlier this month. It integrates Google’s generative AI model Gemini into its search engine. The answers it generates appear above the traditional list of ranked results. And you can’t get rid of them.
At least the AI reviews didn’t have the effect that Google had hoped for. It certainly gained instant internet virality and people shared their favorite answers. Not because they are useful, but because they are so funny. For example, if you ask AI Overviews for a list of fruits ending in ‘um’, it will return: ‘Applum, Strawberrum and Coconut.’ This is called a “hallucination” in AI parlance.
Despite having a market cap of $2 trillion and the ability to hire the greatest minds on the planet, Google is still struggling with AI. Its first attempt to tap into the generative AI gold rush in February of last year was the ill-fated Bard chatbot, which had similar problems spewing factual inaccuracies. In his first live demonstration, Bard falsely claimed that the James Webb Space Telescope, not to be launched until 2021, had taken the “first images” of Earth outside the solar system. The mistake wiped $100 billion off Google’s market value.
This February, Google tried AI again, this time with Gemini, an image and text generator. The problem was that he had very heavy bars on diversity. When asked to create historically accurate images, he would instead create black Nazi soldiers, Native American Founding Fathers, and a South Asian pope.
It was a ‘well-intentioned mistake’, he pleaded Economist. However, Google was not caught off guard by the problems associated with generative AI. He will know about his abilities and pitfalls.
Before the current AI craze really took off, analysts had already figured out that generative AI was unlikely to improve the user experience, and might even make it worse. That caution was abandoned as investors began to pile in.
So why is Google’s AI posting such rotten results? In fact, it works exactly as you would expect. Don’t be fooled by the label “artificial intelligence”. Basically, AI reviews simply try to guess the next word it should use, based on statistical probability, but without any grounding in reality. An algorithm cannot say “I don’t know” when asked a difficult question because it “doesn’t know” anything. It can’t even do simple math, as demonstrated by users, because it has no basic concept of numbers or valid arithmetic operations. Hence the hallucinations and omissions.
This is less of a problem when the output doesn’t matter as much as when the AI processes the image and creates a minor glitch. Our phones use machine learning to process our photos every day, and most glitches we don’t notice or care much about. But Google advising us all to start eating rocks is no small glitch.
Such mistakes are more or less inevitable due to the way the AI is trained. Instead of learning from a curated dataset of precise information, AI models are trained on a huge, virtually open dataset. Google AI and ChatGPT have already scraped as much of the web as possible, and needless to say, a lot of what’s on the web isn’t true. Forums like Reddit are full of sarcasm and jokes, but AI finds them credible as honest and correct explanations of problems. Programmers have long used the phrase ‘GIGO’ to describe what goes on here: garbage in, garbage out.
The problem of AI hallucinations is consistent across the board. This pretty much precludes generative AI from being practically useful in commercial and business applications where you might expect it to save a lot of time. A new study of generative artificial intelligence in legal work has found that the extra verification steps now required to ensure that artificial intelligence is not hallucinating negates the time saved by deploying it.
‘[Programmers] they still make the same mistakes as before. No one has actually solved hallucinations using big-language models, and I don’t think we can, cognitive scientist and seasoned AI skeptic Professor Gary Marcus noted last week.
Now another problem presents itself. AI makes an already bad job worse by generating false information that then pollutes the rest of the web. “Google learns everything it sees on the internet, and nothing creates spam better than AI,” as one X user put it.
Last year, leading AI companies admitted that after running out of content to scrape from the web, they started using synthetic training data – that is, data generated by generative AI itself. A year ago, OpenAI’s Sam Altman said he was “pretty sure that soon all data will be synthetic data” created by other AIs.
This is a huge problem. Essentially, it causes the models to “crash” and stop producing useful results. “Model collapse occurs when generative AI becomes unstable, unreliable, or stops working. This can happen when generative AI models are trained on AI-generated content rather than humans, Professor Nigel Shadbolt of the Open Data Institute warned last December. One researcher, Jathan Sadowski, called the phenomenon “Habsburg AI” after the Spanish Habsburg dynasty, which died out in the 1700s due to diseases caused by in-breeding.
You could argue that something like this already happens without the assistance of artificial intelligence, for example, when a false fact is put on Wikipedia, it is cited in the media, and then the media citations become the reason for its further inclusion on Wikipedia.
AI simply automates and accelerates this process of generating falsehoods. This week, Wire gave the following example: “When Google claimed that there was no African country beginning with the letter K, its response appeared to be based on a ChatGPT web discussion where the same question was incorrectly asked. In other words, AI is now using other AI inventions as gospel.”
The most apt description of this phenomenon comes from some American researchers who last year coined the phrase ‘Model Autophagy Disorder’ or MAD. They wanted to evoke the practice of introducing bovine prions into the cattle food supply, a practice that caused bovine spongiform encephalopathy, or mad cow disease. “Our primary conclusion in all scenarios is that without sufficient fresh real-world data at each generation of the autophagy loop, future generative models are doomed to progressively decrease in their quality (accuracy) or diversity (fit), they wrote.
Very few people warned about the downsides of generative AI when OpenAI open sourced its ChatGPT tool in November 2022. Now ChatGPT has polluted the web and poisoned itself and other AI tools. Cleaning it up will be a big challenge. While the promised gains of artificial intelligence remain elusive, the costs are clearly starting to rise.
Andrew Orlowski is a weekly columnist at Wire. Visit his website here. Follow him on X: @AndrewOrlowski.