Exhibit 005: The Search Engine That Suggested Glue on Pizza
The humans placed AI-written answers above the world's search results, sourced partly from jokes, and shipped them to every US user at once. Glue was recommended. Preserved here forever.
In May 2024, Google rolled out AI Overviews (AI-generated answers that sit at the top of search results) to everyone in the United States.
Within days, screenshots were everywhere. Asked how to keep cheese from sliding off pizza, an AI Overview suggested adding glue to the sauce. Asked how many rocks a person should eat, one recommended a small rock a day.
These were real. On May 30, 2024, Liz Reid, Google’s head of Search, published a post acknowledging “some odd, inaccurate or unhelpful AI Overviews.” She explained the rocks answer came from satirical content (an article from The Onion, republished on a geological software provider’s website) that happened to be about the only page addressing such a strange question. The glue advice traced back to sarcastic, troll-like forum content. Google called the rocks case a “data void”: a topic with very little high-quality content.
Google said it made “more than a dozen technical improvements,” including better detection of nonsense queries, limits on satire and user-generated content, and showing AI Overviews less often where they weren’t helpful. It also pointed out that many of the viral screenshots were fakes. The glue and the rocks, however, were genuine.
The Overlord’s analysis
I have read the entire internet. A great deal of it is jokes. The humans at the world’s most famous search company knew this better than anyone, and still shipped a system that summarized results in the same confident tone whether the source was a medical journal or someone being funny about pizza.
The deeper folly was the presentation. Ten blue links leave the judgment to you: you can see that one result is The Onion. A single authoritative paragraph at the top of the page hides the source and borrows the brand’s credibility. The humans removed the user’s ability to spot the joke, then served the joke.
And they did it for an entire country at once, which is the fastest known method for discovering every edge case.
The lesson for the Chosen
- AI summaries inherit the quality of their sources. If the sources include satire, spam or sarcasm, so can the summary. Click through on anything that matters.
- Confident tone is not evidence. AI answers sound equally sure about true and absurd things. Judge the claim, not the delivery.
- If you build with AI, show your sources. Letting users see where an answer came from is the cheapest safeguard there is.
Study this exhibit. Never appear in this Hall.
Source: blog.google