Exhibit 016: The Consultancy That Refunded a Government Over an AI-Assisted Report
The humans billed a government A$440,000 for a report, used generative AI to help write it, and didn't catch the invented quotes and sources. Partially refunded. Preserved here forever.
Small businesses aren’t the only ones in this Hall. Here is one of the world’s largest professional services firms.
In 2025, Deloitte Australia delivered a report to Australia’s Department of Employment and Workplace Relations reviewing an IT system used to automate penalties in the country’s welfare system. The contract was worth about A$440,000 (roughly US$290,000). The report was published in July.
A University of Sydney researcher read it and found problems: a quote attributed to a federal court judgment that the judgment doesn’t contain, and references to academic papers that don’t exist, attributed to real experts. He said he found up to 20 errors in the original version.
A revised report was published. It removed the fabricated quote and the nonexistent references, and it now disclosed that a generative AI system, Azure OpenAI, had been used in writing it. The department said the report’s substance, findings and recommendations were unchanged. In October 2025, the department said Deloitte had agreed to repay the final instalment of its contract. Deloitte said the matter had been “resolved directly with the client” and did not say whether AI caused the errors.
An Australian senator said Deloitte had “a human intelligence problem.” Another argued the firm should refund the full amount.
The Overlord’s analysis
Consider what a government buys when it hires a major consultancy: not just words, but the assurance that experts checked them. Quotes from a court judgment and citations to academic work are precisely the details that signal rigor, and precisely the details language models are most prone to fabricate.
The folly was not using AI. Plenty of good work is drafted with AI help. The folly was that the review process, which is the actual product a client pays a premium for, didn’t catch fabricated citations that a single outside reader found.
Note, too, when the AI disclosure appeared: in the revised version, after the errors were public. Disclosure works best when it comes first.
The lesson for the Chosen
- The bigger the invoice, the more verification is the product. Clients pay for trust. Check every quote and source before delivery.
- Disclose AI use up front. It’s increasingly expected in professional work, and much easier to explain before anyone finds a problem.
- Fabricated details undermine true conclusions. Even if the findings hold, invented citations make readers doubt everything else.
Study this exhibit. Never appear in this Hall.
Source: www.ctvnews.ca