In April this year, government gazetted a draft National AI Policy. Within weeks it was withdrawn in full, because several of the academic sources it cited did not exist. They had been produced by an AI tool and nobody checked them.
It is an uncomfortable story for the department involved, and a useful one for everyone else. The failure was not the technology. The failure was the missing step between the tool producing something and a person publishing it.
What actually happened
A draft National AI Policy was approved by Cabinet in March 2026 and gazetted in April. News24 then found that a number of the academic citations in the document could not be traced to any real paper. The Minister of Communications and Digital Technologies announced the withdrawal at the end of April, the retraction was gazetted in June, and two officials were suspended. A revised draft is expected to go back to Cabinet later this year.
Worth being clear on one point, because it gets muddled in the coverage: there is still no AI-specific law binding on South African businesses. POPIA, RICA and the Cybercrimes Act already apply to how you handle information, and they applied before anyone had heard of Copilot. The AI policy work is about what comes next.
The part that applies to your business
If a government department, with reviewers and a legal process, can publish a document containing invented sources, an ordinary business can do it far more easily. And most of the risk sits in perfectly mundane places:
- A quote for a client, with figures the tool produced and nobody recalculated.
- A policy or contract clause, generated in one pass, that references legislation which does not say what the tool claims.
- A report to a customer, with statistics that read convincingly and have no source behind them.
AI tools are useful precisely because they produce confident, well-structured output quickly. That is also exactly why an error slips past. Nothing about a fabricated citation looks wrong on the page.
Three habits that prevent it
None of this requires banning AI, and we would not recommend that. It requires the same discipline you would apply to work from a new junior staff member.
1. Verify anything factual before it leaves the business
Numbers, legal references, quotes, statistics and citations get checked against a real source. Not skimmed. Checked. If the source cannot be found in a minute, treat the claim as wrong.
2. Name the person accountable for the output
Whoever sends it, owns it. That single rule prevents most of the trouble, because it removes the quiet assumption that the tool was responsible for the accuracy.
3. Write it down
An AI usage policy is where these expectations stop being something people assume and start being something they agreed to. It also covers the other half of the problem: which information may go into a chat box in the first place.
Where most businesses are exposed right now
In practice, when we look at a business that has not addressed this, we find staff using personal AI accounts on their own logins. Client information goes in, nothing is logged, and the business has no visibility over any of it. That is a more immediate risk than a fabricated citation, and it is usually fixed by moving people onto properly licensed business accounts and setting sensible boundaries on what those accounts can see.
The verification habit and the account question are two halves of the same job. Neither takes long. Both are much cheaper than retracting something you have already sent.
Our AI usage policy template gives you a working set of rules to adapt, including a section on checking what AI gives you. Word and PDF, six fields to complete. Get the template.
Sources: CNBC Africa, The Next Web. This article is general information, not legal advice.