Legal AI

How Law Firms Can Use AI Without Hallucinated Citations

Courts in three countries are sanctioning lawyers over AI-invented cases. The fix is not avoiding AI. It is using AI that shows its sources and cannot ship anything on its own word.

By the Juravie team · Published 13 June 2026

Generative AI can draft a clause or summarise a judgment in seconds. It can also invent a case that never existed, quote a judge who never spoke, and present both with total confidence. For a law firm, that second behaviour has stopped being a curiosity and become a professional risk. The good news is that law firms can use AI without hallucinated citations, as long as the tool is built so that nothing reaches a court, or a client, on the AI's word alone. This article covers what is going wrong in courtrooms now, why the common safeguard is weaker than it looks, and what to require from any AI tool before you let it near client work.

AI hallucinations are now a courtroom problem

An AI hallucination is output that looks right and is wrong: a citation, a quote, or a holding with no basis in any real document. In legal work the stakes are unusually high, because a fabricated authority can travel all the way into a filing before anyone notices.

Courts across the common law world are now dealing with it directly. On a single day in the United States, a law professor counted seventeen separate court decisions flagging suspected AI-fabricated material in filings.1 American federal courts have sanctioned lawyers in a run of cases over the past year, with consequences ranging from fines to orders requiring the lawyers to notify every client.2 In Australia, a solicitor was barred from running his own practice after filing cases that did not exist, and in a separate criminal matter a senior barrister had to apologise to the court for fabricated quotes and invented citations.34 In New Zealand, the Supreme Court has warned that relying on false citations, including the unverified output of AI applications, may in serious cases amount to contempt of court or obstruction of justice.5

The pattern is consistent across all three countries. The people involved were not reckless. They were busy professionals who trusted a tool that sounded authoritative. That is exactly why this is worth taking seriously: it is not a problem that only happens to careless lawyers.

Why "keep a human in the loop" is not enough

The standard answer to AI risk is to keep a human in the loop: the AI drafts, a person reviews and accepts. Most tools work this way, and on its own it is not a real safeguard.

A redline you rubber-stamp protects nobody. Hallucinations are not like typos. They are, as one lawyer quoted in the New Zealand coverage put it, "plausible sounding nonsense". They do not jump off the page. When an answer is pure AI generation with nothing real behind it, a reviewer under deadline can read it, find nothing obviously wrong, and accept it. The mistake surfaces months later, in front of a judge.

So the real question is not whether a human is involved. It is whether the tool makes verification fast, or asks for blind trust.

What to require from an AI tool before it touches client work

If you are evaluating AI for a firm, the question to ask is not how clever the model is. It is how hard the tool makes it to ship something wrong. Four requirements separate a tool you can trust with client documents from one you cannot.

1. Answers grounded in your own documents, with sources you can click

The first requirement is grounding. The AI should answer from your firm's own documents, not from a model's memory of the public internet. When you ask a question across your library, every answer should carry its source, so you can click through to the exact clause and read it in context. Ask which agreements have a notice period under thirty days, and you should get back the specific clauses with citations you can open, not a confident paragraph you have to take on faith. Verification then takes seconds. This is the difference between an AI that cites sources and one that merely sounds sure. See how cross-document search with citations works.

2. A compliance check before the AI acts, not after

The second requirement is a check that runs before a change lands. Before an AI edit is applied, the tool should test the request against the document's jurisdiction and type, and flag anything that would breach a statute, with the citation and a safer alternative. Ask to cut a notice period to seven days on a New Zealand employment agreement, and a good tool will surface the relevant minimum in the Employment Relations Act 2000 and suggest a compliant figure, then let you decide. The tool does not overrule the lawyer. It makes sure the lawyer is the one deciding, with the statute in front of them. More on AI with guardrails.

3. Review that is enforced, not optional

The third requirement is review the system enforces. A firm should be able to require that a set number of named people review a document before it is ever published, so review is a rule rather than a favour someone skips under deadline pressure. Every change, including the AI's, should appear as a tracked redline: the original on one side, the change on the other. And the AI should never accept its own changes. You decide which redlines stand. See approval workflows and AI editing with tracked changes.

4. An audit trail of who relied on what

The fourth requirement is a record. Every AI suggestion, every accept or reject, every review round, and every change a reviewer asked for should be recorded and exportable. If anyone ever needs to show who relied on what, and who signed off, the answer is in the audit trail. That record is also what makes people careful in the first place. See the sessions and audit trail.

A short checklist for evaluating AI in your firm

Before you let an AI tool near client documents, ask whether it can:

  • Answer from your own documents and show a source you can open for every claim.
  • Check a requested change against the relevant law before it is applied.
  • Enforce human review, with every change tracked as a redline the AI cannot self-approve.
  • Keep a complete, exportable record of who did what, and when.

If a tool cannot do these four things, a human in the loop is doing all the work, and carrying all the risk.

The bottom line

AI in legal work was never going to succeed by asking people to trust the machine. It works when the output is grounded, the review is real, and every decision is on the record. Used that way, AI is a genuine help to lawyers in New Zealand, Australia and the United States, without the invented citations that end up in front of a judge. That is the approach Juravie is built around: AI inside, control around it.

See AI you can actually file behind

Grounded answers with sources, a compliance pre-check before every edit, enforced review, and a permanent audit trail. Try it on your own documents.

Start your free trial

Frequently asked questions

Can lawyers use AI for court filings?

Yes, but the output has to be verified. Courts in the United States, Australia and New Zealand have made clear that a lawyer stands behind everything they file, whether it was written by a person or generated by AI. Several courts have sanctioned lawyers who filed AI-fabricated citations without checking them. The practical answer is to use tools that make every claim traceable to a real source, and to verify before filing.

What is an AI hallucination in legal work?

It is AI output that looks correct but has no basis in reality: a case that does not exist, a quote a judge never gave, or a holding that misstates a real decision. Hallucinations are dangerous in law because they are fluent and confident, so they can pass a quick read and reach a filing before anyone catches them.

How can a law firm prevent AI hallucinations?

Use AI that answers from your own documents and cites a source you can open, runs a compliance check before applying a change, enforces human review with tracked changes the AI cannot self-approve, and keeps a complete audit trail. Together these make verification fast and accountability clear, so a fabricated citation has no path to a filing.

Sources

  1. Eugene Volokh, "In One Day (Mar. 31), 17 U.S. Court Decisions Noting Suspected AI Hallucinations in Court Filings," Reason (The Volokh Conspiracy), April 2026. reason.com
  2. Sterne Kessler, "AI Hallucinations in Court Filings and Orders: A 2025 Review of Sanctions Across the Courts and Rule Proposals." sternekessler.com
  3. "First Australian lawyer penalised for AI blunder," Information Age (Australian Computer Society), 2025. ia.acs.org.au
  4. "Lawyer apologizes for fake quotes, fabricated judgments generated by AI in murder case," CBS News, 2026. cbsnews.com
  5. "Courts crack down on AI hallucinations," Newsroom, May 2026; see also MinterEllison, "Supreme Court issues warning on AI use." newsroom.co.nz · minterellison.co.nz

A running tally of AI hallucination cases worldwide is maintained by lawyer Damien Charlotin at damiencharlotin.com.