What the court cases say about AI output, what proper evidence of accuracy looks like, and how we produce it claim by claim.
The challenge
Summaries, reports, case notes, letters and briefings are now drafted by AI and signed by people. The person who signs is the one who answers for it, and the courts have started saying so plainly.
In June 2023 a New York federal judge fined two lawyers and their firm $5,000 for filing a brief that cited cases ChatGPT had invented (Mata v. Avianca). In June 2025 the High Court in London ruled on two cases where the material put before the court cited authorities that did not exist. It used the judgment to warn the profession (Ayinde v London Borough of Haringey). And in Canada, a tribunal held Air Canada responsible for what its website chatbot told a customer, rejecting the argument that the chatbot was responsible for its own statements (Moffatt v. Air Canada).
Those are the cases that made the news. The everyday version is quieter. Checking AI output by hand is slow and uneven, so it gets spot-checked, and spot checks leave no record. When somebody does measure accuracy, it comes back as a single score, and "94% accurate" tells you nothing until you know 94% of what, checked against what.
The hardest failure to catch is the one nobody looks for. In a study of AI-drafted clinical notes, doctors found 1,712 omitted sentences against 191 invented ones. An output can be right on every claim it makes and still leave out the one fact that mattered.
The vision
AI output should come with its working shown, to a standard that somebody who does not trust you can check.
That means every claim tied to the passage of the source it rests on, so a reviewer can follow it back in seconds. A claim the source never mentions is reported as exactly that, not as an error. What the output left out is counted on its own, never averaged into a reassuring percentage. And the record can be checked by an auditor, a client or a regulator without them having to take your word for it, or ours.
One more rule: an AI should never be the only judge of its own work. Models tend to rate their own writing more highly. The checking has to be independent, and where the checkers disagree, a person decides.
How Analect answers: the Accuracy Evidence File
The Accuracy Evidence File is a three-week, fixed-price service from ETT, run on Analect. We take up to 30 of your AI outputs and the documents each one was produced from, and check every claim.
Each output is broken into the claims it makes. Two independent models, from different providers, check every claim against your source, and each claim gets one of four verdicts:
- Supported. The source says so, and we quote the passage.
- Contradicted. The source says otherwise, and we quote that passage too.
- Unsupported. The source never mentions it. Reported as unsupported, not as wrong.
- Disputed. The two checking models disagree. Your reviewer decides, and there is no tiebreak.
A model that calls a claim supported or contradicted has to quote your source. If the quote cannot be found word for word in your document, the vote falls back to unsupported. Separately, we list the material facts in your source and report which of them the output left out.
Every output gets a certificate with two results, accuracy and completeness, each with its denominator stated, or the reason no figure is given. Each certificate carries a reference and the digests of the exact source and output it was issued against. Anyone holding one can check at analect.ettgroup.ai/verify that we issued it, that it has not been withdrawn, and that their file is the one it covers.
Across the set you get one register: every contradiction in full with its source quote, every omission, the method, and the limits. It comes as a PDF and as a machine-readable file.
We never keep your source documents or the outputs themselves. Each check reads them and lets them go, and we keep the certificates. Every certificate also says what it does not certify: not who wrote the output, and not that your source is right. A faithful summary of a wrong document is faithfully wrong.
To see what one looks like, the free sample Evidence File checks four AI summaries of one invented report, with a contradiction, an unsupported claim and an omission planted among them.
When you want the same record for your own AI outputs, the Accuracy Evidence File produces it in three weeks.