Accuracy Assurance Pack
The Accuracy Assurance Pack is a fixed-price service from ETT that runs for one quarter. Each month we certify a set of one AI system's outputs against their sources, your reviewers record a decision on every flagged output, and at the end you receive a signed pack that maps the evidence clause by clause to ISO/IEC 42001 and the EU AI Act, with what it does not cover stated beside it. It runs on Analect.
Your auditor will ask how you know. One certificate cannot answer that
A certificate shows whether one AI output agrees with its source. An ISO/IEC 42001 auditor, or anyone applying the EU AI Act, asks something else: whether you measure accuracy as a routine, against criteria you set, and whether people act on what the measurement finds.
That evidence is usually assembled by hand when somebody asks for it, from spot checks, spreadsheets and email threads. It rarely says what was checked against what, who looked at the failures, or what they decided.
- Accuracy checks that happen, but leave no record an auditor can follow.
- No acceptance criteria written down, so nobody can say whether a result was acceptable.
- Flagged outputs fixed quietly, with no decision recorded and no reason given.
Evidence for an audit, not a certificate of conformity
Neither framework certifies an AI output, so no product can certify one for you. What the pack gives you is the evidence both frameworks ask for, mapped to the clause it answers, with what it does not cover stated beside it.
ISO/IEC 42001
Certifies an organisation's AI management system, through an accredited certification body. The auditor asks for evidence that your controls operate: that accuracy is verified against stated criteria, that operation is monitored, and that people act on what monitoring finds.
The EU AI Act
Assesses an AI system and its provider's quality management. Providers of high-risk systems declare accuracy levels and metrics, document their testing, design human oversight and monitor the system once it is in use. Deployers monitor its operation and keep its logs.
This pack does not confer a presumption of conformity with the EU AI Act, and it is not a certification under ISO/IEC 42001.
The EU AI Act at 2 October 2026. Annex III high-risk obligations apply from 2 December 2027 and Annex I from 2 August 2028, after the 2026 amendment. Article 50 has applied since 2 August 2026. No harmonised standard for Article 15 was cited in the Official Journal at the retrieval date.
Certificates over time, your criteria, and your people’s decisions
Every output is still certified the way the Accuracy Evidence File certifies it, claim by claim against its source, with omissions on their own axis. The pack adds what an auditor asks about next.
- 01
The system
We define the AI system with you: what it is for, what produces its outputs, how you classify it under the AI Act and your role. Your statements, recorded as yours, and kept as a new version when they change.
- 02
Your criteria
You set what counts as acceptable, such as no contradicted claim in any output. Every certificate is checked against the criteria in force when it ran.
- 03
Your reviewers
Every flagged output goes to a reviewer you name, who records a decision and a reason within a window you set.
- 04
The pack
At the end of the quarter, the evidence in one signed pack, mapped clause by clause and checkable by anyone you give it to.
What sends an output to your reviewer
- A claim contradicted by the source
- A claim that the source says something it does not
- Checks that did not settle, so no grade was issued
- A claim the two checking models disagree on
- A consequential fact the output left out
- Any of your per-output criteria breached
A plain claim the source never mentions does not send an output for review. It is unsupported, not wrong.
What your reviewer records
- Accepted
- Used as it stands. The reviewer judged the finding immaterial, and says why.
- Corrected
- Corrected before use. The corrected text is not kept.
- Withdrawn
- Not used, or withdrawn.
- Escalated
- Raised as an incident in your own process, with its reference.
- Finding disputed
- The reviewer believes the certificate is wrong about this output.
A review never changes a verdict. When a reviewer disputes a finding, the pack counts it as evidence about our checker and prints it beside the checker's own record. Reviews by our engagement lead are labelled ETT and counted apart from yours, because the oversight has to be your own.
Every clause rated, and nothing above partial
The pack lists each clause it bears on, from a dated card per framework, and rates the evidence for it. Partial is the highest rating it ever prints, because a check on outputs cannot show a whole management system or a whole conformity assessment.
The pack is direct evidence for part of what this asks.
Useful input to it, not evidence of the requirement itself.
Listed so nobody assumes it is covered.
- Clauses about monitoring rate as context until at least two calendar months hold certificates.
- Clauses about verification need criteria you have set, and a measured record of our checker for the exact models and settings used.
- Clauses about oversight need decisions by your own reviewers.
- Documented information rates as context if any certificate in the quarter is unsigned.
- Clauses that do not apply to the role or classification you state, or that apply only from a later date, are shown greyed with the reason and their rating still worked out, so you can see where you would stand. For a high-risk system under Annex III, that is every EU AI Act clause before 2 December 2027.
| Clause | What it asks about | Highest rating |
|---|---|---|
| 7.5 | Documented information | Partial |
| 9.1 | Monitoring, measurement, analysis and evaluation | Partial |
| 10.2 | Nonconformity and corrective action | Partial |
| A.6.2.4 | AI system verification and validation | Partial |
| A.6.2.6 | AI system operation and monitoring | Partial |
| A.6.2.8 | AI system recording of event logs | Context |
| A.7.5 | Data provenance | Context |
| A.8.2 | System documentation and information for users | Context |
| A.8.4 | Communication of incidents | Context |
| A.9.2 | Processes for responsible use of AI systems | Partial |
| A.9.4 | Intended use of the AI system | Context |
| A.5 (all) | Assessing impacts of AI systems | Not evidenced |
| A.7.4 | Quality of data for AI systems | Not evidenced |
| Clause | What it asks about | Falls on | Highest rating |
|---|---|---|---|
| Art. 9 | Risk management system | Provider | Context |
| Art. 10 | Data and data governance | Provider | Not evidenced |
| Art. 11, Annex IV point 2(g) | Validation and testing procedures, and the metrics used | Provider | Partial |
| Annex IV point 3 | Capabilities and limitations, including degrees of accuracy | Provider | Partial |
| Art. 12 | Automatic recording of events by the system | Provider | Context |
| Art. 13 | Instructions for use: accuracy level and metrics | Provider | Partial |
| Art. 14 | Human oversight | Provider, Deployer | Partial |
| Art. 15(1)-(3) | Accuracy, and declaring its level and metrics | Provider | Partial |
| Art. 15(4) | Robustness | Provider | Not evidenced |
| Art. 15(5) | Cybersecurity | Provider | Not evidenced |
| Art. 17 | Quality management system | Provider | Not evidenced |
| Art. 19, Art. 26(6) | Keeping logs for at least six months | Provider, Deployer | Context |
| Art. 26(5) | Deployer monitors operation | Deployer | Partial |
| Art. 27 | Fundamental rights impact assessment | Deployer | Not evidenced |
| Art. 50 | Transparency to people interacting with AI | Provider, Deployer | Not evidenced |
| Art. 72 | Post-market monitoring | Provider | Partial |
| Art. 73 | Reporting serious incidents | Provider | Context |
Each card records who reviews its mapping and when. Until a card has been independently reviewed, every pack that uses it says so on its cover.
We measure our own checker, and publish where it falls short
The checker is itself AI grading AI output, so the first question about any accuracy figure is how good the grader is. We run a battery of synthetic outputs with planted faults through it, three times each, and record what it found.
A pack quotes the record only when its certificates ran on exactly the same models and settings. Otherwise it says the checker is not validated for that configuration, and the clauses that depend on it rate lower. No pass mark is set.
- Planted contradictions caught: a negation, a changed number, a swapped date or name, and one in the sentence after the passage that matches48 of 48
- Planted omissions found, in short outputs and in long ones24 of 24
- True claims the source never makes, wrongly called contradicted0 of 15
- Claims untouched by any planted fault, wrongly called contradicted2 of 1,646
- Outputs given the same overall band in every run34 of 34
- Claims whose verdict changed between runs of the same output33 of 569
- Misattributions caught, where an output says the source states something it does not9 of 12The rest: 3 disputed, left for a person.
- Faithful outputs given a clean result12 of 15The rest: 3 indeterminate, meaning the checks did not settle and no grade was issued.
- Facts carried in words unlike their own, wrongly recorded as dropped6 of 21Our weakest result. The record says so, and a pack quotes it.
Measured on 2 October 2026: 34 synthetic outputs over 3 sources, each run 3 times, 102 runs in all. Counts are planted cases across all three runs, and the run-to-run rows compare each output’s runs with one another.
What a pack records
One pack per system per quarter, written for the person who has to rely on it without taking our word for anything.
Delivered as PDF, as signed JSON, and as a coverage table in CSV for your governance, risk and compliance tool.
- Validity, first
- What could weaken the pack, before any figure: reviews outstanding or overdue, certificates unsigned or withdrawn, configurations without a checker record.
- The system
- As you described it, with every earlier version in force during the quarter.
- Accuracy and completeness
- Per configuration and never pooled: verdict totals with their denominators, the lowest, median and highest per output, and a monthly series. Omissions on their own axis.
- Your criteria
- Every version in force, and every breach.
- Oversight
- Your reviewers' decisions by type, the median days to decide, what is outstanding or overdue, every escalation with its incident reference, and each output's review state.
- The checker
- The measured record for the models and settings used, or a statement that there is none.
- Coverage
- Each clause, the evidence for it, its rating, why it was rated lower, and what you still have to show.
- The register
- Every certificate's reference and digest, so each one can still be checked on its own.
- What it does not evidence
- A list in the file, not a footer, opening with the sentence under Read this first.
- Signature and reference
- The JSON is signed with a published key and registered under an AAP- reference. With that file, anyone can check that we issued it, that it has not been withdrawn, that their copy matches, that the signature is good, and that every certificate it lists is still current.
One quarter, with your reviewers on every flagged output
A quarter, because monitoring is only evidence over time: with fewer than two months of certificates, the pack rates its monitoring clauses as context.
- Weeks 1 and 2
Set up
We define the system with you, agree your criteria, name your reviewers and their window, and set how long the records are kept. We certify your first three outputs together.
Yours: A system owner, your reviewers, and the first outputs with their sources.
- Months 1 to 3
Certify and review
Each month we certify up to 30 outputs, chosen the way you state: every output, a random sample or a targeted set. Flagged outputs go to your reviewers, and we meet for thirty minutes at each month end.
Yours: Each month's outputs and sources, and reviewer time on what is flagged.
- Week 13
Issue
We issue the signed pack for the quarter and present it to your risk or compliance lead and, if you wish, your auditor.
Yours: A sixty-minute readout.
From you
- One AI system in use, and the person who owns it.
- Up to 30 outputs a month and the source documents each was produced from, with what produced each one: the system or model, the date and the purpose or prompt. We record these as you state them.
- How you classify the system under the EU AI Act and your role as provider or deployer, or that you have not decided. We record your statement. We do not assess it.
- Reviewers who know the subject, for a few hours a month depending on how much is flagged.
- Pasted text, PDF or Word, up to roughly a 200-page source and a 40-page output each.
Where it goes
- We never keep your source documents or the outputs themselves. Each check reads them and lets them go.
- We keep the certificates, the system record, your criteria and your reviewers' decisions, in a private store only named people at your organisation and at ETT can open.
- A pack holds only what its certificates hold, plus the system record and the reviews. If any month's file leaves source facts out, every pack for the system does too.
- Records are kept for at least the period you set, which is at least six months where the EU AI Act is one of the system's frameworks, and cannot be deleted inside it without an administrator's recorded override.
- The checks run on models from Anthropic and OpenAI, reached through Vercel's AI Gateway under their commercial API terms.
What the Assurance Pack is not
Not a certification
It is evidence you can put in front of the auditor or assessor who does that work, mapped to the clauses they will ask about.
Not a classification
Whether your system is high-risk, and whether you are its provider or deployer, are your statements. The pack records them and does not decide them.
Not a full assessment
Robustness, cybersecurity, bias, data governance, risk management, impact assessment and your quality management system are not measured. The pack lists them as not evidenced.
Not a substitute for your people
Your reviewers decide what happens to every flagged output. The pack records what they decided.
One fixed fee covers
- One AI system, over one quarter
- Up to 30 outputs a month certified against their sources, with omissions, in an Evidence File for each month
- Setup, a check-in at each month end, and a readout for your risk or compliance lead
- The pack as PDF, signed JSON and coverage CSV
- Logins for your system owner and reviewers for the quarter, and 30 days after delivery
- Optional: further quarters on the same system, or more outputs in packs of 25
What we agree before the quarter starts
- Which system, and which of its outputs are worth certifying
- How you classify it and your role, or that you have not yet decided
- Your acceptance criteria, and how quickly your reviewers can respond
- How outputs will be chosen each month: all of them, a random sample or a targeted set
About the Accuracy Assurance Pack
Does the pack certify us under ISO/IEC 42001 or the EU AI Act?
No. ISO/IEC 42001 certification is of your management system, by an accredited certification body, and the AI Act's conformity assessment is of the system and its provider. Nothing we produce does either. The pack is evidence for both: it shows that you measure output accuracy against criteria you set and that your people act on what is found, mapped to the clauses that ask for that.
Our system may not be high-risk. Is the pack still useful?
Yes. ISO/IEC 42001 applies whatever the classification, and a quarter of monitoring is a record you keep if the classification changes. AI Act clauses that do not apply to the classification and role you state, or that apply only from a later date, are shown greyed with the reason.
Why a quarter rather than three weeks?
Because monitoring is only evidence over time. With fewer than two calendar months of certificates, the pack rates its monitoring clauses as context rather than partial evidence. Your first certificates still arrive in the first two weeks.
Who decides what happens to a flagged output?
Your reviewers. Each records a decision and a reason, and a review never changes a verdict. Reviews by our engagement lead are labelled ETT and counted apart, because the oversight the frameworks ask about is yours.
Can our auditor check the pack?
Yes. Each pack's JSON is signed with a key we publish and registered under an AAP- reference. With that file, anyone can check at analect.ettgroup.ai/verify that we issued it, that it has not been withdrawn, that their copy matches, that the signature is good, and that every certificate it lists is still current. The reference alone checks issue, withdrawal and the signature.
Do you keep our documents?
No. Each check reads the source and the output and lets them go. We keep the certificates, the system record, your criteria and your reviewers' decisions, for the retention period you set.
Start with one system
A thirty-minute scoping call to choose the system, agree how you classify it and what counts as acceptable, and name your reviewers. Your first certificates arrive in the first two weeks.