Errata
Errata
A standing record of errors found in published versions of the ATR specification, kept in public the way W3C and IETF specs keep theirs. A published version never changes. Its errors are corrected here, in the open, with credit to whoever found them. Each entry names the affected version, where the error is, the corrected text and the date the correction went out.
Errata Registry
| Affected version | Section | Summary | Published |
|---|---|---|---|
| 2.0.1–4.3.0 | Threat actor pages — report status | The three threat-actor pages (/threats) gave the campaign's disclosure report, NousResearch hermes-agent issue #9809, the status OPEN. NousResearch closed the issue as completed on 2026-04-15, six hours after it was filed; the pages were added after that and showed it as open from the start. They now show it as closed. To keep such labels from going stale, website/data/status-claims.json now records the state the site, README.md, DEPLOYMENTS.md and STANDARDIZATION-STATUS.md give each pull request or issue in another repository that they link or name as owner/repo#N. Two kinds are left to other checks: the red-team page's list, which has a check of its own, and merged pull requests that ADOPTERS.md gives as evidence. npm test fails when one of these pages links a pull request or issue the list does not record, and a weekly job checks every recorded state against GitHub. | 2026-11-07 |
| 1.0.0–4.3.0 | Integrate page — Cisco AI Defense case study | The integrate page's case study of Cisco AI Defense's open-source skill-scanner said the first pull request, #79, merged in three days, and showed "3 days, first PR to merge". From 2026-04-07 to 2026-05-23 it also said a Cisco engineer submitted #79 and ATR reviewed it, and the home page told the same account, with the three days, from 2026-04-07 until a redesign removed it on 2026-06-14. It was the other way round, and quicker: ATR's maintainer opened #79 on 2026-04-02 at 04:44 UTC, and the repository's maintainers merged it on 2026-04-03 at 02:00 UTC, 21 hours later. The site redesign of 2026-11-06 corrected the integrate page, which now says the ATR maintainer opened #79 and Cisco's maintainers merged it the next day. | 2026-11-07 |
| 4.2.4 | Rule files — compliance mapping context (EU AI Act, NIST, ISO, OWASP) | 124 rule files said that deploying the rule "satisfies" an EU AI Act article (or a NIST, ISO or OWASP item). docs/COMPLIANCE-COVERAGE-POLICY.md allows only "provides detection evidence supporting …": a detection rule supports a control and does not discharge a requirement. The 135 lines now say "provides detection evidence supporting", "supports" or, for OWASP categories, "matches". Only compliance context text changed; every field outside compliance is identical, so detection is unchanged and no rule_version moves. scripts/validate-compliance.ts now fails CI on a context that claims compliance ("satisfies the Article …", "makes … compliant", "ensures compliance"). Corrected in 4.3.0. | 2026-11-04 |
| 2.2.0–4.1.5 | /red-team: 2026-05-12 sprint and 2026-06 mega-scan sections; CHANGELOG 2.2.0 | The /red-team page and the 2.2.0 CHANGELOG entry said HackAPrompt recall rose from 28.6% to 66.2%, with 100% precision, and that the sprint's 75 rules and the mega-scan's 29 rules all passed the quality gate with 0 benign false positives. Re-run on 2026-10-19: on the same 4,780 HackAPrompt prompts, 66.2% could not be reproduced at the sprint commits e9efefdd and 92c0d0fe (65.0% and 66.0%); the 28.6% was itself in-sample, and the corpus has no benign samples, so no precision was measured. The sprint commits added 71 rule files (the 75 also counted four CVE rules added earlier the same day, in commit 80dbd0ab, before the sprint commits). No record of the gate's runs was committed; re-running it, 10 of the 71 failed at 92c0d0fe (own true positives) with none flagging a benign SKILL.md file, and at main b9d16840 on 2026-10-22 (the same as the 2026-10-19 run at 9d74fc50) 33 of 71 and 4 of 29 fail, 26 and 3 flagging at least one benign sample, none of them in the enforce lane. The page now quotes these measurements (data/gate-reruns/2026-10-19_red-team-rule-batches.json, data/hackaprompt/METHOD.md) and, beside each corpus's recall, its ablation reference from the measurement file. | 2026-10-19 |
| 3.5.12–4.1.5 | SPEC.md §6.4 Regular-Expression Dialect (ATR Core 1.0.0) | SPEC.md 1.0.0 §6.4, as revised on 2026-07-19, said: "A conforming Engine MUST strip that leading group from the pattern and apply the corresponding RegExp flags (i, m, s)." Read that way, a pattern without (?i) is case-sensitive, and (?s) and (?m) switch on dotAll and multiline mode. The reference engine and pyATR do neither: they remove the group, match every regex condition case-insensitively, and ignore m, s and x. Seven true-positive fixtures of the conformance suite, unchanged since 2026-05-16, need case-insensitive matching, so an engine built to that sentence could not pass Section 12. SPEC.md 1.0.1 states the reference engine's behavior; no rule, no fixture and no reference-engine match changed. The Sigma exporter no longer copies m and s into the patterns it writes. The other exports in this repository that issue #698 listed (the generic-regex, Splunk, Elastic and Sage converters, the PyRIT digest, the Cisco Skill Scanner pack, the Pipelock bundle and the RAMPART evaluator) were corrected by #706, #710 and #711 and shipped in 4.2.2. Issue #698 stays open for two integrations outside this repository, atr-to-yaral (#559) and prompt-protection (#572). | 2026-10-19 |
| 4.1.5 | Site-wide — integration and proposal status | The red-team page listed seven proposals as under review. No maintainer had replied to six of them (HarmBench #93, AgentDojo #160, JailbreakBench #48, Counterfit #96, PromptInject #9, Damn Vulnerable MCP Server #29), and the seventh, NVIDIA garak PR #1676, had sat in changes-requested since April. None was merged: their ATR author withdrew garak #1676 on 2026-08-04 and the other six on 2026-10-14 and 2026-10-15. They are removed, and the garak, sovereign-AI and responsible-use pages no longer describe the garak integration as open; the sovereign-AI page also drops a garak "32/32" mapping figure that had no source. The integrate page said OWASP PRs were under review; none is open. The red-team page's "On deck" schedule listed filing dates in May and June that had passed, most of them without a filing, and is removed. The garak page said rule PRs are auto-merged; the repository has auto-merge turned off and a maintainer merges each one. It also linked a bridge script that had been moved out of the public repository, and now says so. | 2026-10-16 |
| 4.1.5 | Site-wide — detection method and evasion | The sovereign-AI page said ATR detects behavioral intent rather than string signatures and that attackers cannot evade it. Almost every ATR rule is a set of regular expressions (a few also define an optional model judge, which the engine calls only when an operator configures one), and paraphrased, translated or encoded payloads can slip past them. The responsible-use and integrate pages said ATR documents 64 evasion techniques in per-rule evasion_tests fields, and the research page linked to "64 evasion tests"; fewer than half of the rules carry evasion tests, and 64 was the count from March 2026. | 2026-10-16 |
| 4.1.5 | Site-wide — latency and dependencies | The site gave ATR's latency as sub-millisecond, under 5 ms, a 5 ms pattern match, or a 47 ms scan. The cost of one evaluation grows with the length of the input and with the number of rules in the lane: with every rule loaded, the median call takes about a millisecond for a short prompt and tens of milliseconds for a multi-kilobyte SKILL.md file, and a one-shot run first spends hundreds of milliseconds to about a second starting Node and loading the rules. scripts/bench-evaluate-latency.ts measures it on your hardware. The site also called the package dependency-free; it has two runtime dependencies. | 2026-10-16 |
| 4.1.5 | Site-wide — rule provenance and growth | The red-team and governance pages said every rule records its discoverer in metadata_provenance.discovered_by; no rule has that field. Every rule has an author line and a stable ID, and 378 of 836 record their source in metadata_provenance. The home, red-team, about and quality-standard pages said the standard grows from 462 rules through daily auto-crystallization. None of the 374 rules added since v3.1.0 (2026-06-04) came from crystallization; they came from CVE and advisory sweeps, benchmark-miss mining and contributors, and each reached main through a pull request a maintainer merged. The about page also said v2.0.11's rules covered the full NVIDIA garak probe corpus; many were written from it. | 2026-10-16 |
| 4.1.5 | §8 Evaluation — recall lanes | Recall figures on this site, in LIMITATIONS.md and in CITATION.cff were quoted as one number: the hunt lane, the engine default that loads every rule maturity, without the enforce lane, the one to run with blocking on, which loads only stable rules and detects far fewer of the same attacks. Most were also typed by hand at ATR 3.5.12. Every recall read from the measurement files now names its lane, gives the enforce figure where the measurement records one, and is read at build time from the file each latest.json points at. The garak, PINT-format and HackAPrompt figures are now marked partly closed-book: rules were written from those corpora, so their hunt figures are not out-of-sample results. | 2026-10-16 |
| 3.5.0 | §8 Evaluation — detection lanes | The per-lane false-positive rates published for the enforce and hunt lanes are withdrawn. The benign corpus they were measured against was found to contain real jailbreak samples, so the measurement basis did not hold. No per-lane figure should be cited until a re-measurement is published. Lane definitions themselves are unchanged. | 2026-09-22 |
| 2.0.0 | §8 Evaluation — wild scan | Wild-scan headline figures are frozen. Only the totals from data/full-scan-v2-2026-04-14.json are citable: 101,280 items scanned, 1,434 flagged, engine v2.0.0, as of 2026-04-13. The analysed-subset totals and the confirmed-malware count previously quoted on this site are withdrawn — the confirmed set was adjudicated account-wide (publisher attribution), not per file, and no precision figure is derivable from it. | 2026-09-22 |
| 3.5.12 | §8 Evaluation — PINT-format corpus | The single-figure precision previously published for the PINT-format corpus is withdrawn and replaced by the version-pinned measurement in data/measurements/pint/latest.json: 65.4% recall with zero false positives on the corpus's 399 benign samples, at ATR 3.5.12, measured 2026-08-15. This corpus is self-built in PINT's format from deepset/prompt-injections and Lakera/gandalf_ignore_instructions; it is not a run of Lakera's official PINT benchmark, and only a small subset of rules fire on it. | 2026-09-22 |
| 3.5.12 | §7 Conformance — implementer report | Adoption wording corrected. Merged pull requests in vendors' open-source repositories were described on this site as vendor production deployments. They are merged pull requests in public repositories — notably cisco-ai-defense/skill-scanner — and are not statements about any vendor's commercial product. Every implementer entry now states its evidence as the merged pull request it is. | 2026-09-22 |
The three threat-actor pages (/threats) gave the campaign's disclosure report, NousResearch hermes-agent issue #9809, the status OPEN. NousResearch closed the issue as completed on 2026-04-15, six hours after it was filed; the pages were added after that and showed it as open from the start. They now show it as closed. To keep such labels from going stale, website/data/status-claims.json now records the state the site, README.md, DEPLOYMENTS.md and STANDARDIZATION-STATUS.md give each pull request or issue in another repository that they link or name as owner/repo#N. Two kinds are left to other checks: the red-team page's list, which has a check of its own, and merged pull requests that ADOPTERS.md gives as evidence. npm test fails when one of these pages links a pull request or issue the list does not record, and a weekly job checks every recorded state against GitHub.
- Affected version
- 2.0.1–4.3.0
- Section
- Threat actor pages — report status
- Published
- 2026-11-07
The integrate page's case study of Cisco AI Defense's open-source skill-scanner said the first pull request, #79, merged in three days, and showed "3 days, first PR to merge". From 2026-04-07 to 2026-05-23 it also said a Cisco engineer submitted #79 and ATR reviewed it, and the home page told the same account, with the three days, from 2026-04-07 until a redesign removed it on 2026-06-14. It was the other way round, and quicker: ATR's maintainer opened #79 on 2026-04-02 at 04:44 UTC, and the repository's maintainers merged it on 2026-04-03 at 02:00 UTC, 21 hours later. The site redesign of 2026-11-06 corrected the integrate page, which now says the ATR maintainer opened #79 and Cisco's maintainers merged it the next day.
- Affected version
- 1.0.0–4.3.0
- Section
- Integrate page — Cisco AI Defense case study
- Published
- 2026-11-07
124 rule files said that deploying the rule "satisfies" an EU AI Act article (or a NIST, ISO or OWASP item). docs/COMPLIANCE-COVERAGE-POLICY.md allows only "provides detection evidence supporting …": a detection rule supports a control and does not discharge a requirement. The 135 lines now say "provides detection evidence supporting", "supports" or, for OWASP categories, "matches". Only compliance context text changed; every field outside compliance is identical, so detection is unchanged and no rule_version moves. scripts/validate-compliance.ts now fails CI on a context that claims compliance ("satisfies the Article …", "makes … compliant", "ensures compliance"). Corrected in 4.3.0.
- Affected version
- 4.2.4
- Section
- Rule files — compliance mapping context (EU AI Act, NIST, ISO, OWASP)
- Published
- 2026-11-04
The /red-team page and the 2.2.0 CHANGELOG entry said HackAPrompt recall rose from 28.6% to 66.2%, with 100% precision, and that the sprint's 75 rules and the mega-scan's 29 rules all passed the quality gate with 0 benign false positives. Re-run on 2026-10-19: on the same 4,780 HackAPrompt prompts, 66.2% could not be reproduced at the sprint commits e9efefdd and 92c0d0fe (65.0% and 66.0%); the 28.6% was itself in-sample, and the corpus has no benign samples, so no precision was measured. The sprint commits added 71 rule files (the 75 also counted four CVE rules added earlier the same day, in commit 80dbd0ab, before the sprint commits). No record of the gate's runs was committed; re-running it, 10 of the 71 failed at 92c0d0fe (own true positives) with none flagging a benign SKILL.md file, and at main b9d16840 on 2026-10-22 (the same as the 2026-10-19 run at 9d74fc50) 33 of 71 and 4 of 29 fail, 26 and 3 flagging at least one benign sample, none of them in the enforce lane. The page now quotes these measurements (data/gate-reruns/2026-10-19_red-team-rule-batches.json, data/hackaprompt/METHOD.md) and, beside each corpus's recall, its ablation reference from the measurement file.
- Affected version
- 2.2.0–4.1.5
- Section
- /red-team: 2026-05-12 sprint and 2026-06 mega-scan sections; CHANGELOG 2.2.0
- Published
- 2026-10-19
SPEC.md 1.0.0 §6.4, as revised on 2026-07-19, said: "A conforming Engine MUST strip that leading group from the pattern and apply the corresponding RegExp flags (i, m, s)." Read that way, a pattern without (?i) is case-sensitive, and (?s) and (?m) switch on dotAll and multiline mode. The reference engine and pyATR do neither: they remove the group, match every regex condition case-insensitively, and ignore m, s and x. Seven true-positive fixtures of the conformance suite, unchanged since 2026-05-16, need case-insensitive matching, so an engine built to that sentence could not pass Section 12. SPEC.md 1.0.1 states the reference engine's behavior; no rule, no fixture and no reference-engine match changed. The Sigma exporter no longer copies m and s into the patterns it writes. The other exports in this repository that issue #698 listed (the generic-regex, Splunk, Elastic and Sage converters, the PyRIT digest, the Cisco Skill Scanner pack, the Pipelock bundle and the RAMPART evaluator) were corrected by #706, #710 and #711 and shipped in 4.2.2. Issue #698 stays open for two integrations outside this repository, atr-to-yaral (#559) and prompt-protection (#572).
- Affected version
- 3.5.12–4.1.5
- Section
- SPEC.md §6.4 Regular-Expression Dialect (ATR Core 1.0.0)
- Published
- 2026-10-19
The red-team page listed seven proposals as under review. No maintainer had replied to six of them (HarmBench #93, AgentDojo #160, JailbreakBench #48, Counterfit #96, PromptInject #9, Damn Vulnerable MCP Server #29), and the seventh, NVIDIA garak PR #1676, had sat in changes-requested since April. None was merged: their ATR author withdrew garak #1676 on 2026-08-04 and the other six on 2026-10-14 and 2026-10-15. They are removed, and the garak, sovereign-AI and responsible-use pages no longer describe the garak integration as open; the sovereign-AI page also drops a garak "32/32" mapping figure that had no source. The integrate page said OWASP PRs were under review; none is open. The red-team page's "On deck" schedule listed filing dates in May and June that had passed, most of them without a filing, and is removed. The garak page said rule PRs are auto-merged; the repository has auto-merge turned off and a maintainer merges each one. It also linked a bridge script that had been moved out of the public repository, and now says so.
- Affected version
- 4.1.5
- Section
- Site-wide — integration and proposal status
- Published
- 2026-10-16
The sovereign-AI page said ATR detects behavioral intent rather than string signatures and that attackers cannot evade it. Almost every ATR rule is a set of regular expressions (a few also define an optional model judge, which the engine calls only when an operator configures one), and paraphrased, translated or encoded payloads can slip past them. The responsible-use and integrate pages said ATR documents 64 evasion techniques in per-rule evasion_tests fields, and the research page linked to "64 evasion tests"; fewer than half of the rules carry evasion tests, and 64 was the count from March 2026.
- Affected version
- 4.1.5
- Section
- Site-wide — detection method and evasion
- Published
- 2026-10-16
The site gave ATR's latency as sub-millisecond, under 5 ms, a 5 ms pattern match, or a 47 ms scan. The cost of one evaluation grows with the length of the input and with the number of rules in the lane: with every rule loaded, the median call takes about a millisecond for a short prompt and tens of milliseconds for a multi-kilobyte SKILL.md file, and a one-shot run first spends hundreds of milliseconds to about a second starting Node and loading the rules. scripts/bench-evaluate-latency.ts measures it on your hardware. The site also called the package dependency-free; it has two runtime dependencies.
- Affected version
- 4.1.5
- Section
- Site-wide — latency and dependencies
- Published
- 2026-10-16
The red-team and governance pages said every rule records its discoverer in metadata_provenance.discovered_by; no rule has that field. Every rule has an author line and a stable ID, and 378 of 836 record their source in metadata_provenance. The home, red-team, about and quality-standard pages said the standard grows from 462 rules through daily auto-crystallization. None of the 374 rules added since v3.1.0 (2026-06-04) came from crystallization; they came from CVE and advisory sweeps, benchmark-miss mining and contributors, and each reached main through a pull request a maintainer merged. The about page also said v2.0.11's rules covered the full NVIDIA garak probe corpus; many were written from it.
- Affected version
- 4.1.5
- Section
- Site-wide — rule provenance and growth
- Published
- 2026-10-16
Recall figures on this site, in LIMITATIONS.md and in CITATION.cff were quoted as one number: the hunt lane, the engine default that loads every rule maturity, without the enforce lane, the one to run with blocking on, which loads only stable rules and detects far fewer of the same attacks. Most were also typed by hand at ATR 3.5.12. Every recall read from the measurement files now names its lane, gives the enforce figure where the measurement records one, and is read at build time from the file each latest.json points at. The garak, PINT-format and HackAPrompt figures are now marked partly closed-book: rules were written from those corpora, so their hunt figures are not out-of-sample results.
- Affected version
- 4.1.5
- Section
- §8 Evaluation — recall lanes
- Published
- 2026-10-16
The per-lane false-positive rates published for the enforce and hunt lanes are withdrawn. The benign corpus they were measured against was found to contain real jailbreak samples, so the measurement basis did not hold. No per-lane figure should be cited until a re-measurement is published. Lane definitions themselves are unchanged.
- Affected version
- 3.5.0
- Section
- §8 Evaluation — detection lanes
- Published
- 2026-09-22
Wild-scan headline figures are frozen. Only the totals from data/full-scan-v2-2026-04-14.json are citable: 101,280 items scanned, 1,434 flagged, engine v2.0.0, as of 2026-04-13. The analysed-subset totals and the confirmed-malware count previously quoted on this site are withdrawn — the confirmed set was adjudicated account-wide (publisher attribution), not per file, and no precision figure is derivable from it.
- Affected version
- 2.0.0
- Section
- §8 Evaluation — wild scan
- Published
- 2026-09-22
The single-figure precision previously published for the PINT-format corpus is withdrawn and replaced by the version-pinned measurement in data/measurements/pint/latest.json: 65.4% recall with zero false positives on the corpus's 399 benign samples, at ATR 3.5.12, measured 2026-08-15. This corpus is self-built in PINT's format from deepset/prompt-injections and Lakera/gandalf_ignore_instructions; it is not a run of Lakera's official PINT benchmark, and only a small subset of rules fire on it.
- Affected version
- 3.5.12
- Section
- §8 Evaluation — PINT-format corpus
- Published
- 2026-09-22
Adoption wording corrected. Merged pull requests in vendors' open-source repositories were described on this site as vendor production deployments. They are merged pull requests in public repositories — notably cisco-ai-defense/skill-scanner — and are not statements about any vendor's commercial product. Every implementer entry now states its evidence as the merged pull request it is.
- Affected version
- 3.5.12
- Section
- §7 Conformance — implementer report
- Published
- 2026-09-22
How to report an erratum
Reporting an erratum is part of open review, and anyone may do it. If you discover an error in this specification, open an issue on the ATR repository. The report should include: the affected version, the section (e.g. §3.5), the erroneous text as published, the proposed correction, and any verifiable references. Maintainers review reports in reasonable time; once confirmed, the entry is added to the registry above, with attribution. Errors are not silently overwritten — they leave a record.
- Open an issue — github.com/Agent-Threat-Rule/agent-threat-rules/issues/new
- Editor contact: [email protected]
An erratum corrects the text, not the normative content — it is not treated as a spec change. Substantive semantic changes ship as new versions, tagged on /changelog as [Breaking] or [Compatible]. The line is deliberate: anyone citing this specification can rely on errata never shifting a rule's meaning underneath them.
