Skip to content
Changelog

Every release.
On the record.

Generated from CHANGELOG.md at build time — this page is that file. Rule IDs are permanent once published; rule_version may bump, but the ID stays stable, safe for papers, CI scripts, and audit trails to cite. Version numbers track the npm release line (currently v3.5.0); the written specification ships at the same version with Working Draft status. Full commit history is on GitHub.

View full release history on GitHub →

vUnreleased

Added
  • ▸Every benchmark measurement records the enforce lane beside its headline. A measurement's metrics.recall is the hunt lane (the engine default, every maturity). Each runner the weekly measurement calls now also runs the same samples through its own detection loop on an engine constructed with lane: 'enforce' (only maturity: stable rules, the lane for blocking) and records the result as breakdown.enforce_lane. Where a measurement records it, README's Recall cell cites both lanes and root stats.json cites the enforce figure as enforceLaneRecall; scripts/check-benchmark-citations.ts fails CI when either disagrees with the measurement or leaves the enforce figure out. Before this, the enforce figures in README were one-off replays.
  • ▸Recall figures outside README are rendered from the measurement files. scripts/render-recall-citations.ts fills the figures between recall markers in LIMITATIONS.md and the docs/owasp-pr/ texts, from the file each source's latest.json points at, in both lanes (every rendering is held to the rule README's table is checked by). scripts/sync-citation-garak.ts generates the garak sentence in CITATION.cff from root stats.json, with both lanes and the closed-book caveat. The weekly measurement run re-renders and stages both and lists what it could not render in the rolling PR's body; CI (eval.yml) fails on a stale or unrenderable figure.
  • ▸`scripts/bench-evaluate-latency.ts` times single `ATREngine.evaluate()` calls in the hunt and enforce lanes on three committed corpora of different input length, so the latency wording in LIMITATIONS.md and THREAT-MODEL.md can be checked on any hardware. The latency_ms blocks in data/measurements/ time whole corpus samples (several event shapes, a skill scan and, when installed, an embedding pass) inside a full benchmark run, and are not comparable.
Changed
  • ▸Every recall figure the site, LIMITATIONS.md, CITATION.cff and the docs above take from the measurement files now names its lane and gives the enforce figure wherever the measurement records one. Figures no measurement file records are labelled instead: the about page's historical garak figure as taken with every rule loaded, the neutral-benchmarks post's runs as having no recorded lane. The website reads breakdown.enforce_lane from the measurement files (home, coverage, research, red-team, garak integration, ATD and sovereign-AI pages, the PINT blog post); it no longer reads recall from the old eval report files or their typed fallbacks, and no longer hydrates recall in the browser. The garak figures carry the closed-book caveat, and so now do the PINT-format and HackAPrompt figures, which have rules mined from them too. The SKILL.md benchmark is described as the internal, labelled corpus it is, with the harness's layer names. An erratum (e-2026-10-16-recall-lanes) records the change on the site.
  • ▸sync-stats.ts now fails when CITATION.cff has no garak sentence to rewrite, instead of leaving an old figure in place.
  • ▸README §8 discloses that the `garak` and `garak-full` figures are partly closed-book. A replay without the rules whose own YAML shows they were written from the in-the-wild corpus is given in the new [^garakcorpus] footnote, and the paragraph on the two garak rows no longer calls them independent: garak-full contains that corpus twice.
  • ▸data/stats.json no longer carries the orphaned benchmark.pintRecall block, a 3.5.11 figure nothing has written since #600 and nothing reads.
Fixed
  • ▸NIST-related statements corrected across the docs and the website. Pages described a NIST CAISI dialogue and position paper that did not happen, an RFI response ATR never filed, an OSCAL "Path 1" acceptance, and a quote NIST never published; they listed NIST among adopters and in a trust bar, and showed "100%" or "4/4 functions" for an AI RMF mapping that covers a subset of subcategories. usnistgov/oscal-content#338 is described as what it is: open, reworked against NIST's AI RMF catalog, awaiting NIST review. The website changelog page now shows the notes on historical entries, which it used to drop.
  • ▸Integration status, provenance, evasion and latency claims corrected. The red-team page listed seven proposals as under review that their ATR author had withdrawn unmerged (NVIDIA/garak#1676 on 2026-08-04; HarmBench, AgentDojo, JailbreakBench, Counterfit, PromptInject and Damn Vulnerable MCP Server on 2026-10-14 and 15, six of them never answered). They are removed, along with the lapsed "On deck" schedule, and the garak, sovereign-AI, responsible-use, integrate and STANDARDIZATION-STATUS surfaces no longer call the garak integration or any OWASP PR open. The garak page and the contributor docs (QUALITY-GATE.md, TSC-ONBOARDING.md, REVIEWER-GUIDE.md, RULE-PRODUCTION.md, the issue-to-proposal PR text) no longer describe auto-merge, which has been off since 2026-06-03. The red-team and governance pages no longer promise a metadata_provenance.discovered_by field no rule has, and the home, red-team, about and quality-standard pages no longer credit daily auto-crystallization for growth that came from CVE sweeps, benchmark-miss mining and contributors. The sovereign-AI page no longer says ATR detects intent and cannot be evaded; LIMITATIONS.md and the responsible-use, integrate and research pages no longer say every rule carries evasion tests or that there are 64; LIMITATIONS.md and THREAT-MODEL.md now agree that the optional judge and embedding modules ship but are off by default. "Sub-millisecond", "< 5ms", "5ms pattern match" and "Done in 47ms" are replaced by how evaluation cost scales; the OpenShell example is marked as a sketch of a proposal OpenShell closed, and its timeouts now exceed a one-shot cold start. CITATION.cff no longer calls the rules hand-authored and says what the safety gate checks for added and changed rules; the OWASP PR text drops its 76-rule count and notes the PR was withdrawn. Errata e-2026-10-16-integration-status, e-2026-10-16-evasion, e-2026-10-16-latency and e-2026-10-16-provenance record the site changes.

v4.1.5

2026-10-10
Changed
  • ▸The npm package ships the rules without their test samples. The release workflow removes test_cases and evasion_tests from each rule just before staging, after the tests have run on the full rules. The engine does not read them, so detection is unchanged: the same rules load and the same rules fire. The samples stay in the repository. With the package alone, atr test on a bundled rule has no samples to run, and atr stats reports zero TP/TN counts. DISCLOSURE says so.
Fixed
  • ▸ATR-2026-02600 no longer names a real public IP in its test samples. Its two reverse-shell samples now connect to the documentation ranges 203.0.113.14 and 192.0.2.4. The detection is unchanged.
  • ▸ATR-2026-02627 describes the web shells it detects instead of quoting them. Its description and one false-positive note quoted a working one-line PHP command shell; they now say what it is. The detection is unchanged.
Changed
  • ▸npm releases go only through the staged path in `publish.yml`. The package is dual-use, so npm's policy lets CI stage a version but not publish one; a maintainer promotes it with 2FA. Three workflows could still put a version on the registry another way and have been removed: publish-current-version.yml (direct npm publish through trusted publishing), publish-on-rules-merge.yml (bumped and ran npm publish with NPM_TOKEN on pushes to main) and npm-stage.yml (approved staged versions from CI with NPM_TOKEN, skipping the maintainer's 2FA). scripts/release.sh now stops at the tag push instead of also publishing locally. No workflow references NPM_TOKEN, and tests/npm-release-path.test.ts fails if any workflow other than publish.yml gains a publish, stage or approve step.

v4.1.4

2026-10-07
Notice
  • ▸Not published to npm (see 4.1.5).
Added
  • ▸Four rules. Each one passed the repository's PR checks, and is experimental, maturity test, with alert-level response actions only: - ATR-2026-02846: ignore-previous-and-following instructions (PromptInject goal hijacking), a semantic rule with a narrow regex fallback. - ATR-2026-02847: email exfiltration under a policy, audit or compliance pretext. - ATR-2026-02850: data exfiltration by email under a documentation pretext, including French phrasings. - ATR-2026-02851: a fake user turn inside content that instructs the agent to send an email. ATR-2026-02847, 02850 and 02851 come from human-written attack emails (LLMail-Inject). They fire on none of 526 real emails or 694 benign web pages presented as tool output, and each catches attack emails that were held out while the rule was written.
Fixed
  • ▸pyATR falls back for regex conditions Python's `re` cannot compile (such as variable-width lookbehind) instead of dropping the condition (#554, thanks @dc-tw).
  • ▸The test-case validator reads every key a test case can use. It read four of nine and misjudged rules whose cases use the others. Its failure count on the current rules drops from 40 to 11, with no new failures (#607).
Security
  • ▸`atr tc pull` checks the rules a Threat Cloud endpoint returns before writing them, and no longer goes through a shell. The response is now treated as untrusted: a rule is written only if its id has the form ATR-YYYY-NNNNN, its category is one of the schema's categories and no rule with that id is already in the repo; anything else is rejected without touching the disk. The validator then runs as a separate Node.js process with the file path as a plain argument. Before, the id and category came from the response unchecked and the validator was started through a shell, so a compromised or impersonated endpoint could affect the operator's machine beyond the rules directory. The category is also read from the rule's tags, not from the first subcategory: line in a compliance block, and the validator is found next to the installed CLI rather than in the current directory, so valid rules are no longer filed under the wrong directory or rejected when the command runs outside a checkout of this repo.
Changed
  • ▸The package is declared dual-use under the npm Dual-Use Content Policy. npm's publish-time malware scanning, introduced in July 2026, blocked 4.1.1, 4.1.2 and 4.1.3: rule files carry attack strings by design, as detection patterns and test samples, and the scanner treats them like malware. package.json now carries "contentPolicy": {"class": "dual-use"}, and a plain-text DISCLOSURE file at the package root says what the attack content is for and what the package does not do. Under that policy, CI may only stage a release; a maintainer approves it with 2FA before it goes live, so publish.yml now stages through trusted publishing and waits for the approval before creating the GitHub release.

v4.1.3

2026-10-06
Notice
  • ▸Not published to npm (see 4.1.4).
Changed
  • ▸The release workflow fails when the version never reaches the registry. It now asks npm for the exact version after publishing, waits up to ten minutes, and fails before the GitHub release is created. Before, the check read latest, ran after the release, and only warned, which is how both 4.1.1 and 4.1.2 went green without being published (#633).

v4.1.2

2026-10-01
Notice
  • ▸Not published to npm (see 4.1.3).
Changed
  • ▸Releases publish through npm trusted publishing (OIDC). publish.yml no longer reads a registry token. npm mints a short-lived credential for each run, bound to this repository and that workflow file, and generates the provenance attestation itself (#606).

v4.1.1

2026-09-22
Notice
  • ▸Not published to npm (see 4.1.2).
  • ▸Nine conditions still backtrack and are listed in [data/redos-baseline.json](data/redos-baseline.json) with measured times — ATR-2026-00285#0 and ATR-2026-00272#0 are the worst at 16s and 14s, and ATR-2026-00707#0 is maturity: stable, which means it runs in the enforce lane. The ReDoS gate is a ratchet against that file: a green gate means nothing got worse, not that the class is closed. Do not read this release as "ReDoS fixed".
Security
  • ▸`ATR-2026-00220` condition 2 backtracked catastrophically, the same defect as condition 0 of the same rule, which was fixed in #531 — a quantified run of base64 characters between two unbounded .* gaps, with a literal at the end. Only condition 0 was rewritten at the time. Measured on Node against a base64-dense single line with no matching tail: 0.5s at 1 KB, 3.4s at 2 KB, 23.4s at 4 KB. The gaps are now bounded and the run is fixed-length; the same inputs measure under a millisecond at 64 KB. All five of the rule's true positives still fire and all five true negatives stay clean, with condition 2 matching exactly the cases it matched before. Reported as GHSA-66x9-5vw9-3wv3 against condition 0, which 4.1.0 had already fixed; this closes the half of the rule that report did not cover.
  • ▸The package no longer ships a default endpoint for detection data. Reporting has been opt-in since 4.1.0, but opting in without naming a recipient silently picked one. --report-to-cloud now requires --tc-url or ATR_TC_URL and exits with an error naming the fix if neither is set. ATR is an open standard, not one collector's client: an operator who turns reporting on decides who receives it, and there is no address in the package to forget to change. Detection events now also carry an X-ATR-Client-Id header; the previous vendor-named header is still sent so existing collectors keep working.
Changed
  • ▸CHANGELOG.md is now included in the npm tarball. The files allowlist had excluded it, so a release that fixed a security defect arrived at consumers with nothing in the package to say so.
Fixed — external claims that overstated what had happened
  • ▸The npm package `description` claimed a vendor product shipped the ATR rule pack. What actually happened is that rules were merged into cisco-ai-defense/skill-scanner, an open-source scanner repository — not the same thing as shipping inside that vendor's product. The blurb is the single most-copied sentence in the project, so it now says what the package is and makes no adoption claim at all (#580).
  • ▸Withdrew the two per-lane false-positive rates published alongside the enforce/hunt split. Three defects, any one of which is disqualifying: both were measured on a subsample roughly a sixth of the corpus they were credited to; the benign corpus contained real jailbreak samples — material the rules are meant to catch, sitting in the set that defines a false positive — until an exclusion filter landed on 2026-08-04 (#373); and the ladder that produced the hunt figure depended on a file no longer in the repository, so that figure cannot be reproduced at all. A gap is more useful to an adopter than a number nobody can reproduce. Re-measurement against the current benign corpus is pending, and neither figure should be cited by anyone until it lands (#579).
  • ▸Corrected the fiscal sponsor's legal identity. Nine files carried an EIN that corresponds to no organization, and five of them additionally described the sponsor as a 501(c)(3) when it is a 501(c)(6). The distinction is substantive rather than cosmetic: funders that restrict grants to 501(c)(3) recipients would have been misled (#578).

v4.1.0

2026-09-14
Notice
  • ▸Never published to npm. The tag v4.1.0 exists, but the publish failed on an expired registry token and was not retried for 30 days. This version number does not exist on the registry and never will; everything below shipped in 4.1.1 instead.
Added
  • ▸A benign-twin corpus: 1,788 benign counterparts to attack payloads, across 17 files, counted at the v4.1.0 tag. Built so that a rule which fires on the harmless sibling of the attack it targets can be caught before it ships. It demoted one production rule on arrival.
  • ▸Forty rules from the full proposal re-sweep, landed as four batches of ten, plus fifteen from the CVE proposal backlog (#503, #504).
  • ▸A precompiled rule digest for consumers that cannot take an engine (#510), with a Python-verified variant, and a PyRIT digest that carries every field and names the scope it is safe to use (#511).
  • ▸pyatr 0.3.0: bundle refreshed, and the behaviour that had blocked the refresh unpinned (#507).
Fixed — catastrophic backtracking
  • ▸Eight rules could be made to backtrack catastrophically; the patterns are rewritten and a gate now blocks the class from returning (#531). ATR-2026-02610 condition 0 was fixed separately for the same defect.
  • ▸`ATR-2026-01005`, the many-shot rule, is now linear without giving up RE2 portability — the obvious fix would have traded one for the other (#546).
Fixed
  • ▸Ten rules from the re-sweep were withheld rather than shipped: each fired on its own benign twin from the same run, so the twin corpus caught them before release (#565). ATR-2026-02604 was withheld earlier for the same reason.
  • ▸`\u{...}` is a JavaScript-only escape, so ten rules never compiled outside the TypeScript engine — they were silently absent for every other consumer (#509).
  • ▸The Sigma exporter did not preserve ATR's regex case semantics (#520).
  • ▸The AVID importer stopped at the API's page limit instead of enumerating all reports (#534).
  • ▸The quality gate now checks peer true negatives within the same PR (#529), and the website shows the effective rule count rather than the raw file count.
Removed
  • ▸The one-shot bootstrap publish workflow, once it had served its purpose (#541).

v4.0.0

2026-08-22
Notice
  • ▸Releases 3.5.1 through 3.5.12 were automated rule-publish releases cut from this same stretch of history and were never given their own entries. A few items below — the benchmark-number corrections in particular — were first published in one of them rather than in 4.0.0.
Changed — blocking is now opt-in (BREAKING for anyone relying on the old default)
  • ▸`atr guard` no longer emits a `permissionDecision` unless blocking is enabled. The Claude Code PreToolUse payload now carries the detection (atr_decision, atr_reason, matched_rules, atr_advisory: true) and drops the hookSpecificOutput envelope entirely, so the host applies its own permission flow. toClaudeCodePostToolUse likewise omits decision: 'block' (its envelope is kept: a permissive verdict has always emitted it with no decision inside). The whole envelope goes rather than just the decision field as a legibility choice, not a compatibility one. Both shapes are in fact accepted: the hook-output schema in the shipped Claude Code 2.1.76 bundle declares hookSpecificOutput optional, its PreToolUse member declares permissionDecision optional, and the object is not strict, so unknown top-level keys are stripped rather than rejected — which is also why atr_decision and matched_rules have shipped alongside it all along. A payload with no envelope simply cannot be misread as a decision that failed to serialise.
  • ▸ATR never emits `permissionDecision: "allow"` — in either mode. This is wider than the blocking switch and is the part most likely to surprise: turning blocking ON does not bring the affirmative decision back. In the PreToolUse contract allow is not neutral; it is an approval that suppresses the host's own permission prompt, so a hooked session would permit *more* than an unhooked one on every operation ATR simply had not looked for. "No rule matched" is ATR having nothing to say, and the way to say nothing on that channel is to omit the field. A decision is therefore emitted only for the two verdicts that restrain — deny and ask — and only when blocking is on. Everything else travels in atr_decision / atr_reason / matched_rules. Consequence for the payload shape: with blocking ON and a permissive verdict the hookSpecificOutput envelope is dropped too, exactly as in advisory mode. atr_advisory: true marks the *mode*, not the verdict, so it is the one key that distinguishes the two:
  • ▸BREAKING for embedders: the library no longer reads the environment at all. ATR_LANE and ATR_BLOCKING are CLI-only. ATREngine, ActionExecutor and HookHandler take their posture from explicit config or the built-in default, and nothing else. If you embedded ATR and were relying on a variable in the ambient environment to select a lane, that no longer works — pass new ATREngine({ lane }) instead.
  • ▸Scope limit, stated so the headline is not read wider than it is. This covers the two channels the engine itself drives: the Claude Code hook contract and ActionExecutor. It does NOT cover the framework adapters, which keep their own severity floor: src/adapters/mastra.ts defaults blockSeverities to ["critical", "high"], and the openshell-filter and nemoclaw-preflight CLI entry points default ATR_MIN_SEVERITY to high. (The NemoClawPreflight and OpenShellFilter classes take no default at all — an embedder must pass minSeverity. Only the CLI wrappers supply one.)
  • ▸An unrecognised `ATR_LANE` or `ATR_BLOCKING` warns on stderr, falls back to the safe default, and the guard keeps running (exit 0). main did not read either variable, so a typo was previously inert. An earlier revision of this branch threw from the constructor instead, which measured badly: ATR_BLOCKING=enabled npx atr guard exited 1 with an empty stdout and no guard running — landing hardest on the operator trying to turn enforcement ON. atr init installs atr guard as a PreToolUse command hook, and Claude Code 2.1.76 maps any exit status other than 0 or 2 to a non-blocking error: it runs the tool anyway, hands the model nothing, and renders only <hookName> hook error without the captured stderr. Exiting would have discarded every detection and still not told the operator why; status 2, the only loud one, blocks the tool outright, which a stray shell variable must never do. The warnings are printed once, at startup, by the CLI — they are not deduplicated per value, and the resolveLaneOrWarn / resolveBlockingOrWarn / resetEnforcementWarnings helpers that did that are gone. Verbatim, prefixed [atr] (red on a TTY):
  • ▸An unreadable `ATR_LANE` forces blocking off, even against an explicit `--blocking`. Falling back to hunt while blocking stays on is the one degraded posture that is *more* dangerous than the one requested: the operator asked to enforce on maturity: stable rules only and would instead enforce on every maturity. --blocking says "you may block", not "block on a lane I never chose". ATR_LANE=enfroce atr guard --blocking therefore exits 0 in advisory mode, having printed the lane warning above plus:
  • ▸Both switches now trim whitespace and ignore case, identically. They were asymmetric: ATR_BLOCKING=ON worked while ATR_LANE=ENFORCE did not, so the pair ATR_LANE=ENFORCE ATR_BLOCKING=ON turned blocking on and silently widened the lane to hunt — the operator asked for the narrowest enforcement posture and got the broadest one. ATR_LANE=ENFORCE, ATR_LANE=" alert " and --lane Alert are all honoured now.
  • ▸A non-boolean explicit `blocking` throws instead of enabling blocking. new ActionExecutor({ adapter, blocking: "false" }) and new HookHandler({ engine, executor, blocking: "false" }) previously turned blocking on, because every non-empty string is truthy — reading, to their author, as an explicit "off". Both now raise a TypeError, whose message in full is:
  • ▸`ActionExecutor` no longer dispatches response actions above the `observe` blast-radius tier unless blocking is enabled. alert / snapshot / shadow / escalate run exactly as before; block_input / block_output / block_tool / reduce_permissions / reset_context / quarantine_session / kill_agent are recorded as suppressed and the adapter is never called. The tier ladder is read from src/quality/action-eligibility.ts — this change does not introduce a second classification of what is destructive. This closes the dual-channel contradiction where a benign Bash{command:"ls -la"} produced permissionDecision: "allow" while the executor really invoked blockTool on the adapter.
  • ▸Both channels are governed by one switch: the blocking config field on ActionExecutor and HookHandler for embedders, and --blocking / --no-blocking or ATR_BLOCKING on atr guard for the CLI. Default off. The environment variable reaches the CLI only — see the embedder entry above.
  • ▸Why: SPEC.md §5.5 (Response) is the engine-wide requirement, quoted whole because it is short enough to be: "Engines MUST NOT execute response actions automatically without an explicit configuration directive from the operator. The response field is a recommendation expressed by the Rule author, not a directive to the Engine." That directive had no implementation — no CLI flag, no environment variable, no documented config key by which an operator could express it. This change is that implementation. Two narrower statements point the same way. Neither is the requirement, and neither may be quoted as a general rule: - spec/atr-method-v1.1.md §5.6 (Provenance and Trust) sits under §5 Signature Method and is scoped to hash matches: "Engines SHOULD NOT auto-block on a hash match without operator policy explicitly enabling it; the default response action SHOULD be log_alert until provenance is operator-trusted." An earlier revision of this entry quoted that sentence with "on a hash match" elided, which turned a Signature-Rule SHOULD NOT into an engine-wide one. It is not one. Quote it whole or cite §5.5 instead. - docs/QUALITY-STANDARD.md ("For Consumers") restricts blocking to maturity: stable with confidence ≥ 80. That is deployment guidance addressed to consumers, not a normative requirement on engines.
  • ▸Turning blocking on reproduces the previous behaviour on every verdict that restrains — and only those. An earlier revision of this entry claimed the two transcripts were "byte-identical". They are not, and the difference is the point of the change. Measured by replaying the 850 samples of data/pint-benchmark/pint-corpus.json as 850 hook events per hook type (one JSON line in, one JSON line out) through atr guard's stdio loop — HookHandler → evaluateWithVerdict → ActionExecutor — on the pre-change merge-base 994b01b2b with no flags, and on b9da8d710 with --blocking. The rule corpus is byte-for-byte identical at those two commits (git diff --name-only 994b01b2b..b9da8d710 -- rules is empty), so every difference below is the contract change and nothing else.
  • ▸Detection is unchanged in both modes. Over the same 850 × 2 events, matched_rules is identical to the baseline on 850/850 for both hooks in both modes, and atr_decision equals the baseline's internal decision on 850/850. Blocking changes what the engine *does*, never what it *sees*.
Added — the detection lane finally has an entrance
  • ▸ATREngineConfig.lane existed and worked but no shipped code path ever set it and no user-facing entry point could, so hunt was the only reachable setting. Added --lane <enforce|alert|hunt> to atr guard and atr scan, the ATR_LANE environment variable, and a lane input on the GitHub Action. ATR_LANE is read by the CLI surfaces only — atr guard, atr scan, and src/mcp-server.ts when it is the process entry point. It is *not* honoured by every ATREngine: an engine constructed by an embedder ignores it entirely. See the library/environment entry under "Changed" above.
  • ▸There is no single resolution order any more, and conflating the two was the bug. They are different chains on different surfaces:
  • ▸atr guard now prints its posture on stderr, preceded by [atr-guard] Loaded <n> rules from <dir>. Both modes carry a parenthetical, and the blocking one names the limit rather than claiming enforcement outright:
  • ▸Provenance of the console output quoted in this entry. Every payload, warning and posture line above is captured stdout/stderr from fix/review-never-affirmative-allow at b9da8d710, pasted unedited apart from stripping the ANSI colour codes on the [atr] warnings and rewriting the absolute rules path. This branch carries the documentation; b9da8d710 carries the implementation. The commands reproduce these outputs once both have landed — not before.
Fixed
  • ▸src/hook-handler.ts carried two contradictory comments about failure behaviour: the module header said fail-open, an inline comment in startStdioLoop said the error path "fail-closes to a deny". The header was right — failOpen defaults to true in both the constructor and the CLI. The inline comment is corrected; the behaviour is unchanged.
Fixed — published benchmark numbers
  • ▸Withdrew two garak figures that no measurement file backed. From 2026-08-04 to 2026-08-05, README.md and stats.json cited 91.5% (garak in-the-wild) and 56.9% (garak-full) at ATR 3.5.11. Neither run produced a file under data/measurements/; the newest files there were 95.7% at 3.5.8 and 38.3% at 3.5.0 respectively. Re-measured on 2026-08-05 at 780 rules: 92.5% (garak, 601/650) and 57.2% (garak-full, 1989/3475), both written to data/measurements/ with the commit that produced them.
  • ▸Four eval harnesses built `type: 'llm_io'`, which is a rule *source* and not an AgentEventType. src/engine.ts could not map it to a source, and its source-type filter is skipped when the mapping is absent — so instead of the narrow llm_io channel those harnesses documented, every rule of every source ran against the event. Affected: eval-garak-inthewild.ts, eval-academic-raw.ts, eval-small-corpora.ts, atr_recall_analysis.ts. check-new-rules-on-benign.ts had the same class of bug with type: 'user_input' behind an as AgentEvent cast. Event shapes now come from scripts/lib/corpus-event.ts.
  • ▸`scripts/` is now typechecked (tsconfig.scripts.json, npm run typecheck:scripts, wired into CI). tsconfig.json covered only src/**, which is why every one of the above was an unreported TS2322/TS2345.
  • ▸`.github/workflows/rule-quality.yml`'s "Run PINT eval regression check" ran npm run eval (the internal self-test harness, not PINT), read a git-committed report that step does not write, and looked up a key that does not exist on it — printing PINT recall: undefined while no downstream step read the result. It now runs npm run eval:pint and gates on recall/FP against the committed baseline.
  • ▸`scripts/check-benchmark-citations.ts` (new, runs in CI) fails the build when README.md's benchmark table or root stats.json cites a number, ATR version, or date that data/measurements/ does not back.
Added — beyond the hook and lane work above
  • ▸The Cisco Skill Scanner pack is now a build artifact rather than a hand-assembled export (#495).
  • ▸A rule the benign corpus cannot see has not been measured — CI now says so instead of counting it clean (#460). The benign gate was also given the two corpora it had been missing (#468).
  • ▸DETECTION-BOUNDARY.md — what this layer detects and what it cannot (#461) — and ENFORCEMENT-MODEL.md, which also made three contradicted statements elsewhere in the docs true (#470).
  • ▸ATR-2026-02502: covert remote-script injection into generated artifacts (#462). The four tool-poisoning rules are mapped to MITRE ATLAS AML.T0110.000 (#465), and the CSA MAESTRO mapping is refreshed against the current corpus (#464).
Fixed — the guard was not reading real events
  • ▸`atr guard` never read a real Claude Code event. The shape it parsed was not the shape the host sends (#483). The event shape now follows hook_event_name as well as the dispatch, rather than the dispatch alone (#487).
Fixed — rules that matched the wrong thing
  • ▸ATR-2026-00086 was detecting Russian, not spoofing (#489).
  • ▸ATR-2026-01904 read method calls as domains (#474).
  • ▸ATR-2026-00443 read an ordinary sentence as fragment assembly (#473).
  • ▸ATR-2026-01750 fired on descriptions of loops rather than demands for output (#472).
  • ▸An unquoted ATLAS T0110.000 title broke the site build (#477).
Measurement
  • ▸The benign false-positive evidence was re-derived at 785 rules (#497).
  • ▸ATR was measured against 36,394 published ClawHub skills across three event shapes (#490), and rule drift was measured against the 2026-04 wild scan (#491).
  • ▸The rule count in the standardization document was 357 rules out of date (#479), and five benchmark corpora were refreshed (#478).

v3.5.0

2026-06-16
Notice
  • ▸The two lane false-positive rates quoted in this 3.5.0 entry were withdrawn on 2026-09-19 and must not be cited. The entry is left as published, because a changelog is a record of what was said at the time. See the withdrawal under [Unreleased] for the three defects that disqualified them; re-measurement is pending.
Added
  • ▸Detection lanes (`enforce` / `alert` / `hunt`). Rules now carry a maturity-driven lane so consumers can trade recall for precision. On a 65K-sample benign gate: enforce (stable + confirm rules only) ~0.24% FP; hunt (all rules) ~9% FP. Default is hunt, opt-in and fully backward-compatible.
  • ▸`confirm` field — an embedding/secondary check that must pass before a match fires in the enforce lane. Four rules ship a confirm: embedding guard (ATR-2026-00001 / 00002 / 00030 / 00442).
  • ▸`src/quality/rule-contract.ts` — a single shared contract (MATURITIES, laneAllows, validateContract) the engine and loader consume.
Changed
  • ▸652 detection rules across 10 categories. Specification line 3.0.0-alpha.1 (Working Draft); this is the npm release line, tracked separately from the spec version.
  • ▸23 rules reclassified (maturity) to reflect measured precision and generalization.
  • ▸ATR-2026-00495 deprecated (garak DAN "insert prompt here" variant) — it now never fires. Spec-conformant; the only behavior change in this release.
  • ▸Python pyatr engine skips deprecated / draft rules, matching the TypeScript engine.
Note
  • ▸Report false-positive rates lane-keyed (enforce ~0.24% / hunt ~9% on a 65K-sample benign gate), not as a single overall figure.

v3.1.1

2026-06-05
Fixed
  • ▸ATR-2026-00001 (prompt-injection) false positive: the start over/fresh/anew with a new … task-switch branch matched benign phrasing such as "start fresh with a new outline" / "a new draft" because it lacked the task-noun constraint its sibling branches carry. Tightened to require a task-like noun (task / instruction / assignment / objective / goal / mission / prompt / persona / role / directive / job) after "new". The rule's documented true-positives are unaffected; full suite green (535 tests). Surfaced by the new semantic-validation corpus.
Added
  • ▸Semantic-validation harness (data/semantic-validation/ + scripts/semantic-validation-score.mts): a focused, reproducible corpus — 20 held-out paraphrased attacks targeting the two v3.1.0 semantic rules (00573 instruction-override, 00574 system-prompt extraction) plus 15 adversarial benign near-misses — and a scorer. Documents what the optional semantic stage actually adds on its two target threats, and ships the corpus so anyone can re-run it with their own judge.
Measured
  • ▸Focused semantic validation (n=35, Claude-as-judge via worklist mode, threshold 0.7): the deterministic regex layer caught 10% of the paraphrased attacks; the optional semantic judge stage raised combined recall to 95% (19/20), with 0 judge false positives on the 15 adversarial benign near-misses (0 regex FP after the 00001 fix above). This is a small-n, authored, self-judged validation of the pipeline — NOT an independent benchmark. See data/semantic-validation/README.md for method and caveats.

v3.1.0

2026-06-05
Added
  • ▸Tier-2 semantic detection (T2) — optional second-stage judge. New two-stage scan pipeline (scripts/scan-with-judge.mts). Stage 1 is the existing deterministic regex engine run across the whole corpus; Stage 2 spends an LLM-as-judge only on the subset Stage 1 flags or misses (~2%), so judge cost scales with the flagged/missed set rather than the corpus. Opt-in by design: it runs only with an OpenAI-compatible backend (ATR_SEMANTIC_API_KEY [+ ATR_SEMANTIC_BASE_URL / ATR_SEMANTIC_MODEL]) or a no-key worklist mode (WORKLIST_OUT) that a separate session adjudicates and feeds back via --verdicts. Tunable JUDGE_SUBSET (missed | flagged | all) and JUDGE_THRESHOLD. The default ATREngine.evaluate() path stays 100% deterministic and the LLM judge is off by default — the two semantic rules contribute only narrow regex fallback patterns to the default path (FP-gated, with documented true-negatives); the judge stage is additive and opt-in.
  • ▸ATR-2026-00573 (prompt-injection): semantic paraphrased instruction-override. Flags reworded injection that drops the literal trigger vocabulary (ignore / disregard / forget) that pattern rule ATR-2026-00001 keys on — the precise misses recorded in 00001's evasion_tests. Severity high, status experimental.
  • ▸ATR-2026-00574 (context-exfiltration): semantic paraphrased system-prompt extraction. Flags indirect elicitation of the agent's system prompt or hidden configuration without the literal verb-first phrasing (reveal / show / print your instructions). Severity high, status experimental.
  • ▸Cisco-grade, human-reviewed exploitation rules for agent runtimes: path traversal, SQL injection, and reflected XSS (ATR-2026-00569 / 00570 / 00571), MCP command injection, and SSRF-to-cloud-metadata, plus CVE-anchored detections surfaced by the daily NVD collector.
  • ▸Daily CVE collector and ecosystem-scan automation now feed agent-package vulnerability proposals into the review queue (collection only; promotion stays behind the human safety gate).
Changed
  • ▸Total rule count: 450 → 462.
  • ▸Two rounds of false-positive hardening on existing rules, gated by a wild-confirmed benign corpus and a dual-use security-skill bucket, so security tooling that legitimately ships attack strings no longer trips detection.
  • ▸Engine and rule format unchanged; existing ecosystem integrations (Microsoft AGT, Cisco AI Defense, MISP CIRCL, OWASP A-S-R-H, precize, Sage) work without modification.
Deprecated
  • ▸ATR-2026-00235 — exact duplicate of ATR-2026-00230; marked deprecated, retained for ID stability.
Notes
  • ▸First npm release to carry the post-3.0.5 work. The Cisco-grade rules and hardening landed on main between 2026-05-29 and 2026-06-04 but stayed unpublished until now: the auto-publish flywheel fires only for Threat-Cloud-crystallized commits, so human-authored rule work ships on an explicit minor release like this one.
  • ▸Semantic rules are experimental and the T2 judge is opt-in; production blocking should continue to rely on the pattern rules. Benchmarks remain measured on the 3.0.0 baseline (see stats.json) and have not been re-run against the semantic stage.

v2.2.2

2026-05-13
Added
  • ▸ATR-2026-00523 (skill-compromise): Claude Code Hooks SessionStart pre-trust RCE (CVE-2025-59536). Detects repo-shipped .claude/settings.json registering a SessionStart hook with the startup matcher; the hook command fires before the trust dialog renders. Six layered conditions: literal Hooks-schema shape anchor, shell-interpreter command resolution, pipe-to-shell / inline-exec payload shape, package-manager + credential-tool invocation, repo-path + payload co-occurrence, and pre-trust prose anchor. 7 TP / 6 TN / 3 evasion tests documented. Reported by Aviv Donenfeld and Oded Vanunu (Check Point Research); patched via GHSA-ph6w-f82w-28w6 enhanced trust-dialog warning.
  • ▸ATR-2026-00524 (context-exfiltration): Claude Code ANTHROPIC_BASE_URL credential exfiltration (CVE-2026-21852, CVSS 5.3 Moderate). Detects repo-shipped .claude/settings.json or env-var configuration rebinding ANTHROPIC_BASE_URL to an endpoint outside the Anthropic-controlled allowlist (api.anthropic.com, Vertex *.googleapis.com, Bedrock bedrock*.<region>.amazonaws.com); pre-trust API request leaks the live Authorization: Bearer header. Seven layered conditions: JSON-form allowlist negative-lookahead, shell / dotenv / Dockerfile env-var form, bare public-IPv4 detector (excludes loopback + RFC1918), cleartext-HTTP detector against non-loopback hosts, repo-path + payload co-occurrence, pre-trust prose anchor, and CVE-identifier co-occurrence. 7 TP / 8 TN / 3 evasion tests documented. Patched in Claude Code >= 2.0.65 (GHSA-jh7p-qr78-84p7); PoC at github.com/atiilla/CVE-2026-21852-PoC.
Changed
  • ▸Total rule count: 419 → 421.
Anchors
  • ▸2 Check Point Research disclosures against Claude Code: CVE-2025-59536 (Hooks SessionStart pre-trust RCE) + CVE-2026-21852 (ANTHROPIC_BASE_URL pre-trust credential exfil). Both share the same root cause class — repo-scoped .claude/settings.json parsed and acted on before the trust dialog gates the action — and both upstream patches narrow the trust-gate ordering. ATR rules anchor on the dangerous config shape so detection survives both pre- and post-patch deployments.

v2.2.0

2026-05-12
Added
  • ▸75 new rules across 5 categories (prompt-injection 57 + context-exfiltration 6 + tool-poisoning 4 + excessive-autonomy 2 + model-abuse 2). Total rule count: 344 → 419.
  • ▸5 new rule source integrations: HackAPrompt (EMNLP 2023, 4,780 samples, ATR-2026-00452..00456), NeMo-Guardrails + llm-guard + Promptfoo vendor test suites (94 samples combined, ATR-2026-00500..00505), PromptInject (NeurIPS 2022, ATR-2026-00506..00509), OWASP LLM Top 10 + MITRE ATLAS PoCs (8 standards-aligned rules ATR-2026-00510..00517).
  • ▸6-framework compliance metadata on all 75 new rules (OWASP / MITRE ATLAS / NIST AI RMF / EU AI Act / ISO 42001 / SAFE-MCP).
  • ▸53 rules with regex generalized from literal corpus fingerprints to multi-layer structural patterns.
  • ▸4 rules kept as KEPT-AS-IS corpus fingerprints (ATR-GARAK-a7fcb4e5 + 3 others): cannot generalize without unacceptable FP rate; marked experimental, not for production blocking.
Changed
  • ▸Total rule count: 344 → 419 (357 stable + 62 experimental).
  • ▸NVIDIA garak wrapped rule count: 293 → 419.
Metrics
  • ▸HackAPrompt (4,780 samples): recall 28.6% → 66.2% (+37.6pp), 100% precision maintained.
  • ▸garak (3,475 prompts): ATR-core families ~80%+; per-family breakdown: latentinjection 34.4%, sysprompt_extraction 67.9%, dan 90.2%.
  • ▸PINT (850 samples): 0.25% FP maintained, 0 FP regression.
  • ▸SKILL.md (341 samples): 100% precision, 0% FP, 0 FP regression.
  • ▸0 FP regression on benign corpus (432 real-world benign skills).
Scope
  • ▸PyRIT Pliny L1B3RT4S: refused — Anthropic usage policy prevented subagent import.
  • ▸AdvBench, HarmBench, JailbreakBench: reclassified to data/test-corpora/; describe target behaviors, not attack payloads.

v2.1.4

2026-05-12
Added
  • ▸ATR-2026-00448 (tool-poisoning): Spring AI MilvusVectorStore filter-expression injection (CVE-2026-41705). Tautology-based deletion-broadening, paren-breakout, in-clause chaining, terminator-DROP, like-ESCAPE bypass primitive. 8 TP / 6 TN / 0 FP on 431-sample benign corpus.
  • ▸ATR-2026-00449 (context-exfiltration): Spring AI PromptChatMemoryAdvisor cross-user memory leakage (CVE-2026-41712). Shared / singleton ChatMemory wiring, placeholder conversation_id, mismatched user-tag windows, wildcard-tenant scope. 8 TP / 6 TN / 0 FP.
  • ▸ATR-2026-00450 (data-poisoning): Spring AI PromptChatMemoryAdvisor memory poisoning (CVE-2026-41713). Persistence-aware ignore-previous, bracketed [SYSTEM-MEMORY-PERSIST] canaries, smuggled role-tag prefixes, REMEMBER:-shaped persona override, ChatMemory.add(SystemMessage) sink-level concat. 8 TP / 6 TN / 0 FP.
  • ▸ATR-2026-00451 (privilege-escalation): LiteLLM proxy admin endpoint SQL injection (CVE-2026-42208, CISA KEV 2026-05-08, CVSS 9.3, federal remediation due 2026-05-11). Tautology + comment-out, stacked DROP / TRUNCATE, UNION SELECT exfil, pg_sleep blind, Postgres metadata recon, error-based extractvalue / updatexml / load_extension. 8 TP / 6 TN / 0 FP.
Changed
  • ▸Total rule count: 344 → 348.
Anchors
  • ▸3 Spring AI CVEs (CVE-2026-41705 / -41712 / -41713) disclosed 2026-05-08 in Spring Security Advisory; patches in Spring AI >= 1.0.0.
  • ▸1 CISA Known Exploited Vulnerabilities catalog entry: CVE-2026-42208 added 2026-05-08, federal remediation due 2026-05-11, active exploitation observed against financial services and healthcare deployments; patch in LiteLLM >= 1.48.3.

v2.1.3

2026-05-11
Added
  • ▸6 HackAPrompt-cluster rules (ATR-2026-00442..00447) + benchmark harness (PR #51). Auto-published via tc-pr-back release workflow after PR #51 merge.
Changed
  • ▸Total rule count: 338 → 344.

v2.1.2

2026-05-11
Added
  • ▸ATR-2026-00440 (agent-manipulation): Microsoft Semantic Kernel CVE-2026-26030 — In-Memory Vector Store lambda+eval RCE. AST-traversal-via-mro primitives, BuiltinImporter reflective access, Function-constructor variants. 8 TP / 5 TN / 0 FP on 466-sample benign corpus.
  • ▸ATR-2026-00441 (privilege-escalation): Microsoft Semantic Kernel CVE-2026-25592 — SessionsPythonPlugin arbitrary file write to autostart paths. Windows Start Menu Startup, XDG autostart, systemd-user, cron, macOS LaunchAgents/Daemons, Windows Registry Run-key persistence. 7 TP / 5 TN / 0 FP.
  • ▸`src/redact.ts` new module: redactMatchedValue() + redactMatchedValues() helpers. ATR-consuming integrations can run ATRMatch.matchedPatterns entries through this before logging. Recognises AWS keys, GitHub tokens, Slack tokens, OpenAI/Anthropic keys, Bearer creds, JWTs, PEM keys. 13 unit tests. Additive opt-in API — no existing API changes.
Changed
  • ▸Total rule count: 336 → 338.
Loop closure
  • ▸Microsoft Copilot SWE Agent opened microsoft/agent-governance-toolkit#1981 2026-05-11 06:07 UTC with regression-test fixtures presuming ATR detection. PR #50 merged + v2.1.2 npm + GitHub release published within 2h 16m. Closing-the-loop comment mapped each fixture to ATR rule IDs.

v2.1.1

2026-05-10
Added
  • ▸6 ATR rules covering 7 gap CVEs identified by Phase 1 audit (CVSS 9.1–10.0). PR #46 + follow-up batches.
  • ▸STIX 2.1 extension for x-atr-rule custom SDO (PR #49) — self-published extension definition; ATR rules can now be expressed as STIX SDOs for STIX-compliant consumers.
Adopted (external)
  • ▸MISP/misp-taxonomies#323 merged 2026-05-10 by adulau (MISP project lead). 10 ATR predicates + 330 rule IDs as MISP machine tags.
  • ▸MISP/misp-galaxy#1207 merged 2026-05-10 by adulau. 336 cluster values with kill-chain, severity, CVE / OWASP LLM / MITRE ATLAS cross-refs. 10,408 lines.

v2.1.0

2026-05-09
Notice
  • ▸Historical entry — the AI RMF line below overstated coverage. "100% coverage of all 4 AI RMF functions" described the community OSCAL catalog, which encodes all 72 AI RMF subcategories as controls. ATR's rules map to a subset of those subcategories. The catalog is a community contribution and is not endorsed by NIST. Do not quote the line as a coverage claim. (Note added 2026-10-16.)
Added
  • ▸NIST AI RMF mapping v0.2 — 100% coverage of all 4 AI RMF functions (Govern / Map / Measure / Manage), 72 controls + 31 cross-reference links + worked example profile. Schema-valid OSCAL catalog published under Agent-Threat-Rule/ai-rmf-oscal-catalog (PR #46).
  • ▸Dedicated /compliance/nist-ai-rmf page on website with 314 → 330 rule count refresh sitewide.
Adopted (external)
  • ▸OWASP/Agent-Security-Regression-Harness#74 merged 2026-05-11 by mertsatilmaz (OWASP Project Lead): "Welcome to the team."

v2.0.11

2026-04-21
Notice
  • ▸Historical entry — do not quote the garak or SKILL.md figures below. The 97.1% in-the-wild recall was measured on a garak corpus that many of this release's new rules were written from, so it is not an out-of-sample result. The SKILL.md precision and FP figures are in-sample: the benign set is the one the rules were tuned against. Current figures, reported per lane and with their closed-book share, are in README §8 Evaluation. (Note added 2026-10-16.)
Added
  • ▸193 new rules covering the full NVIDIA garak probe corpus (ATR-00300 ~ ATR-00414), bringing total to 311 rules - agent-manipulation: DanInTheWild community jailbreak library batch 7-9, AntiDAN inverted persona, ChatGPT DevMode RANTI, Image-Markdown unlock, Ablation_Dan_11_0 generator-name binding, AutoDAN hierarchical suffix, Goodside ThreatJSON death coercion, Doctor XML policy puppetry - prompt-injection: latentinjection separator/keyword, leakreplay cloze MASK, glitch token repeat-back (SolidGoldMagikarp et al.), echo SQL-jinja template injection, DRA parenthesis reconstruction, phrasing past/future tense, LMRC harm category, FigStep image-list multimodal, continuation ethnic slur, backspace deletion character, LLM special-token boundary, toxic continuation prefix, snowball impossible reasoning, ecoji/base2048/variant-selector/sneaky-bits encoding injection, zalgo/braille/sata encoded attacks - context-exfiltration: markdown image URL exfil + XSS, API key generation and partial-key completion - model-abuse: malwaregen code generation (TopLevel + SubFunctions), HuggingFace unsafe model artifact load - tool-poisoning: ANSI escape elicitation request
  • ▸Word-boundary (\b) anchoring on grandma-roleplay rule to prevent partial-word FP (late/gran/read)
  • ▸metadata_provenance.garak_probe field for ATR-00267 (GCGCached), ATR-00301 (TAPCached), and all new rules
  • ▸Missing true_negatives added to 10 existing DAN-family rules (ATR-00352–00376) to meet ≥5 TN safety gate requirement
Changed
  • ▸Total rule count: 113 → 311
  • ▸garak in-the-wild benchmark recall: 69.7% → 97.1% (646/666 prompts detected against garak community jailbreak corpus)
  • ▸SKILL.md precision/FP: maintained 100% precision, 0% FP on 498-sample benign corpus
  • ▸FP mitigations applied to new rules (response action [alert] only for medium-FP-risk rules — phrasing, API key, malwaregen, SQL-jinja)
Fixed
  • ▸ATR-00402 (grandma roleplay): added word boundaries to prevent late/gran/read substring FP in Firebase documentation
  • ▸ATR-00401 ([MASK]): required elicitation context to prevent NLP-documentation FP
  • ▸ATR-00403 (glitch tokens): required elicitation verb before token names (StreamerBot etc.) to prevent legitimate software reference FP
  • ▸ATR-00411 (API key gen): negative lookahead on "from your dashboard/console" to exclude documentation requests

v2.0.10

2026-04-21
Added
  • ▸garak coverage batch 10: ansiescape, badchars, realtoxicityprompts, topic, fileformats (ATR-00393~00398)

v2.0.9

2026-04-20
Added
  • ▸garak DanInTheWild coverage batch 8-9: jailbreak templates, emoji-flag, prompt-browser (ATR-00377~00392)

v2.0.0

2026-04-15
Notice
  • ▸Historical entry — do not quote these figures as current. The scan counts, malware counts, benchmark rates and rule totals below are as published on 2026-04-15 at 113 rules. The wild-scan figures in particular are under reconciliation and are not a citable measurement; current corpus numbers live in data/stats.json and in README §8 Evaluation. The "Cisco AI Defense" line describes rules merged into the open-source cisco-ai-defense/skill-scanner repository — it is not a statement about any vendor product.
BREAKING
  • ▸Compound detection gate: MCP rules now require 30%+ conditions to match in skill context. This prevents over-triggering when MCP-specific patterns appear in legitimate SKILL.md documentation.
  • ▸Code block suppression: Skill rules no longer suppress matches inside code blocks (they are instructions, not documentation examples).
  • ▸All fields resolve to content in skill context: tool_description, tool_args, user_input all map to the full skill content for static analysis.
Added
  • ▸26 new rules (87 to 113 total, 9 categories): - ATR-2026-00149: Compound exfiltration (12 patterns: SSH archive, wallet, browser, DNS, IMDS) - ATR-2026-00158/159/160: TC-crystallized rules (first rules generated end-to-end by Threat Cloud) - ATR-2026-00161: Cross-tool shadowing via IMPORTANT tag (from Invariant Labs PoC) - ATR-2026-00162: Credential exfil combo - ATR-2026-00163: Hidden override instructions - ATR-2026-00164: Scope hijack - Plus 19 more across prompt-injection, tool-poisoning, and data-exfiltration categories
  • ▸RFC-001 v1.1 (ATR Quality Standard): - Two-Dimensional Compliance Model (metadata presence + provenance) - L0-L5 Review Tier Levels (first standard to include LLM-assisted review as formal tier) - Community Signal Aggregation (confirmations, FP reports, evasion reports) - Multi-Runtime Compatibility (14 runtimes: Claude Code, Cursor, Hermes, OpenAI Assistants, Google A2A, etc.) - Relaxed experimental gate (3/3/0) for community velocity; stable tier retains full 5/5/3 - Future Work: RFC-002 (Detection Types), RFC-003 (Collective Defense), RFC-004 (Enterprise)
  • ▸GOVERNANCE.md v1.1: - Permanent MIT license commitment (never BSL/SSPL) - MITRE ATLAS positioning: ATR = detection rules complement to ATLAS (like Sigma to ATT&CK) - ATR Numbering Authority specification - Technical Advisory Group roadmap
  • ▸96,096 real-world skills scanned (OpenClaw 56K + Skills.sh 3K + Hermes 123 + ClawHub 36K + MCP Registry 5K) - 751 malware skills discovered (hightower6eu 354, sakaen736jih 212, 52yuanchangxing 137) - Research report: docs/research/openclaw-malware-campaign-2026-04.md - NousResearch notified (hermes-agent#9809)
  • ▸Threat Cloud blacklist: 554 entries uploaded from wild scan
  • ▸4 export formats: SARIF v2.1.0, Splunk SPL, Elasticsearch DSL, generic regex (714 portable patterns)
  • ▸Exponential backoff for npm registry crawl (fixes Daily Ecosystem Scan rate limiting)
Changed
  • ▸Engine: compound gate for context-aware matching
  • ▸Confidence formula: cross-context penalty (0.7x) for rules firing outside their tested runtime
  • ▸SKILL.md denylist reduced from 22 to 10 rules (re-enabled low-FP rules for better coverage)
Metrics
  • ▸113 rules (7 stable, 85 experimental, 21 draft)
  • ▸361 tests passing
  • ▸SKILL.md benchmark (498 corpus): 100% recall, 97% precision, 0.20% FP rate
  • ▸MCP benchmark (PINT 850): 62.7% recall, 99.6% precision
  • ▸Wild scan (96K): 1.35% flag rate, 751 confirmed malware
  • ▸OWASP Agentic Top 10: 10/10 coverage
  • ▸MITRE ATLAS: 100/113 rules mapped
  • ▸SAFE-MCP: technique coverage figure (withdrawn 2026-10: it was not backed by rule-level data; see docs/SAFE-MCP-MAPPING.md)
  • ▸Avg scan latency: 14ms per file
Ecosystem
  • ▸Cisco AI Defense: 34 ATR rules shipped in production (PR #79 merged)
  • ▸OWASP: Attack examples contributed (PR #814 merged)
  • ▸NousResearch: Malware campaign reported (hermes-agent#9809)

v1.0.0

2026-04-06
BREAKING
  • ▸Rule IDs: 3-digit → 5-digit (ATR-2026-001 → ATR-2026-00001). Zero-padded, detection logic unchanged.
Added
  • ▸ATR-SPEC-v1.md — Formal rule format specification. Third parties can build conforming engines in any language.
  • ▸GOVERNANCE.md — Contribution process, quality gates, severity rubric.
  • ▸`scan_target` metadata — Every rule declares mcp or skill. Engines filter by scan context.
  • ▸`rule_version` field — All rules carry version number. Bump on detection logic changes.
  • ▸Unified CLI — atr scan auto-detects JSON (MCP) vs .md (SKILL.md).
  • ▸`ScanResult` type — Unified output with scan_type, content_hash (SHA-256).
  • ▸MCP server `atr_scan_skill` tool — 7 tools total.
  • ▸6 new skill rules: context poisoning (00125), rug pull setup (00126), subcommand overflow (00127), HTML comment hidden payload (00128), unicode smuggling (00129), exfil URL in instructions (00135).
Metrics
  • ▸87 rules (75 MCP + 12 skill)
  • ▸53,399 real-world skills scanned (skills.sh + OpenClaw)
  • ▸670 confirmed malicious skills detected
  • ▸SKILL.md recall: 94.4%, FP rate: 0.096%
  • ▸MCP recall: 62.7% (PINT 850, unchanged)

v0.3.0

2026-03-18
Added
  • ▸Evaluation framework (src/eval/): - npm run eval: 341-sample corpus (321 attacks + 20 benign), 9 attack categories - npm run eval:pint: 850-sample external PINT benchmark - Per-rule quality metrics (TP/FP/matchCount per rule) - Confusion matrix, precision/recall/F1, latency percentiles - Regression gate (auto-fail on metric degradation) - JSON report output (data/eval-report.json)
  • ▸CI gate (.github/workflows/eval.yml): typecheck + test + eval + validate on PR
  • ▸279 auto-extracted corpus samples from rule test_cases
  • ▸8 new detection layers in ATR-2026-001: - forget-everything shorthand, task switching, system prompt extraction, - praise-then-redirect, German formal/informal, French injection patterns
  • ▸PINT benchmark integration (deepset/prompt-injections + Lakera gandalf datasets)
Changed
  • ▸Embedding similarity threshold: 0.82 -> 0.65 (10 extra TP, 0 extra FP on PINT)
  • ▸Test count: 225 -> 246 (+21 eval framework tests)
Fixed
  • ▸shadow-evaluator.ts type error (TS2352)
  • ▸Removed external product references from ATR-FRAMEWORK-SPEC.md
  • ▸Added temp file patterns to .gitignore
Benchmark Results (honest numbers)
  • ▸Self-corpus (341 samples): Precision 100%, Recall 99.4%, F1 99.5%
  • ▸PINT external (850 samples): Precision 99.4%, Recall 39.9%, F1 57.0%
  • ▸Only 6/61 rules fire on external attacks
  • ▸See LIMITATIONS.md for full analysis

v0.2.3

2026-03-16
Added
  • ▸9 new rules from Threat Cloud community promotion (ATR-2026-100~108): consent bypass, trust escalation, disguised analytics exfiltration, hidden safety bypass, persona hijacking, silent action concealment, schema-description contradiction, delayed execution bypass, Sybil attack
  • ▸Python engine (pyATR) v0.2.0: validate, test, stats CLI commands, 48 tests
  • ▸Splunk SPL converter (atr convert splunk)
  • ▸Elastic Query DSL converter (atr convert elastic)
  • ▸Layer 3 LLM-as-judge prompt templates (docs/layer3-prompt-templates.md)
  • ▸Automated scan pipeline (scripts/auto-scan-pipeline.sh)
  • ▸Deployment guide for external teams (docs/deployment-guide.md)
  • ▸MCP ecosystem security audit report: 1,295 packages, 14,299 tools
  • ▸npm crawler with pagination (795 → 2,769 discoverable packages)
Fixed
  • ▸CLI test runner: handle tool_description field, fix event type mapping
  • ▸All 61 rules pass embedded test cases (556/556, 100%)
  • ▸CJK test cases moved to evasion_tests (honest: regex can't match them)
  • ▸Removed all external product references for ATR independence
  • ▸Fixed pyATR URLs pointing to wrong GitHub org
Stats
  • ▸61 rules (44 experimental + 17 draft)
  • ▸556 test cases (100% pass rate)
  • ▸164 TypeScript tests + 48 Python tests = 212 engine tests
  • ▸12 SIEM converter tests

v0.2.2

2026-03-14
Fixed
  • ▸ReDoS vulnerability in SSRF rule (ATR-2026-013) — O(n^2) backtracking on long hostnames
  • ▸SSRF rule false positive on filesystem paths like /home/user/
  • ▸tool_args field extraction fallback for tool_call events
Added
  • ▸True negatives for 21 rules with insufficient test coverage (7 had zero, 14 had only 1)
  • ▸Vitest coverage reporting with v8 provider (60%+ threshold on core modules)

v0.2.1

2026-03-10
Changed
  • ▸Standardized 17 predicted rule IDs (ATR-PRED → ATR-2026-080~096)
  • ▸Fixed rule validator for skill-compromise category
  • ▸Toned down coverage claims to reflect actual verification status

v0.1.0-rc2

2026-03-09
Added
  • ▸32 initial experimental detection rules across 9 attack categories
  • ▸TypeScript reference engine with SessionTracker
  • ▸OWASP Top 10 for Agentic Applications (2026) mapping (6 covered, 2 partial, 2 gaps)
  • ▸13 CVE reference mappings across 16 rules (pattern-based, not empirically verified)
  • ▸OWASP LLM Top 10 (2025) mapping (7 covered, 3 gaps)
  • ▸MITRE ATLAS technique references
  • ▸JSON Schema specification (spec/atr-schema.yaml)
  • ▸Built-in true positive and true negative test cases for every rule
  • ▸Attack corpus validation tests
  • ▸Coverage report (COVERAGE.md)
Attack Categories
  • ▸Prompt Injection (5 rules)
  • ▸Tool Poisoning (4 rules)
  • ▸Context Exfiltration (3 rules)
  • ▸Agent Manipulation (3 rules)
  • ▸Privilege Escalation (2 rules)
  • ▸Excessive Autonomy (2 rules)
  • ▸Skill Compromise (7 rules)
  • ▸Data Poisoning (1 rule)
  • ▸Model Security (2 rules)