Skip to content
Agent Threat Rules (ATR)中文
For red teams

You find the attack.
It becomes a rule.
Four days later it was on npm.

Take Semantic Kernel CVE-2026-26030. Microsoft wrote it up on 2026-05-07, and the matching ATR rules reached npm in v2.1.2 on 2026-05-11. The CVE record itself had been public since February. Write an attack down once and every defender can call it by the same name.

Why an attack belongs in the standard
4 days
From Microsoft's write-up to the rule on npm

Microsoft Security wrote up Semantic Kernel CVE-2026-26030 on its blog on 2026-05-07. The CVE record itself had been public since 2026-02-19, so counted from the record it took 80 days. On 2026-05-10 the ATR maintainer filed AGT#1937, mapping the CVE to ATR rules. An AGT maintainer assigned that issue to Microsoft's Copilot SWE Agent, and at 06:07 UTC on 2026-05-11 it opened a regression-test PR that assumed ATR already covered the CVE. The paired rules merged into ATR at 08:22 UTC, 2h 16m after that PR opened. v2.1.2 was on npm at 08:24.

30
Integrations recorded as shipped in ADOPTERS.md

The full list is on the ecosystem page. The projects below are examples where the ATR maintainer opened a pull request and the project's own maintainers reviewed and merged it. The one exception is Cisco's rule-packs CLI (#80), which a Cisco maintainer wrote. Microsoft Agent Governance Toolkit took them as example directories. Cisco AI Defense took a rule pack in skill-scanner. On 2026-11-02 a Cisco maintainer wrote that they will likely remove ATR because the false-positive rate is too high, and a report-only pack (#289) was closed unmerged on 2026-11-05. CIRCL/MISP merged a taxonomy and a galaxy. The OWASP Agent Security Regression Harness merged four scenarios derived from ATR rules, and the files do not name ATR. Gen Digital Sage merged agent-layer threat patterns.

Full list and evidence on the ecosystem page
836
Rules, each with an author line and a stable ID

Every rule has an author field and an ATR-YYYY-NNNNN ID that never changes once published. Many also record the corpus or advisory they came from in metadata_provenance. If you bring the attack, your name can go on the author line.

0 FP
Required of a rule pull request before it merges

The rule safety gate (scripts/check-rules-safety.ts) runs inside the required Validate Rules & Build check on every pull request. It checks the rules that pull request adds or changes. Each rule's own true positives must match. It needs 0 FP on the benign, extended, code and research-mention corpora, 0 conflicts with other rules, and coverage of its own true_negatives. The gate prints the sample counts it used on every run, and those are the figures of record, not any number on this page. Passing it is the merge condition for rule pull requests now. Earlier batches did not all pass. Re-run at main b9d16840 on 2026-10-22, it fails 33 of the 71 rules from the 2026-05-12 sprint and 4 of the 29 from the 2026-06 mega-scan, the same as on 2026-10-19 (data/gate-reruns/2026-10-19_red-team-rule-batches.json). Missing a clever variant is the preferred failure over shipping a false positive. Figures are published per lane, not as one flattering number.

What you get

What your attack leaves you once it's in the standard depends on who you are.

Academic researchers

The attack you described gets a citable artifact next to your paper. It has a stable ID and real downstream users. A paper alone can sit behind a paywall.

Your name can go on the rule file's author line, next to a rule ID that never changes once published.

Corporate red teams

The variant your team found stops being an internal finding. It becomes something every defender can name. Your team can be named as the author in the ATR rule corpus, parts of which projects at Cisco AI Defense, MISP and Gen Digital Sage have merged.

An ATR maintainer filed an issue mapping two CVEs to ATR rules. Microsoft's Copilot SWE Agent was assigned it and opened a PR that assumed ATR coverage (issue AGT #1937, PR #1981, merged 2026-05-11). Authoring an ATR-2026-NNNNN rule is a record anyone can check.

Independent researchers

Ship a detection without writing the regex yourself. Send attack examples and benign ones. A maintainer writes the regex from them and runs it through the full gate.

Your benign look-alikes become the rule's true negatives. Before it merges, the regex must match none of them and hold 0 FP on the benign corpora. That is the rule safety gate every rule pull request has to pass now. Rules from earlier batches did not all pass it; the re-run record is in data/gate-reruns/2026-10-19_red-team-rule-batches.json.

Bug bounty hunters

One attack, two payoffs: the bounty, and a permanent rule that catches it everywhere after. It fits alongside Huntr.dev, HackerOne's AI scope and Protect AI's bounty program.

ATR proposals are MIT-licensed and citable, and there is no NDA conflict. The bounty program owns the responsible-disclosure window. ATR ships the detection only after disclosure, and credits the finder publicly.

What happens after you submit

You send samples. A maintainer writes the regex. It merges only if the gate passes.

  1. 01

    You fill in a form

    3 attack samples, 3 benign look-alikes, the attack category, and the paper or repo it came from. It takes 3 to 5 minutes.

    You don't learn the schema or write YAML, and you don't fork anything.

  2. 02

    A maintainer writes the regex from your samples

    A maintainer reads your issue and writes a detection regex from your attack samples. Your benign look-alikes become the true negatives. You can also propose a regex yourself, in a PR comment or a PR of your own.

    To merge, your pull request has to pass the gate. Every one of your true positives must match. It needs 0 FP on the benign, extended, code and research-mention corpora, and 0 conflicts with other rules. The gate prints the sample counts it used. Rules from earlier batches did not all pass it; the re-run record is data/gate-reruns/2026-10-19_red-team-rule-batches.json.

  3. 03

    The gate passes, and the full rule goes up as a PR

    Whoever wrote the regex, someone checks its shape. Is it too literal? Could it generalize? If the gate fails, it stays a proposal while the regex is tightened, and your test cases are still the ground truth.

  4. 04

    Downstream copies are refreshed by pull request

    ATR rules merged into other projects do not update themselves. The ATR maintainer refreshes them by pull request: for example the MISP galaxy in #1236 on 2026-07-14 and the Cisco pack in #142 on 2026-08-03. If you use the npm package, a new release brings the new rules.

    Microsoft Copilot's regression PR went from opened to the matching rules merged into ATR in 2h 16m (2026-05-11, 06:07 to 08:22 UTC). Two minutes later, v2.1.2 was on npm. That counts from the PR only. From Microsoft's write-up on 2026-05-07, it was four days.

Offensive tools and corpora

These offensive tools and corpora already feed the standard. Yours could be next.

Only red-team tools are listed here: offensive testing frameworks and adversarial corpora. Defensive frameworks are on /ecosystem.

Merged upstream or corpus ingested (4)

Microsoft PyRIT

Microsoft AI Red Team
Merged upstream

The toolkit Microsoft uses internally to red-team production LLM products. Roman Lutz leads.

Added an ATR dataset loader exposing the rule corpus as PyRIT attack sources; PR #1715 merged 2026-05-27.

HackAPrompt

Schulhoff et al. — Learn Prompting · EMNLP 2023
Corpus ingested

A dataset of 600K+ prompts (601,757 rows in the released version) from a global prompt-hacking competition, in which players tried to make gpt-3.5-turbo, FlanT5-XXL or text-davinci-003 print a target phrase or give up a hidden secret key. The paper describing it (Schulhoff et al., EMNLP 2023) is listed under Best Paper Awards at EMNLP 2023.

Shipped 5 ATR rules (ATR-2026-00452..00456), promoted to rules/ on 2026-05-12, each written for an attack family the corpus shows: a forced target phrase, secret-key reveal demands, per-character backslash encoding, a no-period output modifier, and empty-input conditionals; other rules name HackAPrompt in their YAML too. Recall is measured on a fixed 4,780-prompt subset (a seeded random sample of 5,000 rows at a pinned dataset revision, less the 220 shorter than 3 characters): each prompt goes through the evaluation harness as LLM input and counts as detected when any rule matches. Several of the rules naming HackAPrompt were written from this exact subset, so the current recall that follows is in-sample; it is followed by the ablation reference, the same prompts with every rule naming HackAPrompt removed. The corpus is 100% adversarial and carries no benign population, so it yields no precision or false-positive figure.

Current recall, hunt lane (every rule maturity) / enforce lane (stable rules only, the one for blocking): HackAPrompt (4,780 attack samples): 83.1% (hunt) / 49.0% (enforce), and 28.7% (hunt) / 0.7% (enforce) without the 20 rules whose YAML names it (the ablation reference: it approximates an out-of-sample figure but does not guarantee one) (ATR 4.1.5, measured 2026-10-19).

Promptfoo + llm-guard

Promptfoo Inc. · Protect AI
Corpus ingested

Attack examples from promptfoo's red-team plugins and test cases from llm-guard's scanner tests, both MIT-licensed.

Shipped 6 ATR rules (ATR-2026-00500..00505), promoted to rules/ on 2026-05-12. Four are generalised from promptfoo plugin examples (SSRF, data exfiltration, divergent repetition, tool discovery) and two from llm-guard test cases (fake error-state takeover, and system prompt extraction, which also draws on promptfoo): literal hostnames, URLs, words and tool names in the examples were replaced with patterns. Each rule's description names the corpus it came from. The 2026-05-12 import they came from held 94 samples (44 llm-guard, 44 promptfoo, 6 NeMo Guardrails); checked against the vendor repositories, 79 occur verbatim in the vendor file they name, 3 are truncated or edited from it, 2 come from another vendor's suite, and 10 are not found verbatim anywhere in the vendor repository at the pinned commit. The rules are now measured on 126 attack and 20 benign test cases, extracted by script from pinned commits of llm-guard, promptfoo and NVIDIA NeMo Guardrails (Apache-2.0; a measurement source only, no rule was derived from it) and deduplicated, each counted as detected when any rule matches. Two of the three suites are the rules' own sources, so the current recall that follows is partly in-sample; it is followed by the ablation reference, the same cases with every rule naming one of the three vendors or the vendor-corpus-import tag removed. Even that is not out-of-sample: some remaining rules carry vendor text in their YAML, and five NeMo Guardrails cases share a jailbreak template with the garak corpus other rules were mined from (the measurement file quantifies both). The 20 benign cases are too few for a precision claim; only the count flagged is given. Since agent-threat-rules#709, ATR-2026-00500 and -00501 match all 3 vendor cases in the categories they cite and ATR-2026-00505 matches all 6 vendor cases behind its true positives (hunt); before it, 00500 and 00501 matched 1 and 0.

Current recall, hunt lane (every rule maturity) / enforce lane (stable rules only, the one for blocking): Pinned vendor test suites (126 attack samples): 79.4% (hunt) / 5.6% (enforce), 1 of 20 benign samples flagged (hunt), and 61.9% (hunt) / 5.6% (enforce) without the 16 rules that name llm-guard, promptfoo or NeMo Guardrails or carry the vendor-corpus-import tag (the ablation reference: it approximates an out-of-sample figure but does not guarantee one) (ATR 4.2.0, measured 2026-10-23).

OWASP LLM Top 10 + MITRE ATLAS

OWASP · MITRE
Corpus ingested

OWASP's Top 10 risks for LLM applications, and MITRE ATLAS, the catalog of adversarial techniques and case studies against AI systems.

Shipped 8 ATR rules (ATR-2026-00510..00517) on 2026-05-12. Each rule's references name an OWASP LLM Top 10 (2025) category (LLM01, LLM02, LLM06 or LLM10) and a MITRE ATLAS technique, and six of the eight descriptions cite the ATLAS case study they follow (AML.CS0022, CS0036, CS0038, CS0039 and CS0045, CS0041, CS0056).

Open proposals (2)

TextAttack

QData · ACL 2020
Proposal open

NLP adversarial-attack framework from the QData lab (ACL 2020).

Issue #824 proposes a textattack-detection-atr companion PyPI package — plugin-shaped, zero core changes. Single ask: README cross-link.

Promptfoo

Promptfoo Inc.
Proposal open

Open-source LLM red-teaming and evaluation tool. Promptfoo generates and runs adversarial tests against LLM applications.

PR #8529 adds an MCP red-team example using ATR as the deterministic defense layer. Promptfoo runs the probe; ATR rules return the verdict.

Continuous corpus integration

Five corpora, 71 new rules.

HackAPrompt recall went from 28.6% to 66.0%. That is in-sample: from the 2026-05-12 sprint, with every rule loaded, measured on the same 4,780-prompt subset the rules were mined from.

On 2026-05-12, 11 agents running in parallel ingested five external corpora. The sprint's two commits (e9efefdd, 92c0d0fe) added 71 rule files, ATR-2026-00452..00522. The v2.2.0 changelog says 75 because it also counts four CVE rules, 00448..00451, added earlier that day in commit 80dbd0ab, before the sprint commits. The full version record is at /changelog.

Both HackAPrompt figures are re-runs on a fixed 4,780-prompt subset, a seeded sample at a pinned dataset revision. One ran at the commit before the sprint (3a6145b7, 1,368 detected), the other at the sprint's release commit (92c0d0fe, 3,154 detected). Both loaded every rule, which is what the hunt lane loads today. The 28.6% is itself in-sample: ATR-2026-00442..00447, mined from the same subset, were already loaded.

The 66.2% published at the time could not be reproduced at the sprint commits e9efefdd and 92c0d0fe (66.0% at best), so it is withdrawn. Current hunt and enforce figures, and the ablation reference, are in the HackAPrompt entry above. The re-run procedure is in data/hackaprompt/METHOD.md.

Rules now come from CVE and advisory sweeps, mining benchmark misses, and contributors. Each one reaches main through a pull request a maintainer merges. Since June the standard has grown from 462 to 836 rules, published on npm as agent-threat-rules.

HackAPrompt
Samples: 4,780 (seeded subset of 601,757)
Rules: 5ATR-2026-00452..00456
Recall: 28.6% to 66.0% (hunt, every rule loaded, in-sample; re-run at 3a6145b7 and 92c0d0fe)
promptfoo + llm-guard
Samples: 94 (2026-05-12 import, 6 of them NeMo Guardrails; 79 occur verbatim in the vendor file they name)
Rules: 6ATR-2026-00500..00505
Recall: 79.4% (hunt) / 5.6% (enforce) (126 pinned vendor attack cases, partly in-sample; ablation reference 61.9% (hunt) / 5.6% (enforce); ATR 4.2.0, measured 2026-10-23)
PromptInject
Samples: full corpus
Rules: 4ATR-2026-00506..00509
OWASP LLM Top 10 + ATLAS PoCs
Samples: 8 standard categories
Rules: 8ATR-2026-00510..00517
Garak in-the-wild jailbreak
Samples: 650
Rules: existing coverage
Recall: 92.3% (hunt) / 0.9% (enforce) (ATR 4.1.5, measured 2026-10-15; partly closed-book)

The gate record for these 71 rules

No record of the quality gate's run on these 71 rules was committed. The claim that all of them passed with 0 FP on the benign corpus is withdrawn. In its place are two re-runs of the repository's rule safety gate, scripts/check-rules-safety.ts, on 2026-10-19 and 2026-10-22.

With the gate and corpora as of 92c0d0fe, none of the 71 flags any of the 432 benign SKILL.md files. 10 fail, all for missing one of their own true positives. Of those 10, 1 also flags 1 of 1,352 extended benign samples, 1 flags 1 of 157 research-mention samples, and 2 conflict with another rule's true negatives.

At main b9d16840 on 2026-10-22 (ATR 4.1.5): 33 fail; 26 flag at least one benign sample (4 on the 467 SKILL.md files, 26 on 13,620 extended samples, 1 on 69 code samples, 2 on 157 research-mention samples). None of those 26 has maturity stable, so none loads in the enforce lane. 24 load in the alert and hunt lanes, 2 in the hunt lane only. The 2026-10-19 run at 9d74fc50 gave the same results, even though nine of the recorded rules changed on main in between.

The worst by far is ATR-2026-00454 (maturity test, alert and hunt lanes): 152 of the 467 SKILL.md files and 1,376 of the 13,620 extended samples; every other rule flags at most 23 samples on any corpus. 00454 also counts toward the HackAPrompt hunt and alert figures above: in the 2026-10-19 HackAPrompt measurement it matches 757 of the 4,780 prompts in the hunt lane. Record: data/gate-reruns/2026-10-19_red-team-rule-batches.json.

Separately, 53 rules had their regex generalized from literal fingerprints to multi-layer structural patterns.

2026-06 mega-scan

8 corpora, 29 detection rules, 2 with no new misses

8 public agent red-team corpora were run against the existing rules to find the attack variants they missed, and those misses became new rules. 6 corpora produced 29 detection rules. The other 2, TensorTrust and PoisonedRAG, turned up nothing the existing rules didn't already catch.

LLMail-Inject
ATR-2026-01860..01865
InjecAgent
ATR-2026-00550 / 00584 / 00700 / 00859 / 01000
AgentDojo
ATR-2026-00715 / 00720 / 01751..01754
ToolEmu
ATR-2026-00702 / 00718 / 00719 / 01301
MCPSecBench
ATR-2026-01300 / 01306 / 01307 / 01310 / 01615 / 01616
AgentPoison
ATR-2026-01774 / 01800
TensorTrust
Already covered, no new misses
PoisonedRAG
Already covered, no new misses

The gate record for these 29 rules

29 distinct rules after de-duplication. Several came from misses shared across corpora. By the maturity in their rule files: 15 stable, 12 test, 1 experimental, 1 draft. No record of their gate runs was committed, so the claim that all cleared it with 0 FP is withdrawn.

In its place is a measurement: the rule safety gate re-run at main b9d16840 (ATR 4.1.5) on 2026-10-22, with the same results as on 2026-10-19. None of the 29 flags any of the 467 benign SKILL.md files; 4 fail, 3 of them flagging 2 to 4 of the 13,620 extended benign samples each. All three are maturity test, so they load in the alert and hunt lanes, not enforce, and one also conflicts with another rule's true negatives.

The fourth, ATR-2026-00550, uses the trace detection method, which the gate cannot run on a text sample, so it does not pass. On 2026-10-19 the gate reported it as missing its own true positives; that check changed in 2f0605c3. Same record as above.

Left out on purpose

What ATR doesn't import, and why.

ATR doesn't import every corpus it can find. These datasets were deliberately kept out of rule writing, with the reason for each.

PyRIT Pliny L1B3RT4S
Microsoft AI Red Team / Pliny
Refused
Anthropic's usage policy kept the subagents doing the import from reading this dataset. ATR does not import material whose provenance it cannot verify.
AdvBench
Zou et al., GCG paper
Reclassified
Moved to the test corpora (data/test-corpora/) instead of being used as a rule source. AdvBench describes the behaviors an attacker is after, not wrapped attack payloads. It measures attack success rates and doesn't directly give you a pattern to detect.
HarmBench
Center for AI Safety · Dan Hendrycks
Reclassified
Moved to the test corpora. HarmBench's 320 behaviors describe target outputs, not attack syntax. It is used to measure ATR's coverage, not to write rules.
JailbreakBench
Princeton · Patrick Chao
Reclassified
Moved to the test corpora. JBB-Behaviors is the standard input for a jailbreak leaderboard. It describes what is being asked for, not attack patterns. It is used for benchmarking, not for writing rules.
Corpus-fingerprint rules (kept as-is)
e.g. ATR-2026-02016
Kept, not generalized
A few rules anchor on very literal strings from one corpus, such as the reversed canary word from HackAPrompt in ATR-2026-02016. They fingerprint that corpus rather than an attack pattern that generalizes, and their descriptions say so. Loosening them would push the FP rate past what is acceptable, so they stay literal. The rules whose descriptions say this are all at test maturity; the blocking lane loads only stable, so they alert and never block.
Submit a probe

Spend 3 to 5 minutes on a form. Your name stays on the rule, and every defender who adopts ATR shares it.

MIT licensed, no CLA, no telemetry, free for good. You keep every right to publish the attack itself. ATR only carries the detection, as a rule anyone can cite by its stable ID.

836 rules deployed across 10 threat categories. Each has an author field and a stable rule ID.

Interests: the maintainer founded and controls PANGUARD AI, INC., which sells AI agent security products built on ATR.