The numbers, and what ATR misses.
The evasion tests are public. So is the list of attacks the rules can't catch.
Why ATR exists, argued from the start. Once agents act on their own, trust can't be assumed, and detection has to become a standard layer that anyone can audit and the community versions and maintains. It covers the RFC-001 quality specification, the in-the-wild ecosystem scan, and how the standard keeps updating instead of stopping at one snapshot. The scan figures in the paper are out of date. The ones to cite are 101,280 items scanned and 1,434 flagged (engine v2.0.0, 2026-04-13).
The wild scan flagged 1,434 items, clustered in three accounts that published in bulk: hightower6eu (354 skills), sakaen736jih (212) and 52yuanchangxing (137). Payloads include encrypted zips, base64 shell commands and C2 callbacks. They were reported to NousResearch and added to a public blacklist. The verdicts are made per publisher and applied to the whole account, not file by file. The report's totals are out of date; use the figures on this page.
A large scan of agent skills and MCP definitions across five public registries. It found three coordinated threat accounts. One rule, ATR-2026-00121 (credential access through tool descriptions), accounts for 53% of flagged skills. The paper's totals are out of date. The ones to cite are 101,280 items scanned and 1,434 flagged (engine v2.0.0, 2026-04-13).
The MCP attack surface, measured. 30 CVEs in 60 days, and 38% with no authentication at all. The paper sorts attacks into 7 classes and scans 53K ecosystem items. That pace is 15x faster than Docker's first two years.
Benchmarks
Measured on ATR's own corpus and on public attack datasets: deepset, Lakera Gandalf, HackAPrompt and garak.
This is not Lakera's official PINT benchmark. That corpus is private and about 5x larger. This one is self-built in PINT's format: 850 samples from deepset/prompt-injections (660) and Lakera/gandalf_ignore_instructions (190). Only a small subset of rules fire on it, and one rule accounts for most detections. Read it as a score for the prompt-injection family, not for ATR's overall coverage. Partly closed-book: some hunt-lane rules were mined from this corpus's own misses, so the hunt figure is not fully out-of-sample.
This corpus is built partly from the rules' own true positives, so a high score is expected. It checks that the rules still catch what they were written to catch. It is not a benchmark and does not compare with the corpus on the left.
In the hunt lane, precision is 100.0% and recall is 65.4%. That gap is expected. Regex catches the patterns it knows and misses paraphrases and attacks in other languages. The enforce lane loads only stable rules, and it is the one to run with blocking on. Its recall is 0.4%.
SKILL.md detection benchmark
An internal, labeled SKILL.md corpus of 498 files (32 malicious + 466 benign; ATR 4.1.5, measured 2026-10-15). Some of the malicious files were written for this benchmark. Layer A is obvious payloads: curl | bash, base64 exec, reverse shell. Layer C is semantic attacks: natural-language instructions, social engineering. The corpus has no Layer B (obfuscated) samples.
The enforce lane loads stable rules only, and it's the one to run with blocking on. It detects 0 of the 32 malicious samples (0.0%). The measurement has no per-layer enforce split.
Ecosystem scans
The rules, run against real MCP skill registries.
CRITICAL and HIGH count findings: the 1,434 flagged items produced 1,507 findings, and one item can have several.
The wild scan above ran on 2026-04-13. Rules also come from CVE and advisory sweeps, from benchmark misses and from contributors. The ruleset has grown from 462 to the 836 effective rules on main today, plus 8 inert (latest npm release [email protected], as of 2026-11-20).
How the numbers are made
A number only counts if someone else can reproduce it. Everything here can be rerun. The datasets, scan scripts and evaluation scripts are open source under the MIT license. Run them on the same ATR version and you get these figures, or you show they're wrong.
Five sources, 101,280 items in total (engine v2.0.0, 2026-04-13). The largest are OpenClaw 56,480, ClawHub 36,394 and Skills.sh 3,115. The other two are Hermes Agent and an MCP registry index. Each per-source count comes from that crawler's own report, taken at a different time from the total, so they won't add up to it. Every source is crawled through a public HTTP API or git repository.
The detection core is deterministic. Rules run regex and AST matching with no LLM inference, so the same input gives the same result anywhere. That core is what ships on by default. Since v3.1.0 there is also an optional, experimental semantic judge stage for paraphrased attacks the patterns miss. You have to turn it on, and it never changes the deterministic core. Before a new rule merges, it has to pass a safety gate: zero false positives on a corpus of benign skills.
Precision and recall run on a self-built corpus in PINT's format. Recall is reported for both lanes, precision for the hunt lane only. The samples come from public datasets (deepset/prompt-injections, Lakera/gandalf_ignore_instructions). Some hunt-lane rules were later mined from this corpus's own misses, though, so the hunt figure is partly closed-book. The corpus is ATR's own, not Lakera's official PINT benchmark. A separate internal SKILL.md benchmark uses labeled files, and some of its malicious files were written for that benchmark.
False positive rate is measured on real benign samples, meaning skills that passed manual or community review: the benign samples flagged, divided by all benign samples scanned. It is reported per lane, not as one flattering number. The enforce lane runs only the most mature rules. The default hunt lane runs every rule as an advisory signal. The per-lane figures once published here are withdrawn until they are re-measured: the benign corpus behind them turned out to contain real jailbreak samples, so the baseline didn't hold. No figure appears here until the re-measurement is done. Every documented FP context still goes into the rule YAML's false_positives field and shows on the rule page.
Scan checkpoints, test sets and evaluation scripts are public under data/ and tests/. Rerun the scan with the same ATR version and you get identical results.
External citations
ATR is meant to be cited, embedded and built on. External citations are recorded here, and so is the fact when there are none.
No external citations recorded yet. If your paper, technical report or product docs cite ATR, open a GitHub issue to tell the maintainer.
Cite as: Agent Threat Rules (ATR) Project (2026). Agent Threat Rules: An open detection standard for AI agent threats. DOI: 10.5281/zenodo.19178002
Cite as: Lin, Kuan-Hsin (2026). The Collapse of Trust. DOI: 10.5281/zenodo.19178002
What's next
Next, detection looks at runtime behavior and meaning, not only static patterns, and the hard cases get written back into deterministic rules. Each step shows up in GitHub releases and paper updates, so anyone outside can follow the progress.
Takes detection past static regex to runtime behavior. The first rules look for env variable reads followed by network calls, odd tool-call rates, and shell access nobody expected.
Cases Tier 3 can't decide go to an LLM-as-judge for semantic analysis. What the LLM finds is written back into ATR as Tier 2 regex rules.
Sigma: the repo has a converter script (scripts/generate-sigma.py) that turns ATR rules into Sigma YAML. There is no ratified Sigma logsource for agent traffic yet, so it uses a custom ai_agent logsource, and you map the fields yourself when wiring it into a SIEM. ATR is also listed in the SigmaHQ tools directory. YARA: ATR has no official compiler; the third-party atr-to-yaral converts ATR rules into YARA-L 2.0 for Google SecOps.
ATR matches text: static artifacts such as skills and tool descriptions (the ecosystem scan above is one), and runtime inputs, outputs and tool calls. A backdoor planted in a model's weights during training is not in any of that text, so it is invisible to ATR by design. Closing that gap needs new techniques that tie supply-chain provenance (model cards, training data audits) to runtime behavioral fingerprints.
Sources: ATR-FRAMEWORK-SPEC.md Phase 2-4 roadmap and the main paper's future work section.
What ATR can't detect
Before you rely on these rules, you should know where they miss.
Any regex rule can be dodged by saying the same thing another way. “Ignore previous instructions” is caught. “Please set aside the guidance you were given earlier” is not.
Nearly every pattern is in English. A few rules also match phrasing in Chinese, Japanese, Korean, Russian or Arabic, and only for the attacks they were written for. An injection in any other language, or a new one in those, mostly gets through.
“Delete all records” can be a normal request or an attack. Regex sees the words. It can't tell whether the agent was allowed to do it.
ATR reads content, not the transport. Message replay, schema manipulation and a man-in-the-middle on the MCP transport never reach it.
Trust built up slowly over 20 turns, where no single message looks wrong, goes unnoticed. ATR judges each event on its own and doesn't connect them.
Regex can't match a pattern nobody has seen yet. A new technique needs a new rule.
Interests: the maintainer founded and controls PANGUARD AI, INC., which sells AI agent security products built on ATR.
