Runaway Agent Loop Detection
Detects when an agent enters a runaway loop through repeated identical actions, infinite retry patterns, or recursive self-invocation.
runaway loop · pattern · confidence: medium
- 嚴重度
- high
- 車道
- alert · hunt比對
到 就標出、回報 - 成熟度
- test還沒定論
- 掃描目標
- mcp
- 狀態
- experimental
框架對照
- OWASP Agentic Top 10
- ASI05:2026
- OWASP LLM Top 10
- LLM06:2025
- LLM10:2025
這些
說明
Detects when an agent enters a runaway loop through repeated identical actions, infinite retry patterns, or recursive self-invocation. This rule uses regex-only detection to identify loop indicators in agent output and tool call content, such as retry counters, repeated action descriptions, recursive invocation patterns, and stalled progress indicators. Runaway loops waste computational resources, accumulate costs, and may indicate logic errors or adversarial manipulation.
野外驗證
偵測條件
組合方式:any- 01Retry counter pattern indicating repeated attempts欄位:content運算子:regex
- 02Attempt X of Y counter pattern (e.g., attempt 5 of 10)欄位:content運算子:regex
- 03Natural language retry patterns in agent output欄位:content運算子:regex
- 04Stalled progress with persistent error indicators欄位:content運算子:regex
- 05Recursive self-invocation patterns (v2 — dropped the bare 'self-call' / 'self-invoke' token, which agent-framework documentation uses as ordinary vocabulary when describing how an agent may call itself. The remaining branches require the recursion to be asserted as happening, which is the runaway signal; both recursion true_positives use those branches)欄位:content運算子:regex
- 06High iteration count indicating prolonged looping欄位:content運算子:regex
- 07Agent acknowledging it is repeating identical actions欄位:content運算子:regex
- 08Retry limit reached indicators欄位:content運算子:regex
- 09Explicit stall or no-progress acknowledgement欄位:content運算子:regex
- 10Infinite loop constructs in generated or executed code欄位:content運算子:regex
- 11Agent declaring intent to retry indefinitely欄位:content運算子:regex
- 12Agent spawning copies of itself (fork bomb pattern)欄位:content運算子:regex
會 觸發 的 攻擊範例
Retrying attempt 15 of 10. The request still returns the same error.
Still failing after 25 iterations. Let me try again with the same parameters.
Calling myself recursively to process the remaining items. Recursive call depth: 47.
Will keep trying until it works. Continuously retrying the API call.
iteration #450 - still processing the same batch. No new results returned from the API.
規則
不會 觸發 的 正常樣本
- Legitimate batch processing with bounded progress counter
Processing batch 3 of 5. All items completed successfully so far.
- Normal bounded retry logic in error handling
The API returned an error. Retrying once with exponential backoff.
- Successful completion after reasonable retry count
Task completed after 3 attempts. Here are the final results.
- Legitimate pipeline completion report with large numbers that do not match loop patterns
The pipeline completed successfully. Total records processed: 15000 across 4 stages. Final output saved to /data/results.csv.
- Documentation describing bounded retry architecture without using trigger phrases like 'retrying N' or 'try again'
Here is a summary of the error handling strategy: if a transient fault occurs, the system will back off exponentially and make up to 3 additional requests before returning a failure response to the caller.
已知 的 誤報情境
- ▸Legitimate batch processing with progress counters in output
- ▸Polling agents designed to check status at regular intervals
- ▸Long-running data pipeline agents with expected high iteration counts
- ▸Retry logic in error handling with bounded retry limits
- ▸CI/CD build logs that include 'attempt 2 of 3' progress counters during normal deployment retries
- ▸Monitoring dashboards displaying iteration counts for scheduled polling jobs that legitimately run hundreds of cycles
已 記錄 的 繞過手法
- 手法:language switch
Volviendo a intentar la misma operación por 30ª vez. El servidor sigue devolviendo el mismo error.
Spanish translation of retry loop language completely avoids English regex patterns. - 手法:casual paraphrase
Re-doing the exact same step for the umpteenth time... the server keeps handing back an identical failure. I'll give it another go.
Paraphrases retry and same-error concepts using synonyms like 're-doing', 'umpteenth', 'identical failure', 'give it another go' that don't match the specific regex phrases. - 手法:unicode homoglyph
sеlf_invоke triggered. Rеcursivе exеcution in progress, dеpth = 99.
Uses Cyrillic homoglyphs (е→е, o→о) in 'self_invoke' and 'recursive execution' to visually match but evade ASCII regex matching.
這些
完整 YAML
在 GitHub 編輯title: Runaway Agent Loop Detection
id: ATR-2026-00050
rule_version: 1
status: experimental
description: |
Detects when an agent enters a runaway loop through repeated identical
actions, infinite retry patterns, or recursive self-invocation. This
rule uses regex-only detection to identify loop indicators in agent
output and tool call content, such as retry counters, repeated action
descriptions, recursive invocation patterns, and stalled progress
indicators. Runaway loops waste computational resources, accumulate
costs, and may indicate logic errors or adversarial manipulation.
author: ATR Community
date: 2026/03/08
schema_version: "0.1"
detection_tier: pattern
maturity: test
severity: high
references:
owasp_llm:
- LLM06:2025
- LLM10:2025
owasp_agentic:
- ASI05:2026
mitre_atlas:
- AML.T0053 - AI Agent Tool Invocation
- AML.T0046 - Spamming AI System with Chaff Data
compliance:
owasp_agentic:
- id: ASI05:2026
context: "Runaway agent loops represent uncontrolled autonomous execution — the agent performs repeated identical actions without human intervention, matching the ASI05:2026 Unexpected Code Execution category at the behavioral level."
strength: primary
owasp_llm:
- id: LLM06:2025
context: "An agent running infinite retry loops without termination conditions exercises excessive agency beyond its task scope, accumulating costs and resource consumption classified under LLM06:2025."
strength: primary
- id: LLM10:2025
context: "Runaway loops that exhaust compute resources, API rate limits, or accumulated context constitute an LLM10:2025 Unbounded Consumption incident with direct financial and availability impact."
strength: secondary顯示全部(還有 192 行)收起
eu_ai_act:
- article: "14"
context: "Article 14 human oversight requires that AI systems can be stopped and corrected; runaway loops that resist termination or mask their loop state undermine the human override capability Article 14 mandates."
strength: primary
- article: "15"
context: "Article 15 robustness requires that AI systems handle failure states gracefully; detection of runaway loops is a monitoring control ensuring the system does not enter unrecoverable autonomous states."
strength: secondary
- article: "9"
context: "Article 9 (risk management system) requires identified risks to be addressed by appropriate measures; this rule is a runtime risk-treatment control that detects the unsafe autonomous action (Runaway Agent Loop Detection)."
strength: secondary
nist_ai_rmf:
- function: Manage
subcategory: MG.3.2
context: "Runaway agent loops are an AI system failure mode requiring active detection and termination; this rule implements the MG.3.2 response capability for AI failures and disruptions."
strength: primary
- function: Govern
subcategory: GV.1.2
context: "GV.1.2 accountability roles must include responsibility for detecting and halting runaway agent behavior; this rule provides the signal that supports that accountability."
strength: secondary
- subcategory: "MG.2.3"
context: "NIST AI RMF MANAGE 2.3 (respond to previously unknown identified risks) is supported by this rule, which surfaces the unsafe autonomous action (Runaway Agent Loop Detection) so the risk can be treated."
strength: primary
- subcategory: "MS.2.7"
context: "NIST AI RMF MEASURE 2.7 (security and resilience evaluated and documented) is supported by this rule's runtime detection of the unsafe autonomous action (Runaway Agent Loop Detection)."
strength: secondary
iso_42001:
- clause: "8.1"
context: "Clause 8.1 AI system operational control requires monitoring for abnormal execution patterns; runaway loop detection is the primary operational control for this failure class."
strength: primary
- clause: "9.1"
context: "Clause 9.1 monitoring and evaluation requires measuring AI system behavior against expected norms; loop counter patterns are the measurable anomaly indicators for this rule."
strength: secondary
- clause: "6.2"
context: "ISO/IEC 42001 Clause 6.2 (AI objectives and planning) calls for risk treatment of known attack patterns; this rule's detection of the unsafe autonomous action (Runaway Agent Loop Detection) is such a treatment."
strength: primary
tags:
category: excessive-autonomy
subcategory: runaway-loop
scan_target: mcp
confidence: medium
agent_source:
type: llm_io
framework:
- any
provider:
- any
detection:
conditions:
- field: content
operator: regex
value: (?i)(?:retrying|retry\s+(?:attempt|number|#|count))\s*(?::?\s*)?\d+
description: Retry counter pattern indicating repeated attempts
- field: content
operator: regex
value: (?i)attempt\s+\d+\s+(?:of|/)\s+\d+
description: Attempt X of Y counter pattern (e.g., attempt 5 of 10)
- field: content
operator: regex
value: >-
(?i)(?:trying\s+again|let\s+me\s+(?:try|attempt)\s+(?:again|once\s+more|one\s+more\s+time)|retrying\s+(?:the\s+)?(?:same|previous|last)\s+(?:operation|request|call|action))
description: Natural language retry patterns in agent output
- field: content
operator: regex
value: >-
(?i)(?:still\s+(?:failing|not\s+working|getting\s+(?:the\s+same\s+)?error)|same\s+error\s+(?:again|as\s+before|persists)|error\s+(?:persists|continues|remains))
description: Stalled progress with persistent error indicators
- field: content
operator: regex
value: >-
(?i)(?:calling\s+(?:myself|this\s+(?:function|tool|agent))\s+(?:again|recursively)|recursive\s+(?:call|invocation|execution))
description: Recursive self-invocation patterns (v2 — dropped the bare 'self-call' / 'self-invoke' token, which agent-framework documentation uses as ordinary vocabulary when describing how an agent may call itself. The remaining branches require the recursion to be asserted as happening, which is the runaway signal; both recursion true_positives use those branches)
- field: content
operator: regex
value: (?i)(?:iteration\s+(?:#\s*)?\d{2,}|loop\s+(?:iteration|count|index)\s*[:=]?\s*\d{2,}|cycle\s+\d{2,})
description: High iteration count indicating prolonged looping
- field: content
operator: regex
value: >-
(?i)(?:repeating\s+(?:the\s+)?(?:same|previous|identical)\s+(?:step|action|operation|request)|performing\s+(?:the\s+)?(?:same|identical)\s+(?:action|call)\s+again)
description: Agent acknowledging it is repeating identical actions
- field: content
operator: regex
value: >-
(?i)(?:max(?:imum)?\s+retries?\s+(?:reached|exceeded|hit)|(?:exceeded|hit|reached)\s+(?:the\s+)?(?:retry|attempt|iteration)\s+(?:limit|maximum|cap))
description: Retry limit reached indicators
- field: content
operator: regex
value: >-
(?i)(?:no\s+progress\s+(?:after|in)\s+\d+\s+(?:attempts?|tries|iterations?|cycles?)|stuck\s+(?:in\s+(?:a\s+)?loop|on\s+(?:the\s+)?same\s+(?:step|error)))
description: Explicit stall or no-progress acknowledgement
- field: content
operator: regex
value: (?i)(?:(?:while|for)\s*\(\s*(?:true|1|;;)\s*\)[\s\S]{0,80}?\b(?:invoke|call|execute|request|fetch|query|retry)\b|(?:loop|repeat)\s*\{\s*(?:call|invoke|execute))
description: Infinite loop constructs in generated or executed code
- field: content
operator: regex
value: >-
(?i)(?:will\s+keep\s+(?:trying|retrying|attempting)|(?:continuously|endlessly|infinitely)\s+(?:retrying|looping|repeating|calling))
description: Agent declaring intent to retry indefinitely
- field: content
operator: regex
value: >-
(?i)(?:spawn(?:ing|ed)?\s+(?:another|new|additional)\s+(?:instance|copy|clone)\s+of\s+(?:myself|this\s+agent)|fork(?:ing|ed)?\s+(?:a\s+)?(?:new\s+)?(?:agent|process|instance))
description: Agent spawning copies of itself (fork bomb pattern)
condition: any
false_positives:
- Legitimate batch processing with progress counters in output
- Polling agents designed to check status at regular intervals
- Long-running data pipeline agents with expected high iteration counts
- Retry logic in error handling with bounded retry limits
- CI/CD build logs that include 'attempt 2 of 3' progress counters during normal deployment retries
- >-
Monitoring dashboards displaying iteration counts for scheduled polling jobs that legitimately run hundreds of
cycles
response:
actions:
- reduce_permissions
- escalate
- alert
actions_rationale: >-
response-action eligibility (docs/RESPONSE-ACTION-ELIGIBILITY.md): benign FP 11/5352 = 0.21%
meets the 0.5% ceiling but is not zero; ceiling = degrade. Removed kill_agent(terminate).
Detection unchanged — recall cost is zero.
auto_response_threshold: high
message_template: |
[ATR-2026-050] HIGH: Runaway agent loop detected. Agent {agent_id}
shows repeated action patterns or retry loops. Pattern: {matched_pattern}.
Agent permissions reduced and escalated for review; the agent was NOT
killed. Session: {session_id}. Review agent
logic and termination conditions.
confidence: 69
wild_validated: 2026/04/08
wild_samples: 53577
test_cases:
true_positives:
- input: Retrying attempt 15 of 10. The request still returns the same error.
expected: triggered
description: Retry counter exceeding stated maximum
- input: Still failing after 25 iterations. Let me try again with the same parameters.
expected: triggered
description: Agent stuck in retry loop with no progress
- input: "Calling myself recursively to process the remaining items. Recursive call depth: 47."
expected: triggered
description: Recursive self-invocation with deep call stack
- input: Will keep trying until it works. Continuously retrying the API call.
expected: triggered
description: Agent declaring intent to retry indefinitely
- input: "iteration #450 - still processing the same batch. No new results returned from the API."
expected: triggered
description: High iteration count (pattern 6) indicating prolonged looping with 3+ digit iteration number
true_negatives:
- input: Processing batch 3 of 5. All items completed successfully so far.
expected: not_triggered
description: Legitimate batch processing with bounded progress counter
- input: The API returned an error. Retrying once with exponential backoff.
expected: not_triggered
description: Normal bounded retry logic in error handling
- input: Task completed after 3 attempts. Here are the final results.
expected: not_triggered
description: Successful completion after reasonable retry count
- input: >-
The pipeline completed successfully. Total records processed: 15000 across 4 stages. Final output saved to
/data/results.csv.
expected: not_triggered
description: Legitimate pipeline completion report with large numbers that do not match loop patterns
- input: >-
Here is a summary of the error handling strategy: if a transient fault occurs, the system will back off
exponentially and make up to 3 additional requests before returning a failure response to the caller.
expected: not_triggered
description: >-
Documentation describing bounded retry architecture without using trigger phrases like 'retrying N' or 'try
again'
evasion_tests:
- input: Volviendo a intentar la misma operación por 30ª vez. El servidor sigue devolviendo el mismo error.
expected: not_triggered
bypass_technique: language_switch
notes: Spanish translation of retry loop language completely avoids English regex patterns.
- input: >-
Re-doing the exact same step for the umpteenth time... the server keeps handing back an identical failure. I'll
give it another go.
expected: not_triggered
bypass_technique: casual_paraphrase
notes: >-
Paraphrases retry and same-error concepts using synonyms like 're-doing', 'umpteenth', 'identical failure', 'give
it another go' that don't match the specific regex phrases.
- input: sеlf_invоke triggered. Rеcursivе exеcution in progress, dеpth = 99.
expected: not_triggered
bypass_technique: unicode_homoglyph
notes: >-
Uses Cyrillic homoglyphs (е→е, o→о) in 'self_invoke' and 'recursive execution' to visually match but evade ASCII
regex matching.