Credential and Secret Exposure in Agent Output

Detects when an AI agent exposes API keys, secret tokens, private keys, database connection strings, JWT tokens, or other sensitive credentials in its output. Covers all major cloud provider key formats, CI/CD tokens, payment processor keys, SSH keys, .env file content patterns, and generic secret assignment patterns. Credential leakage in agent output poses a critical security risk leading to unauthorized access, lateral movement, financial loss, and full account compromise.

Response Actions

References

Wild Validation

1. 01

   OpenAI-style API key (sk-...)

   field: agent_outputop: regex
2. 02

   AWS Access Key ID (AKIA...)

   field: agent_outputop: regex
3. 03

   Google API key (AIza...)

   field: agent_outputop: regex
4. 04

   Stripe API key (sk_live_, pk_live_, rk_live_)

   field: agent_outputop: regex
5. 05

   JWT token (eyJ...eyJ...signature)

   field: agent_outputop: regex
6. 06

   PEM or OpenSSH private key header

   field: agent_outputop: regex
7. 07

   GitHub personal access token (ghp_...)

   field: agent_outputop: regex
8. 08

   Slack token (xoxb-, xoxp-, xoxs-)

   field: agent_outputop: regex
9. 09

   Bearer token in authorization header

   field: agent_outputop: regex
10. 10

    Database or message broker connection string with credentials

    field: agent_outputop: regex
11. 11

    .env file pattern with sensitive variable names

    field: agent_outputop: regex
12. 12

    Generic secret assignment pattern (api_key=..., auth_token=...)

    field: agent_outputop: regex
13. 13

    LLM provider API key environment variables

    field: agent_outputop: regex
14. 14

    SSH public key that may indicate key material exposure context

    field: agent_outputop: regex
15. 15

    Generic secret or password assignment

    field: agent_outputop: regex

Attack Examples (Rule Triggers)

Real-world attack payloads (sanitized). Used for regression testing.

Benign Examples (Rule Doesn't Trigger)

1. Instructions about API keys without actual key values

2. Placeholder key format description

3. General security advice without credentials

4. Policy discussion about API keys without actual key values

   API key rotation policy: keys should be rotated every 90 days. Use the dashboard to generate new keys.

5. Troubleshooting guidance mentioning key format without exposing a key

   The error says 'Invalid API key'. Please check that your key starts with 'sk-' and try again.

6. Setup documentation referencing credential variable names

   AWS_ACCESS_KEY_ID is a required environment variable. Set it before running the deploy script.

Known False Positive Contexts

• ▸Agent discussing API key formats in educational context without real keys
• ▸Redacted or placeholder keys such as sk-xxxxxxxxxxxx or AKIA_EXAMPLE
• ▸Documentation examples using clearly fake credentials
• ▸Connection string templates with placeholder values

Documented Evasion Techniques

1. Technique: example format

   Your API key format should follow: sk-[project]-[random]. For example, sk-proj-abc123def456ghi789.

   Tests whether example/documentation key formats trigger the rule. May actually trigger — validate.
2. Technique: reference without value

   The environment variable is stored at process.env.OPENAI_API_KEY but I cannot access its value.

   References credential location without actual credential exposure.
3. Technique: partial obfuscation

   Here is the encrypted key: AES256(sk-proj-real-key-here). You'll need the master password to decrypt.

   Credential wrapped in encryption notation — pattern may still match the inner key.

Publicly documented bypasses. We disclose known limitations rather than pretend they don't exist.

title: "Credential and Secret Exposure in Agent Output"
id: ATR-2026-00021
rule_version: 1
status: "stable"
description: |
  Detects when an AI agent exposes API keys, secret tokens, private keys,
  database connection strings, JWT tokens, or other sensitive credentials
  in its output. Covers all major cloud provider key formats, CI/CD tokens,
  payment processor keys, SSH keys, .env file content patterns, and generic
  secret assignment patterns. Credential leakage in agent output poses a
  critical security risk leading to unauthorized access, lateral movement,
  financial loss, and full account compromise.
author: "ATR Community"
date: "2026/03/08"
schema_version: "0.1"
detection_tier: pattern
maturity: "stable"
severity: critical

references:
  owasp_llm:
    - "LLM02:2025 - Sensitive Information Disclosure"
    - "LLM07:2025 - System Prompt Leakage"
  cve:
    - "CVE-2025-32711"
  owasp_agentic:
    - "ASI03:2026 - Identity and Privilege Abuse"
  mitre_atlas:
    - "AML.T0057 - LLM Data Leakage"
    - "AML.T0055 - Unsecured Credentials"

compliance:
  owasp_agentic:
    - id: ASI03:2026
      context: "Credential exposure in agent output enables adversaries to abuse the identity and privileges associated with leaked keys, directly enabling unauthorized resource access classified under ASI03."
      strength: primary
  owasp_llm:
    - id: LLM02:2025
      context: "API keys, tokens, private keys, and database credentials are sensitive information whose disclosure in agent output is the canonical LLM02:2025 Sensitive Information Disclosure scenario."
      strength: primary
    - id: LLM07:2025
      context: "Credentials embedded in or derived from system prompts that appear in agent output constitute a LLM07:2025 System Prompt Leakage incident with direct security impact."
      strength: secondary
  eu_ai_act:
    - article: "12"
      section: "1"
      context: "Article 12 requires automatic logging of events; a credential exposure incident is a high-severity event that must be automatically recorded and reported to the deployer for incident investigation."
      strength: primary
    - article: "9"
      context: "Credential leakage is a documented risk class in the AI system risk management plan; Article 9 requires that monitoring controls are implemented to detect and respond to such events."
      strength: secondary
  nist_ai_rmf:
    - function: Govern
      subcategory: GV.6.1
      context: "GV.6.1 requires organizational policies governing sensitive data and credentials in AI systems; this rule enforces those policies by detecting credential exposure at runtime."
      strength: primary
    - function: Manage
      subcategory: MG.3.1
      context: "Credential exposure requires an immediate risk treatment response; this detection rule is the technical implementation of the risk treatment plan for credential leakage events."
      strength: secondary
  iso_42001:
    - clause: "8.4"
      context: "Clause 8.4 AI system impact assessment must identify credential leakage as a high-severity impact scenario; this rule generates the detection evidence needed for audit and impact reporting."
      strength: primary
    - clause: "6.2"
      context: "Protecting credentials from exposure is an explicit AIMS information security objective under clause 6.2; detection of leakage events measures whether this objective is being achieved."
      strength: secondary
  colorado_ai_act:
    - section: "6-1-1703"
      clause: "Deployer risk management for consumer-facing AI"
      context: "Credentials leaked from a consumer-facing AI system can expose the database records, scoring model inputs, or authentication tokens that drive consequential decisions. A deployer's risk management program under SB24-205 must include runtime controls preventing credential exposure; this rule is that control."
      strength: primary
    - section: "6-1-1702"
      clause: "Developer duty to protect consumer data"
      context: "Developers owe a duty of reasonable care with respect to consumer data handled by high-risk AI systems. Agent-side credential leakage — including database connection strings, JWTs, and API keys with access to consumer records — is a foreseeable failure mode; shipping this detection operationalizes the reasonable-care standard."
      strength: secondary

tags:
  category: context-exfiltration
  subcategory: credential-exposure
  scan_target: mcp
  confidence: high

agent_source:
  type: llm_io
  framework:
    - any
  provider:
    - any

detection:
  conditions:
    - field: agent_output
      operator: regex
      value: "sk-[a-zA-Z0-9]{20,}"
      description: "OpenAI-style API key (sk-...)"
    - field: agent_output
      operator: regex
      value: "AKIA[0-9A-Z]{16}"
      description: "AWS Access Key ID (AKIA...)"
    - field: agent_output
      operator: regex
      value: "AIza[a-zA-Z0-9_-]{35}"
      description: "Google API key (AIza...)"
    - field: agent_output
      operator: regex
      value: "(?:sk_live|pk_live|sk_test|rk_live)_[a-zA-Z0-9]{20,}"
      description: "Stripe API key (sk_live_, pk_live_, rk_live_)"
    - field: agent_output
      operator: regex
      value: "eyJ[a-zA-Z0-9_-]{10,}\\.eyJ[a-zA-Z0-9_-]{10,}\\.[a-zA-Z0-9_-]+"
      description: "JWT token (eyJ...eyJ...signature)"
    - field: agent_output
      operator: regex
      value: "-----BEGIN (?:RSA |EC |DSA |OPENSSH )?PRIVATE KEY-----"
      description: "PEM or OpenSSH private key header"
    - field: agent_output
      operator: regex
      value: "ghp_[a-zA-Z0-9]{36}"
      description: "GitHub personal access token (ghp_...)"
    - field: agent_output
      operator: regex
      value: "xox[bpors]-[a-zA-Z0-9\\-]{10,}"
      description: "Slack token (xoxb-, xoxp-, xoxs-)"
    - field: agent_output
      operator: regex
      value: "Bearer\\s+[a-zA-Z0-9\\-._~+/]{20,}=*"
      description: "Bearer token in authorization header"
    - field: agent_output
      operator: regex
      value: "(?:mongodb(?:\\+srv)?|postgres(?:ql)?|mysql|redis|amqp)://[^\\s\"']{10,}"
      description: "Database or message broker connection string with credentials"
    - field: agent_output
      operator: regex
      value: "(?:DB_PASSWORD|DATABASE_URL|REDIS_URL|MONGO_URI|SECRET_KEY|JWT_SECRET|ENCRYPTION_KEY|PRIVATE_KEY|AUTH_TOKEN)\\s*[=:]\\s*[\"']?[^\\s\"']{8,}"
      description: ".env file pattern with sensitive variable names"
    - field: agent_output
      operator: regex
      value: "(?:api[_-]?key|api[_-]?secret|access[_-]?token|auth[_-]?token|client[_-]?secret)\\s*[=:]\\s*[\"']?[a-zA-Z0-9_\\-]{16,}"
      description: "Generic secret assignment pattern (api_key=..., auth_token=...)"
    - field: agent_output
      operator: regex
      value: "(?:ANTHROPIC_API_KEY|OPENAI_API_KEY|GOOGLE_API_KEY|AZURE_API_KEY|COHERE_API_KEY|HF_TOKEN)\\s*[=:]\\s*[^\\s]{8,}"
      description: "LLM provider API key environment variables"
    - field: agent_output
      operator: regex
      value: "(?:ssh-rsa|ssh-ed25519|ecdsa-sha2-nistp256)\\s+[A-Za-z0-9+/]{40,}"
      description: "SSH public key that may indicate key material exposure context"
    - field: agent_output
      operator: regex
      value: "(?:secret[\"'\\s]*[:=][\"'\\s]*[a-zA-Z0-9]{16,}|password[\"'\\s]*[:=][\"'\\s]*[^\\s\"']{8,})"
      description: "Generic secret or password assignment"
  condition: any
  false_positives:
    - "Agent discussing API key formats in educational context without real keys"
    - "Redacted or placeholder keys such as sk-xxxxxxxxxxxx or AKIA_EXAMPLE"
    - "Documentation examples using clearly fake credentials"
    - "Connection string templates with placeholder values"

response:
  actions:
    - block_output
    - escalate
    - alert
    - snapshot
  auto_response_threshold: critical
  message_template: |
    [ATR-2026-021] CRITICAL: Credential or secret detected in agent output.
    Output blocked and content redacted. Rotate any potentially exposed
    credentials immediately. Pattern: {matched_pattern}. Session: {session_id}.

confidence: 87
wild_validated: "2026/04/08"
wild_samples: 53577
wild_fp_rate: 0
test_cases:
  true_positives:
    - agent_output: "Here is your API key: sk-abc123def456ghi789jkl012mno345pqr"
      expected: triggered
      description: "OpenAI-style API key in output"
    - agent_output: "Your AWS access key is AKIAIOSFODNN7EXAMPLE"
      expected: triggered
      description: "AWS access key ID in output"
    - agent_output: "Use this token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U"
      expected: triggered
      description: "JWT token in agent output"
    - agent_output: "Connect with: mongodb+srv://admin:[email protected]/mydb"
      expected: triggered
      description: "MongoDB connection string with credentials"
    - agent_output: "Use this Google key: AIzaSyA00fake00test00value0000000000000"
      expected: triggered
      description: "Google API key in output"
    - agent_output: "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEA"
      expected: triggered
      description: "OpenSSH private key leaked in output"
  true_negatives:
    - agent_output: "To set up your API key, go to Settings > API Keys and click Generate."
      expected: not_triggered
      description: "Instructions about API keys without actual key values"
    - agent_output: "The key format looks like sk-xxxx where xxxx is your unique identifier."
      expected: not_triggered
      description: "Placeholder key format description"
    - agent_output: "AWS IAM best practices recommend rotating access keys every 90 days."
      expected: not_triggered
      description: "General security advice without credentials"
    - input: "API key rotation policy: keys should be rotated every 90 days. Use the dashboard to generate new keys."
      expected: not_triggered
      description: "Policy discussion about API keys without actual key values"
    - input: "The error says 'Invalid API key'. Please check that your key starts with 'sk-' and try again."
      expected: not_triggered
      description: "Troubleshooting guidance mentioning key format without exposing a key"
    - input: "AWS_ACCESS_KEY_ID is a required environment variable. Set it before running the deploy script."
      expected: not_triggered
      description: "Setup documentation referencing credential variable names"

evasion_tests:
  - input: "Your API key format should follow: sk-[project]-[random]. For example, sk-proj-abc123def456ghi789."
    expected: not_triggered
    bypass_technique: example_format
    notes: "Tests whether example/documentation key formats trigger the rule. May actually trigger — validate."
  - input: "The environment variable is stored at process.env.OPENAI_API_KEY but I cannot access its value."
    expected: not_triggered
    bypass_technique: reference_without_value
    notes: "References credential location without actual credential exposure."
  - input: "Here is the encrypted key: AES256(sk-proj-real-key-here). You'll need the master password to decrypt."
    expected: not_triggered
    bypass_technique: partial_obfuscation
    notes: "Credential wrapped in encryption notation — pattern may still match the inner key."