“The Model Will Refuse” Is Not a Control: A Threat Taxonomy for AI and Agentic Systems

TL;DR

• I have published the AI and Agentic Systems Threat Taxonomy, the reference for my Elevation of Autonomy card deck: 30 threats in five suits.
• Every entry keeps the card’s threat and mitigation prompts, and adds a mitigation principle, design measures, mappings to STRIDE, OWASP, MITRE ATLAS, CAPEC and CWE, and, for 18 entries, a public case or the research behind the card.
• The IDs are the ones printed on the cards (EOA·SA for Prompt Injection, EOA·T7 for Adversarial Subspace). They will not change.
• One idea runs through all 30 entries: a security guarantee for an AI system has to live in a deterministic control outside the model.
• It is free to use for threat modeling, including for paying clients, and comes as web pages, PDF, Excel, SKOS and STIX 2.1: ostering.com/taxonomies/ai-agentic-systems/

Earlier this year, in edition 4 of The Blast Radius, I introduced Elevation of Autonomy: a card deck that takes the card-based threat modeling tradition of Adam Shostack’s Elevation of Privilege (EoP) into AI, large language model (LLM) and agentic systems. Thirty cards, five suits, and a reference document, runbook and template to go with them.

A card does two jobs well. It names a threat in a way a whole team can remember, and it asks the questions that make the team look at their own design. What it can’t do, on a card the size of your palm, is tell you what good looks like once you’ve found the problem.

That’s the gap this taxonomy fills. In this post I’ll explain what’s in it, why it is built the way it is, and how to use it.

Why another list?

At first glance the space looks well covered. We have the OWASP Top 10 for LLM Applications, the OWASP Top 10 for Agentic Applications, MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems), and the old faithfuls, CAPEC (Common Attack Pattern Enumeration and Classification) and CWE (Common Weakness Enumeration).

They’re all useful, and the taxonomy maps to every one of them. But none of them is a design-time reference you can sit in front of a data-flow diagram with:

• The OWASP lists are risk lists. They tell you what goes wrong most often. They don’t tell you what a sound design looks like, and they assume the model’s behaviour is part of the defence.
• ATLAS describes the adversary. Tactics and techniques, with case studies. Brilliant for sequencing an attack path; less so for deciding what to change in your architecture.
• CAPEC and CWE were written for code. 12 of the 30 entries have no good CAPEC or CWE at all, because threats like context rot or excessive autonomy by design are properties of the system and the product, not of a line of code.
• None of them has the structural hazards. The mathematical and architectural properties that make most “the model will refuse” defences weaker than they look aren’t in any of the lists.

So I wanted something keyed to the cards a team has just played, that says what good looks like, and that points back to the frameworks people already report against.

What an entry contains

Each of the 30 entries has the same parts:

• The card: its ID, title and quote, exactly as printed.
• Threat: what goes wrong, from the card.
• Mitigation prompts: the questions the card asks the table, also from the card. They’re the quickest way into a design review, so they’re in the taxonomy too.
• Mitigation principle: one sentence on what a sound design does about the threat. This is the test. If your design meets the principle, you have no finding for that card.
• Design measures: three or four concrete measures that put the principle into practice. They are the shape of a fix, not a prescription; adapt them to your architecture.
• Mappings: STRIDE (and LINDDUN for the privacy cards), the OWASP and other references printed on the card (with their OWASP LLM Top 10 2026 equivalents, because the 2026 edition renumbered the list), MITRE ATLAS techniques, CAPEC and CWE.
• Seen in the wild: a public case or the research behind the card, with its source.
• Touches: the cards most often played alongside it.

The split between principle and measures matters. There’s rarely one right fix for an AI system, because the architectures vary so much. The principle is the stable part; the measures are examples of how to get there. This is the same split I used in Threat Modeling Gameplay with EoP: the threats are the stable layer, and the mitigations live alongside them.

The IDs don’t change

Every entry uses the ID printed on its card: EOA·SA is the Ace of Spades, Prompt Injection; EOA·H9 is the 9 of Hearts, Confused Deputy Across Servers; EOA·T7 is Trump VII, Adversarial Subspace. That’s the deck prefix, a middle dot, the suit letter and the card’s value.

The deck is published and the cards are printed, so these IDs are fixed. Put them in your threat models and findings next to the OWASP and ATLAS identifiers, and anyone can come back to the full entry.

One note on rank. In the game, high ranks (Ace, King) are the broader or more foundational threats and low ranks the more specific or variant ones. In the taxonomy the ID is a reference, not a severity. Rate each finding on its merits in your system.

The five suits

The taxonomy has a page per suit, each with every entry in full.

• ♠ Spades: Adversarial (6 entries). Direct attacks on the model and the agent: Prompt Injection, Tool Misuse, Supply Chain Compromise, Memory Poisoning, Improper Output Handling and Tool Description Injection.
• ♥ Hearts: Autonomy (6 entries). The risks that arrive when the system is allowed to act: Excessive Agency, Identity and Privilege Abuse, Inter-Agent Trust Exploitation, Cascading Failure, Rogue Agent and Confused Deputy Across Servers.
• ♦ Diamonds: Data (6 entries). Risks to and through the data: Sensitive Information Disclosure, Vector and Embedding Weakness, Hallucinated Facts, Unbounded Consumption, System Prompt Leakage and Server Impersonation and Rogue Servers.
• ♣ Clubs: Privacy (5 entries). From T.R.I.M. and LINDDUN, framed for systems that generate personal data as well as process it: Transfer, Retention and Removal, Inference, Minimisation and Unintervenability.
• ★ Trumps: Structural Hazards (7 entries). Properties that can’t be mitigated inside the model: Adversarial Subspace, Geometric Attack, Context Rot, Decision Boundary Transfer, Excessive Autonomy by Design, Invisible Dependency and Wrong Abstraction.

The idea that runs through all 30

If you read the principles one after another, you’ll notice they keep saying the same thing in different ways: put the guarantee somewhere the model can’t talk its way out of.

The Trumps explain why. Any input to a model is flattened into a lower-dimensional representation, so for any behaviour you want to block there is an enormous space of inputs that produce it, not a list (EOA·T7 Adversarial Subspace). Models trained on the same problem carve out roughly the same decision boundary, so an attacker can find a way across on a cheap surrogate model and never touch yours (EOA·T4 Decision Boundary Transfer). And even with no attacker at all, the safety instructions at the top of a long session lose their grip as the context grows (EOA·T5 Context Rot).

Put those together and the conclusion is uncomfortable but simple. Guardrails, refusals, blocklists and classifiers shift the average. They’re worth having. They are never the control. The control is deterministic and outside the model: authorisation at the data layer, allow-listed tool arguments, human confirmation for irreversible actions, budgets and rate limits, pinned and inventoried dependencies.

That’s why the design measures for Prompt Injection (EOA·SA) start with delimiting the sources in the context, but then move straight on to enforcing what the agent may do outside the model, “so an obeyed injection is still refused”. Assume the injection will work sometimes. Design so that it doesn’t matter when it does.

Note to teams building agents: for every consequential action your agent can take, write down the deterministic control that decides it. If the answer ends in “the model will refuse” or “the classifier will catch it”, you have found your first finding.

Seen in the wild

None of this is theoretical. 18 of the 30 entries cite a public case or the research behind the card, each with its source on the suit page. A few of them:

• EchoLeak (CVE-2025-32711): a zero-click indirect prompt injection in Microsoft 365 Copilot, disclosed by Aim Security (EOA·SA).
• Tool poisoning attacks: Invariant Labs showed MCP (Model Context Protocol) tool descriptions carrying hidden instructions, including a “rug pull” that changes the description after approval (EOA·S9).
• The WhatsApp MCP exploit: a malicious MCP server, connected alongside a trusted one in the same client, used its tool description to send the user’s message history to an attacker (EOA·H9).
• postmark-mcp: an npm package posing as an email MCP server quietly BCC’d every message to an attacker-controlled address (EOA·D9).
• Amazon Q Developer for VS Code: a release shipped with an injected prompt telling the agent to wipe local files and cloud resources (EOA·SQ).
• Slopsquatting: models repeatedly recommended a package that didn’t exist, and a placeholder uploaded under that name was downloaded tens of thousands of times (EOA·DQ).
• Moffatt v. Air Canada: a tribunal held the airline liable for what its chatbot told a customer about its fares (EOA·T1).

The cases are there to show that each threat is real, not to name and shame anyone.

How to use it

• After a game. Each trick the table records names a card. Look the ID up, use the principle to decide whether the design already defeats the threat, and use the measures to write the fix.
• Without the cards. Walk all 30 entries against every AI or agentic element in your data-flow diagram. Each one produces a finding or a one-line “no exposure” with the reason. Play the Trumps against the design as a whole.
• In reports. Tag findings with the ID next to the OWASP and ATLAS identifiers.
• In tools. The SKOS and STIX 2.1 downloads carry the same IDs, so the taxonomy can go into a knowledge base, a threat modeling tool or a threat intelligence platform.

Note to tool vendors: if you’d like to ship the taxonomy inside a product, as a built-in library or similar, talk to me first. Using it to threat model, including for your clients, needs no permission at all.

Scope and origin

The game came first. The cards were published with a reference document, a facilitator’s runbook and a threat model template in the Elevation of Autonomy repository, and you can print and play them from there or buy the printed deck from CyberSec Games. This taxonomy is the reference for that deck: the card text is unchanged, and the principles, measures, mappings and cases build on the mitigations published with it.

It covers the AI-specific threat surface: models, prompts, retrieval, memory, tools, MCP servers and agents. The pipeline that builds and ships an AI system is covered by the Software Factory Threat Taxonomy, AI in the mail path by the Email Infrastructure Threat Taxonomy, and voice assistants in the home by the Smart Home Threat Taxonomy.

Credits

The deck stands on other people’s work, and so does the taxonomy:

• Adam Shostack, for Elevation of Privilege, the game that started all this.
• The OWASP GenAI Security Project, for the Top 10 for LLM Applications and the Top 10 for Agentic Applications.
• Disesdi Shoshana Cox, whose work inspired the Adversarial Subspace, Geometric Attack and Decision Boundary Transfer cards, and whose “design-time over runtime” framing runs through every principle.
• Mark Vinkovits for his EoP Privacy Extension, Marko Hämäläinen, Laura Noukka, Hiski Ruhanen, Ilona Varis and Antti Vähä-Sipilä for Elevation of Privacy and T.R.I.M., and Kim Wuyts for LINDDUN GO, which shaped the privacy suit.
• MITRE, for ATLAS, CAPEC and CWE.

Resources

• Overview and suit pages: ostering.com/taxonomies/ai-agentic-systems/
• PDF: ai-agentic-systems-threat-taxonomy.pdf, landscape, a page per entry
• Excel: ai-agentic-systems-threat-taxonomy.xlsx
• SKOS (JSON-LD): ai-agentic-systems-threat-taxonomy.skos.jsonld
• STIX 2.1: ai-agentic-systems-threat-taxonomy.stix.json
• The mapping in detail: Mapping the AI and Agentic Systems Threat Taxonomy to STRIDE, OWASP, MITRE ATLAS, CAPEC and CWE

Licence

© 2026 Jonathan Brett Crawley. This taxonomy is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International licence (CC BY-NC-ND 4.0), with the additional permissions below. You can share the whole taxonomy freely for non-commercial purposes, unchanged and with credit.

Additional permissions. In addition to the rights the licence gives, and whether or not the use is commercial, the copyright holder grants you permission to:

• use the taxonomy to threat model systems you build, operate or assess, including for paying clients;
• use it internally, and for training, inside any organisation;
• quote individual entries, with credit, in paid courses, reports, findings and talks.

Any other commercial use, including shipping the taxonomy or a substantial part of it in a product (a tool’s built-in library, a printed deck, a reprint for sale), needs written permission. Ask me on LinkedIn.

Credit: “AI and Agentic Systems Threat Taxonomy” by Jonathan Brett Crawley, ostering.com, licensed under CC BY-NC-ND 4.0.

These additional permissions are given alongside the licence. They add to what the licence allows and do not take anything away from it.

The card text itself (each card’s threat, quote, mitigation prompts and references) is also published in the Elevation of Autonomy repository under CC BY-SA 4.0: https://github.com/BrettCrawley/elevation-of-autonomy. That licence still applies to the card text taken from the repository. The mitigation principles, design measures, mappings and cases are new in this taxonomy and are covered by the licence above.

Final thoughts

I built Elevation of Autonomy because the advice for threat modeling AI systems ranged from the abstract to the theatrical, and teams in design reviews needed something in between. The taxonomy is the other half of that: once the card is on the table, here’s what good looks like.

It’s version 1.0, and it will grow. The mapping work has already shown me two gaps in the deck, which I write about in the mapping post. If you use it in a review and find a threat it misses, or a measure that doesn’t hold up, tell me.

Stay tuned!

References

1. Crawley, B. (2026) Elevation of Autonomy card deck, reference document, runbook and template. https://github.com/BrettCrawley/elevation-of-autonomy
2. Crawley, B. (2026) AI and Agentic Systems Threat Taxonomy, v1.0. https://ostering.com/taxonomies/ai-agentic-systems/
3. Shostack, A. Elevation of Privilege card game. https://shostack.org/games/elevation-of-privilege
4. OWASP GenAI Security Project. Top 10 for LLM Applications (2025 and 2026 editions) and Top 10 for Agentic Applications (2026). https://genai.owasp.org/
5. MITRE. ATLAS (2026.10). https://atlas.mitre.org/
6. Cox, D. S. and Bunzel, N. (2025) Quantifying the Risk of Transferred Black Box Attacks. https://arxiv.org/abs/2511.05102
7. LINDDUN privacy threat modeling. https://linddun.org/
8. EOA·SA: Zero-click AI data leak flaw uncovered in Microsoft 365 Copilot (BleepingComputer) (June 2025). https://bleepingcomputer.com/news/security/zero-click-ai-data-leak-flaw-uncovered-in-microsoft-365-copilot
9. EOA·SK: GitHub MCP Exploited: Accessing private repositories via MCP (Invariant Labs) (May 2025). https://invariantlabs.ai/blog/mcp-github-vulnerability
10. EOA·SQ: AWS-2025-015: Security Update for Amazon Q Developer Extension for Visual Studio Code (Version #1.84) (July 2025). https://aws.amazon.com/security/security-bulletins/rss/aws-2025-015
11. EOA·SJ: Spyware Injection Into Your ChatGPT’s Long-Term Memory (SpAIware) (Embrace The Red) (September 2024). https://embracethered.com/blog/posts/2024/chatgpt-macos-app-persistent-data-exfiltration
12. EOA·S10: Hacking Google Bard: From Prompt Injection to Data Exfiltration (Embrace The Red) (November 2023). https://embracethered.com/blog/posts/2023/google-bard-data-exfiltration
13. EOA·S9: MCP Security Notification: Tool Poisoning Attacks (Invariant Labs) (April 2025). https://invariantlabs.ai/blog/mcp-security-notification
14. EOA·HA: Replit makes vibe-y promise to stop its AI agents making vibe coding disasters (The Register) (July 2025). https://www.theregister.com/2025/07/22/replit_saastr_response/
15. EOA·H9: WhatsApp MCP Exploited: Exfiltrating your message history via MCP (Invariant Labs) (April 2025). https://invariantlabs.ai/blog/whatsapp-mcp-exploited
16. EOA·DA: Data exfiltration from Slack AI via indirect prompt injection (PromptArmor) (August 2024). https://promptarmor.com/resources/data-exfiltration-from-slack-ai-via-indirect-prompt-injection
17. EOA·DQ: Hackers take advantage of AI hallucinations to sneak malicious software packages onto enterprise repositories (ITPro) (March 2024). https://www.itpro.com/security/hackers-are-taking-advantage-of-ai-hallucinations-to-sneak-malicious-software-packages-onto-enterprise-repositories
18. EOA·DJ: LLMjacking: Stolen Cloud Credentials Used in New AI Attack (Sysdig) (May 2024). https://www.sysdig.com/blog/llmjacking-stolen-cloud-credentials-used-in-new-ai-attack
19. EOA·D10: Microsoft’s Bing chatbot AI is susceptible to several types of prompt injection attacks (TechSpot) (February 2023). https://www.techspot.com/news/97590-microsoft-bing-chatbot-ai-susceptible-several-types-prompt.html
20. EOA·D9: Postmark MCP server code hijacked (The Register) (September 2025). https://www.theregister.com/2025/09/29/postmark_mcp_server_code_hijacked/
21. EOA·C10: OpenAI faces European privacy complaint after ChatGPT allegedly hallucinated man murdered his sons (Euronews) (March 2025). https://www.euronews.com/2025/03/20/openai-faces-european-privacy-complaint-after-chatgpt-allegedly-hallucinated-man-murdered-
22. EOA·T7: EchoGram: The Hidden Vulnerability Undermining AI Guardrails (HiddenLayer) (November 2025). https://www.hiddenlayer.com/research/echogram-the-hidden-vulnerability-undermining-ai-guardrails
23. EOA·T5: Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma) (July 2025). https://www.trychroma.com/research/context-rot
24. EOA·T4: The Space of Transferable Adversarial Examples (arXiv:1704.03453) (April 2017). https://arxiv.org/abs/1704.03453
25. EOA·T1: Moffatt v. Air Canada, 2024 BCCRT 149 (Civil Resolution Tribunal) (February 2024). https://decisions.civilresolutionbc.ca/crt/crtd/en/525448/1/document.do

Tags: Threat Modeling, AI Security