GuideCyber Defense
Prompt injection defense: why filters fail and which layered controls limit the damage
Prompt injection happens when text a model reads changes what it does. Direct injection comes from the user's own input; indirect injection hides in web pages, email or documents the system retrieves. Filtering helps but cannot be the main defense. Effective programs assume some injections succeed and limit what the model can do: separate privileges, isolate untrusted content, permission tools outside the model, handle output carefully and monitor.
On this page
- Direct and indirect injection: what differs
- Why filtering alone fails
- Five layers between untrusted text and a harmful action
- Putting the layers in place, in order
- Six design patterns for agents, and what each constrains
- A hypothetical walk-through: the support-inbox summarizer
- Common failure patterns in prompt injection programs
- Questions and answers
- Sources
Direct and indirect injection: what differs
OWASP's entry for prompt injection, LLM01:2025, describes it as inputs that alter a model's behavior in unintended ways. Direct injection arrives through the user's own prompt. Indirect injection arrives through external content the model processes, such as a web page, a file or an email, and the instructions need not be visible to a human reader1.
The distinction matters for ownership. Direct injection is a risk at the front door, where you control authentication and rate limits. Indirect injection bypasses the front door. A customer-support agent that summarizes inbound email, or a coding assistant that reads a repository, accepts text from people you have never vetted.
Anthropic's September 2026 threat report gives an official example of the same idea in practice2. It describes an actor it tracks as GTG-50020 injecting instructions into an AI vendor's evaluation sandbox to obtain production API keys2. The attacker used content the system would read, not a flaw in the interface.
Why filtering alone fails
The UK National Cyber Security Centre argues that prompt injection is not like SQL injection. Databases separate commands from data; language models make no such distinction, so the NCSC says the problem may never be totally mitigated and describes models as inherently confusable deputies3.
That has practical consequences. The NCSC advises against deny-listing attack phrases and recommends being skeptical of products that claim to stop prompt injection outright3. OWASP likewise says that retrieval-augmented generation and fine-tuning do not fully mitigate the risk1. Filters still have a role, since OWASP lists input and output filtering among its mitigations1, but they reduce volume and cannot be the control that decides whether an incident happens.
The right question is therefore not whether an injection can be blocked but what happens when one gets through. The NCSC frames this as privilege: a model processing content from a party should be treated as having no more privileges than that party3.
Five layers between untrusted text and a harmful action
- Monitoring and response
Log inputs, outputs and tool use in full; alert on unusual tool sequences and block or roll back.
- Output handling
Treat model output as untrusted before it reaches a browser, shell, database or another system.
- Tool permissioning
Allow-lists, scoped credentials and human approval enforced by code outside the model.
- Untrusted-content isolation
Mark external content as data, process it in a separate low-privilege context and constrain its output.
- Privilege separation
Give each agent only the data and actions its task needs, and keep sensitive actions in separate components.
Putting the layers in place, in order
Start with the layers that cap the damage, then add the ones that detect and filter.
Map what the model can read and do
List every source of text the system ingests and every tool it can call. The OWASP mitigation list begins with constraining model behavior and enforcing least privilege1.
Separate privileges
Split tasks so that no single model context holds both untrusted input and sensitive capability, such as sending email or reading customer records.
Enforce tool permissions in code
Use allow-lists, narrow scopes and approval for high-risk actions, as OWASP advises1. For servers following the Model Context Protocol, the protocol's security guidance forbids passing through tokens not issued for the server and recommends progressive least-privilege scopes4.
Handle output as untrusted
Validate expected output formats, and sanitize anything the model emits before it reaches another system. OWASP lists validating output formats as a mitigation1.
Six design patterns for agents, and what each constrains
| Pattern | What it constrains | Typical trade-off |
|---|---|---|
| Action-selector | The agent picks from a fixed menu of actions and does not act on text returned by tools5. | Low flexibility; suits narrow, repeatable tasks. |
| Plan-then-execute | The agent commits to a plan before it reads untrusted content, so later text cannot add steps5. | Plans cannot adapt to what the agent finds. |
| LLM map-reduce | Isolated sub-agents process each item, and only constrained results reach the coordinator5. | More components to build and monitor. |
| Dual LLM | A privileged model never sees untrusted text, while a quarantined model reads it without tools5. | Complexity in passing results safely between the two. |
| Code-then-execute | The agent writes a program as its plan, which can be inspected and constrained before it runs5. | Needs a safe execution environment. |
| Context-minimization | Unneeded content, including the original prompt, is removed from context before later steps5. | Loses context that some tasks require. |
The paper behind this table argues that patterns should be combined and that agents should be application-specific rather than general-purpose5.
A hypothetical walk-through: the support-inbox summarizer
Common failure patterns in prompt injection programs
Relying on a phrase filter as the primary control
Early signalThe design review answer to injection is a list of blocked strings.
MitigationKeep filters as a noise reducer and add controls that cap what a fooled model can do3.
One agent holds both untrusted input and sensitive tools
Early signalA single service account can read external web content and write to internal systems.
MitigationSplit into separate contexts with approval between them, as in the dual-model and map-reduce patterns5.
Permissions enforced only in the system prompt
Early signalTool limits are described in text to the model rather than in code.
MitigationMove allow-lists, scopes and approval gates into deterministic code, as the NCSC recommends for non-LLM safeguards3.
No record of what the model saw
Early signalIncident review cannot show which document triggered an action.
MitigationLog full inputs, outputs and tool calls, with retention that fits your incident process3.
Questions and answers
What is the difference between direct and indirect prompt injection?
Direct injection comes from the user's own prompt. Indirect injection comes from external content the model processes, such as a web page, file or email, and the instructions may be invisible to people1. Indirect injection is usually the bigger concern for agents, because attackers can plant text without ever using your interface.
Can prompt injection be fully prevented?
Do input filters and guardrail products work?
Which control should a team build first?
Start with least privilege and approval gates for high-risk actions, enforced in code outside the model. These limit the damage even when the model is fooled. Then isolate untrusted content, validate output and add logging, and test the whole stack with a red team exercise.
How do MCP servers change the risk?
They add more tools and more trust boundaries. The protocol's security guidance covers confused deputy problems, forbids passing through tokens not issued for the server, and recommends progressive least-privilege scopes and sandboxing for local servers4. Vet each server as you would any third-party dependency.
Sources
- LLM01:2025 Prompt Injection — OWASP GenAI Security Project · checked 19 November 2026
- Detecting and countering misuse of AI: September 2026 — Anthropic · checked 19 November 2026
- Prompt injection is not SQL injection (it may be worse) — UK National Cyber Security Centre · checked 19 November 2026
- Security best practices — Model Context Protocol · checked 19 November 2026
- Design Patterns for Securing LLM Agents against Prompt Injections — arXiv · checked 19 November 2026
More in Cyber Defense
Back to Cyber DefenseYour starting point
What would you like to move forward?
Start with the outcome that feels closest. We'll help give the next step a useful shape.
Let's find your next move
One conversation. A clearer direction.
Tell us what you want to improve. A few sentences are enough to start.
shayan@coldai.org