GuideCyber Defense

Prompt injection defense: why filters fail and which layered controls limit the damage

Prompt injection happens when text a model reads changes what it does. Direct injection comes from the user's own input; indirect injection hides in web pages, email or documents the system retrieves. Filtering helps but cannot be the main defense. Effective programs assume some injections succeed and limit what the model can do: separate privileges, isolate untrusted content, permission tools outside the model, handle output carefully and monitor.

Reviewed 7 min read

On this page
  1. Direct and indirect injection: what differs
  2. Why filtering alone fails
  3. Five layers between untrusted text and a harmful action
  4. Putting the layers in place, in order
  5. Six design patterns for agents, and what each constrains
  6. A hypothetical walk-through: the support-inbox summarizer
  7. Common failure patterns in prompt injection programs
  8. Questions and answers
  9. Sources

Direct and indirect injection: what differs

OWASP's entry for prompt injection, LLM01:2025, describes it as inputs that alter a model's behavior in unintended ways. Direct injection arrives through the user's own prompt. Indirect injection arrives through external content the model processes, such as a web page, a file or an email, and the instructions need not be visible to a human reader1.

The distinction matters for ownership. Direct injection is a risk at the front door, where you control authentication and rate limits. Indirect injection bypasses the front door. A customer-support agent that summarizes inbound email, or a coding assistant that reads a repository, accepts text from people you have never vetted.

Anthropic's September 2026 threat report gives an official example of the same idea in practice2. It describes an actor it tracks as GTG-50020 injecting instructions into an AI vendor's evaluation sandbox to obtain production API keys2. The attacker used content the system would read, not a flaw in the interface.

Why filtering alone fails

The UK National Cyber Security Centre argues that prompt injection is not like SQL injection. Databases separate commands from data; language models make no such distinction, so the NCSC says the problem may never be totally mitigated and describes models as inherently confusable deputies3.

That has practical consequences. The NCSC advises against deny-listing attack phrases and recommends being skeptical of products that claim to stop prompt injection outright3. OWASP likewise says that retrieval-augmented generation and fine-tuning do not fully mitigate the risk1. Filters still have a role, since OWASP lists input and output filtering among its mitigations1, but they reduce volume and cannot be the control that decides whether an incident happens.

The right question is therefore not whether an injection can be blocked but what happens when one gets through. The NCSC frames this as privilege: a model processing content from a party should be treated as having no more privileges than that party3.

Five layers between untrusted text and a harmful action

Monitoring and response01Output handling02Tool permissioning03Untrusted-content isolation04Privilege separation05
  1. Monitoring and response

    Log inputs, outputs and tool use in full; alert on unusual tool sequences and block or roll back.

  2. Output handling

    Treat model output as untrusted before it reaches a browser, shell, database or another system.

  3. Tool permissioning

    Allow-lists, scoped credentials and human approval enforced by code outside the model.

  4. Untrusted-content isolation

    Mark external content as data, process it in a separate low-privilege context and constrain its output.

  5. Privilege separation

    Give each agent only the data and actions its task needs, and keep sensitive actions in separate components.

Conceptual layering for an LLM application. No layer is assumed to be perfect; each limits what the next must catch.

Putting the layers in place, in order

Start with the layers that cap the damage, then add the ones that detect and filter.

  1. Map what the model can read and do

    List every source of text the system ingests and every tool it can call. The OWASP mitigation list begins with constraining model behavior and enforcing least privilege1.

    Output
    Data-flow map with untrusted sources marked
    Owner
    Security architect
  2. Separate privileges

    Split tasks so that no single model context holds both untrusted input and sensitive capability, such as sending email or reading customer records.

    Output
    Component design with privilege boundaries
    Owner
    Engineering lead
  3. Isolate untrusted content

    OWASP recommends segregating and clearly identifying external content1, and the NCSC recommends marking data separately from instructions3. Treat this as risk reduction, not a guarantee.

    Output
    Separate handling path for external text
    Owner
    Platform team
  4. Enforce tool permissions in code

    Use allow-lists, narrow scopes and approval for high-risk actions, as OWASP advises1. For servers following the Model Context Protocol, the protocol's security guidance forbids passing through tokens not issued for the server and recommends progressive least-privilege scopes4.

    Output
    Tool permission register and approval gates
    Owner
    Platform team
  5. Handle output as untrusted

    Validate expected output formats, and sanitize anything the model emits before it reaches another system. OWASP lists validating output formats as a mitigation1.

    Output
    Output validation and encoding rules
    Owner
    Application developers
  6. Monitor and test

    The NCSC recommends logging full inputs, outputs and tool use3, and OWASP recommends adversarial testing1. Feed red team results back into the layers above.

    Output
    Alerts, runbooks and a retest schedule
    Owner
    Security operations and testing

Six design patterns for agents, and what each constrains

PatternWhat it constrainsTypical trade-off
Action-selectorThe agent picks from a fixed menu of actions and does not act on text returned by tools5.Low flexibility; suits narrow, repeatable tasks.
Plan-then-executeThe agent commits to a plan before it reads untrusted content, so later text cannot add steps5.Plans cannot adapt to what the agent finds.
LLM map-reduceIsolated sub-agents process each item, and only constrained results reach the coordinator5.More components to build and monitor.
Dual LLMA privileged model never sees untrusted text, while a quarantined model reads it without tools5.Complexity in passing results safely between the two.
Code-then-executeThe agent writes a program as its plan, which can be inspected and constrained before it runs5.Needs a safe execution environment.
Context-minimizationUnneeded content, including the original prompt, is removed from context before later steps5.Loses context that some tasks require.

The paper behind this table argues that patterns should be combined and that agents should be application-specific rather than general-purpose5.

A hypothetical walk-through: the support-inbox summarizer

Common failure patterns in prompt injection programs

Relying on a phrase filter as the primary control

Early signalThe design review answer to injection is a list of blocked strings.

MitigationKeep filters as a noise reducer and add controls that cap what a fooled model can do3.

One agent holds both untrusted input and sensitive tools

Early signalA single service account can read external web content and write to internal systems.

MitigationSplit into separate contexts with approval between them, as in the dual-model and map-reduce patterns5.

Permissions enforced only in the system prompt

Early signalTool limits are described in text to the model rather than in code.

MitigationMove allow-lists, scopes and approval gates into deterministic code, as the NCSC recommends for non-LLM safeguards3.

No record of what the model saw

Early signalIncident review cannot show which document triggered an action.

MitigationLog full inputs, outputs and tool calls, with retention that fits your incident process3.

Questions and answers

What is the difference between direct and indirect prompt injection?

Direct injection comes from the user's own prompt. Indirect injection comes from external content the model processes, such as a web page, file or email, and the instructions may be invisible to people1. Indirect injection is usually the bigger concern for agents, because attackers can plant text without ever using your interface.

Can prompt injection be fully prevented?

Not with current approaches. OWASP says prevention is not guaranteed, and the NCSC says prompt injection may never be totally mitigated13. The practical aim is to limit what a manipulated model can reach or do, and to detect misuse quickly.

Do input filters and guardrail products work?

They help with known patterns, and OWASP lists input and output filtering among its mitigations1. They cannot be the only control. The NCSC advises against deny-listing attack phrases and suggests skepticism toward products that claim to stop prompt injection3.

Which control should a team build first?

Start with least privilege and approval gates for high-risk actions, enforced in code outside the model. These limit the damage even when the model is fooled. Then isolate untrusted content, validate output and add logging, and test the whole stack with a red team exercise.

How do MCP servers change the risk?

They add more tools and more trust boundaries. The protocol's security guidance covers confused deputy problems, forbids passing through tokens not issued for the server, and recommends progressive least-privilege scopes and sandboxing for local servers4. Vet each server as you would any third-party dependency.

Sources

  1. LLM01:2025 Prompt Injection — OWASP GenAI Security Project · checked 19 November 2026
  2. Detecting and countering misuse of AI: September 2026 — Anthropic · checked 19 November 2026
  3. Prompt injection is not SQL injection (it may be worse) — UK National Cyber Security Centre · checked 19 November 2026
  4. Security best practices — Model Context Protocol · checked 19 November 2026
  5. Design Patterns for Securing LLM Agents against Prompt Injections — arXiv · checked 19 November 2026

More in Cyber Defense

Back to Cyber Defense

Next step

Review one agent's permissions and untrusted inputs

Send us a description of one assistant or agent, the sources it reads and the tools it can call. We will return a short list of the layers most likely missing and a test plan for them.

Request an agent review

Your starting point

What would you like to move forward?

Start with the outcome that feels closest. We'll help give the next step a useful shape.

Let's find your next move

One conversation. A clearer direction.

Tell us what you want to improve. A few sentences are enough to start.

shayan@coldai.org