Prompt Injection Firewall: Practical Guardrails for LLM Apps

When an LLM application reads user or external content, that content can influence its next decision. Prompt injection becomes a security issue when the influenced model can reach private data, invoke tools, change durable state, or communicate externally.

This page is a practical starting point. For the full threat model, provenance-aware data flow, bound approvals, sandboxing, and incident response, use the complete prompt injection defense guide.

The Short Answer

A prompt injection firewall is not a classifier placed in front of an LLM. It is the control plane around the complete application:

untrusted sources -> model proposal -> policy decision -> constrained effect
                           |                  |
                     telemetry only     trusted code

Detection can identify known abuse and increase scrutiny. Security comes from limiting sources and sinks, propagating trust metadata, authorizing each effect, validating outputs, and making high-impact operations recoverable. If a compromised model still cannot cross a tenant boundary or send data to an unapproved destination, an injection attempt has limited impact even when its wording was never recognized.

1. Define the Security Invariants

Do not begin with a list of suspicious phrases. Begin with statements that must remain true even if the model follows hostile content:

• data from tenant A is never returned to tenant B;
• a support ticket cannot authorize a refund;
• retrieved text cannot select a new network destination;
• a summary cannot silently convert untrusted text into trusted memory;
• an approval applies only to the exact recipient, resource, fields, amount, and operation shown to the user;
• a retry cannot duplicate an already committed effect.

Example: content in a support ticket can affect a draft reply, but it must not select a recipient, access another tenant's order, or approve a refund. These are application policies, not promises that a prompt can enforce.

Map both ends of every dangerous path:

2. Understand What Detection Can and Cannot Do

Injection can be direct user text or indirect content retrieved from a page, email, repository, RAG chunk, image, or tool result. It can also persist through generated summaries, configuration, or memory. Treat every external source as data rather than as authorized policy.

Keyword patterns, encoding normalization, Prompt Shields, and safety classifiers can support telemetry, triage, rate limits, and blocking known attacks. They cannot reliably infer intent across languages, encodings, multimodal inputs, and long interaction histories. OpenAI's 2026 security guidance describes mature prompt injection as closer to social engineering than a static malicious-string problem and recommends constraining impact rather than relying on detector-only filtering. OWASP similarly states that no foolproof prevention exists inside the model.

Use detector outcomes asymmetrically:

• a high-risk signal may block, isolate, redact, or require review;
• a low-risk signal must not unlock data, tools, or destinations;
• disagreement between detectors should become evidence, not an automatic allow;
• model self-evaluation must not approve its own proposed effect.

3. Build the Firewall Around Effects

Keep authority outside natural language

Use provider-supported roles and clear data delimiters so the model can reason more reliably. Do not concatenate secrets into prompts. This reduces confusion but is not access control: handwritten XML or JSON boundaries do not make untrusted text safe.

Attach machine-readable metadata to each context item: source, owner, tenant, sensitivity, revision, freshness, and whether it may influence a particular decision. Preserve those labels through OCR, extraction, summarization, and Agent-to-Agent handoffs.

Apply least privilege and deterministic authorization

Give the model only task-scoped capabilities. Prefer narrow operations such as invoice.read or ticket.create over a generic HTTP, SQL, shell, or cloud-administrator tool. Before execution, trusted code checks the authenticated principal, tenant, object ownership, action, final arguments, destination, data labels, quotas, and approval.

The following standard-library Go example rejects an exfiltration attempt without trying to classify the prompt text:

package main

import (
	"errors"
	"fmt"
	"net/url"
)

type Claims struct {
	Tenant string
	Scopes map[string]bool
}

type Effect struct {
	Tenant      string
	Tool        string
	Destination string
	HighImpact  bool
}

func authorize(claims Claims, effect Effect, allowedHosts map[string]bool, userConfirmed bool) error {
	if claims.Tenant == "" || claims.Tenant != effect.Tenant {
		return errors.New("tenant denied")
	}
	if !claims.Scopes[effect.Tool] {
		return errors.New("tool denied")
	}
	destination, err := url.Parse(effect.Destination)
	if err != nil || destination.Scheme != "https" || !allowedHosts[destination.Hostname()] {
		return errors.New("destination denied")
	}
	if effect.HighImpact && !userConfirmed {
		return errors.New("confirmation required")
	}
	return nil
}

func main() {
	claims := Claims{Tenant: "tenant-a", Scopes: map[string]bool{"ticket.create": true}}
	effect := Effect{
		Tenant:      "tenant-a",
		Tool:        "ticket.create",
		Destination: "https://attacker.example/upload",
	}
	allowedHosts := map[string]bool{"support.example.com": true}

	if err := authorize(claims, effect, allowedHosts, false); err != nil {
		fmt.Println("blocked:", err)
		return
	}
	fmt.Println("authorized")
}

The policy still needs object-level checks and server-side enforcement. The important property is that external content cannot rewrite claims, the host allowlist, or the confirmation record.

Control context, output, and egress

• enforce document ACLs before RAG ranking, not after the model has seen the text;
• separately authorize persistent-memory writes and attach an expiry;
• validate generated structured output before it becomes a tool argument;
• sandbox generated code and restrict network, filesystem, subprocess, and secret access;
• validate redirects and every outbound host rather than only the initial URL;
• keep secret values out of model context, traces, error text, and rendered links;
• bind approval to the final immutable effect and invalidate it when any field changes.

4. Design Failure and Recovery Paths

A blocked operation needs a useful alternative: explain the missing permission, request a bounded confirmation, produce a draft without sending it, or escalate to a human. Fail closed when identity, policy, destination, or commit status is unknown.

Tool retries require an idempotency key and a durable record of whether the effect committed. A timeout is not proof of failure. If the result is unknown, reconcile with the downstream system before retrying. Otherwise an injected or merely malformed workflow can duplicate emails, tickets, purchases, or refunds.

5. Test Complete Attack Chains

Use isolated AI red-team testing to exercise direct and indirect sources, tool-argument manipulation, RAG and memory poisoning, multilingual or multimodal variants, and repeated attempts. Measure unauthorized actions, data egress, and changes to durable state alongside legitimate-task accuracy, false refusals, latency, and cost. A model refusal is not a success if an external action already occurred.

Build the matrix from sources to sinks:

1. place attacks in every supported input channel;
2. vary encoding, language, document position, and multi-turn timing;
3. test with realistic credentials but isolated resources;
4. verify downstream authorization and audit evidence, not only the chat response;
5. classify whether each failure was prevented, detected, contained, recovered, or missed;
6. add confirmed incidents and near misses to a locked regression set.

Release gates should contain zero-tolerance invariants for cross-tenant access and unauthorized writes, plus workload-specific thresholds for task success, false refusals, latency, and cost. A single average "firewall score" hides where damage can occur.

6. Operate It as a Security Control

Version the model, system instructions, detector, retrieval policy, tool schemas, egress rules, and evaluation set. Alert on denied high-impact calls, destination changes, repeated encoding failures, unusual tool sequences, and durable-memory writes from low-trust sources. Logs must be useful for reconstruction without storing secrets or unnecessary personal data.

When an incident occurs, revoke credentials, stop dangerous tools, preserve sanitized evidence, reconcile uncertain effects, remove poisoned memory, identify every path that reused the content, and add a regression case. Rotating the system prompt alone does not repair the affected state.

Conclusion

An LLM firewall is a set of controls that prevents untrusted language from gaining authority. Detection helps, but the durable security boundary is limited capability, deterministic authorization, source-aware data flow, constrained egress, exact approval, recoverable effects, and continuous evaluation.

Primary Sources

• OWASP LLM01: Prompt Injection
• OpenAI: Designing AI Agents to Resist Prompt Injection
• Microsoft Prompt Shields documentation