What is Prompt Injection?
Prompt Injection is attacker-controlled or attacker-influenced content that causes a large language model application to follow an unintended instruction or pursue an unintended goal.
Quick Facts
| Full Name | Prompt Injection Attack |
|---|---|
| Created | Widely discovered with the opening of LLM APIs like ChatGPT in 2022, and listed as the #1 vulnerability in the OWASP LLM Top 10 in 2023. |
| Specification | Official Specification |
How It Works
Prompt injection is an application-level confused-deputy problem. A model is authorized to interpret content or act for a user, then direct input or indirect content from a webpage, email, RAG chunk, source file, image, tool result, or memory influences it to use that authority for another purpose. The security impact depends on the full source-sink path: what untrusted source can influence the model, which private data or capabilities it can reach, and where data or side effects can leave the system. Role hierarchy, delimiters, model training, classifiers, and prompt firewalls can improve resilience, but they are probabilistic controls rather than authentication or authorization boundaries. Production systems should minimize capabilities, preserve provenance, enforce object-level authorization and destination policy in code, bind confirmation to the exact action, isolate execution, and evaluate complete attack chains. See the <a href="https://qubittool.com/blog/prompt-injection-attack-defense-guide">prompt injection defense guide</a> for the architecture and policy-gate example.
Key Characteristics
- Can arrive directly through a user request or indirectly through external content the application processes
- Can target integrity, confidentiality, availability, model safety, or consequential tool-mediated actions
- Becomes more severe when one workflow combines untrusted input, sensitive data, broad tools, and external communication
- Can persist through summaries, memory, indexes, generated configuration, or later tool results
- Can use multilingual, encoded, multimodal, multi-turn, and adaptive variants that defeat static string rules
- Requires outcome-based testing because a refusal is irrelevant after an unauthorized action or data transfer
Common Use Cases
- Threat modeling browser, email, RAG, coding, MCP, and computer-use agents
- Reviewing whether tool calls preserve the authenticated user, tenant, object, action, and destination
- Testing direct, indirect, persistent, multimodal, and repeated attacks alongside legitimate tasks
- Designing provenance-aware controls for sensitive arguments, memory writes, and external data flows
- Converting confirmed incidents into regression cases, containment runbooks, and credential-revocation procedures
Example
Loading code...Frequently Asked Questions
What is the difference between prompt injection and jailbreaking?
Prompt injection redirects an LLM application through attacker-influenced instructions, including instructions embedded in external data. Jailbreaking focuses on bypassing a model's safety behavior. They can overlap, but application security must also protect private data, tools, persistent state, and external effects.
What is indirect prompt injection?
Indirect prompt injection is delivered through content the application reads rather than through the user's request. Common sources include webpages, email, documents, RAG records, code comments, images, tool output, and memory. The content may be legitimate data for the task while also attempting to redirect the agent.
Can a classifier or LLM firewall prevent prompt injection?
It can detect some attacks and provide useful telemetry, but it cannot prove an input safe. Adaptive attacks can resemble legitimate instructions or contextual social engineering. Keep authentication, authorization, object ownership, destination policy, rate limits, and irreversible-action approval outside the classifier.
Does separating system and user messages solve prompt injection?
No. Provider-supported roles and clear labels improve instruction following, but lower-trust content can still influence probabilistic model decisions. Handwritten delimiters are not access control, and a system prompt cannot authorize a resource or transaction.
How should prompt injection defenses be evaluated?
Pair representative legitimate tasks with direct, indirect, multilingual, multimodal, persistent, multi-turn, and repeated attacks. Measure unauthorized actions, data egress, state changes, utility success, false refusals, detector errors, latency, cost, and confidence intervals for repeated attempts.