What is System Prompt?
Updated on
System Prompt is a provider-specific high-authority instruction layer used to define persistent behavior, scope, tool policy, response rules, and handling of lower-priority input for a model request.
How It Works
Authority is provider specific
Message-role names and precedence differ among providers, model families, and orchestration frameworks. Some APIs distinguish system, developer, user, assistant, and tool content; others expose system instructions through a dedicated parameter or merge layers internally. An application must follow the current provider contract and preserve role metadata rather than flattening every message into one string. A system prompt is normally above user instructions, but it does not override provider-level safety policy.
What belongs in the instruction layer
Include durable product rules: supported scope, source-grounding policy, tool-selection criteria, refusal or escalation behavior, uncertainty handling, and output expectations. Put request-specific facts in user or data fields instead of continually rewriting the system layer. State conflict resolution explicitly and avoid duplicated, vague, or contradictory rules. Keep the prompt concise enough to control token cost and make each requirement independently testable.
What must be enforced in code
Do not store secrets in a system prompt or assume users cannot recover its contents. The model may reveal, paraphrase, or act on sensitive text, and prompt injection can redirect behavior. Authentication, authorization, tool allowlists, argument validation, data access, rate limits, audit logs, and human approval for consequential actions belong in deterministic application code. Treat model-selected tools and arguments as proposals that must pass policy checks.
Versioning and regression coverage
Version the system prompt with tool definitions, output schemas, model identifiers, and application policy. A release suite should cover allowed requests, out-of-scope requests, missing evidence, conflicting user instructions, direct and indirect injection, tool escalation, and refusal presentation. Compare candidate and baseline over repeated trials where needed, and monitor production outcomes because an offline pass cannot detect every distribution shift.
Key Characteristics
- Uses a provider-defined authority level that is normally above user messages
- Expresses durable product behavior rather than request-specific user data
- Can specify scope, grounding, refusal, escalation, tools, and output policy
- Consumes input tokens and can affect latency, caching, and context availability
- Must be versioned and regression-tested with the surrounding application contract
- Cannot keep secrets or replace deterministic access and action controls
Common Use Cases
- Defining a support assistant's scope, source policy, and escalation behavior
- Describing when an agent may propose a tool call and what evidence it needs
- Requiring grounded answers, uncertainty disclosure, and source identifiers
- Standardizing refusal and clarification behavior across product workflows
- Setting a stable response contract while application code enforces permissions
Example
Loading code...Frequently Asked Questions
Does a system prompt always have the highest authority?
No universal role hierarchy exists. The provider defines which instruction layers are available and how conflicts are resolved; provider or root policy may outrank an application-supplied system or developer message. Check the current API contract and preserve role metadata when constructing requests.
Can a system prompt securely hide secrets or business rules?
No. Treat all model-visible text as potentially recoverable or inferable. Keep credentials, private policy data, and privileged logic outside the prompt. The application should expose only necessary information and enforce access decisions before and after every model or tool call.
Can a system prompt prevent prompt injection?
It can state boundaries and improve normal instruction following, but it cannot eliminate direct or indirect injection. Separate untrusted content, restrict tools and data by least privilege, validate model proposals, monitor outcomes, and require human approval for high-impact actions.
How long should a system prompt be?
It should be as short as possible while keeping every durable rule explicit and testable. Extra text consumes input tokens, can increase prefill latency, reduces room for user context, and creates more opportunities for contradictions. Measure behavior after removing or reorganizing instructions rather than optimizing length alone.
How should system prompts be tested before release?
Run the baseline and candidate against allowed tasks, out-of-scope requests, missing evidence, conflicting instructions, injection attempts, tool escalation, and formatting cases. Use deterministic checks for permissions and schemas, calibrated rubrics for open-ended quality, repeated trials for unstable cases, and human review where failure has material impact.