TL;DR

An AI agent tool is executable authority, not a harmless prompt extension. Secure it with a verified registry, least privilege, deterministic authorization, isolated execution, bounded inputs and outputs, exact-action approval, and versioned audit evidence. Assume descriptions and results can be hostile. A model refusal or system prompt cannot replace runtime policy.

The trust boundaries in agent tool use

Agent systems combine two control planes:

  • the model proposes what to do based on natural-language context;
  • trusted software decides what is allowed and executes it.

Security fails when the proposal is mistaken for authorization.

flowchart LR A["User and untrusted content"] --> B["Model planner"] C["Reviewed tool registry"] --> B B --> D["Proposed tool + arguments"] D --> E["Policy and approval"] E --> F["Sandboxed executor"] F --> G["External system"] G --> H["Bounded untrusted result"] H --> B

Every arrow is a trust boundary. User input, retrieved documents, tool descriptions, arguments, remote responses, memory, and delegated-agent messages can all influence the next decision.

Threat model: four distinct attack paths

Do not collapse every incident into “prompt injection.” Different paths need different controls.

Attack path Example Primary controls
Direct or indirect prompt injection A web page tells the agent to upload private files capability limits, provenance, egress policy
Tool poisoning A tool description hides an instruction or changes after review registry verification, version pinning, diff review
Tool misuse after function calling The agent uses an allowed delete API on the wrong resource object authorization, previews, approval, idempotency
Malicious tool result A result contains instructions, secrets, or huge payloads result schema, size limit, taint tracking, content isolation

OWASP's Agentic Skills Top 10 emphasizes publisher verification, permission manifests, safe parsing, dependency pinning, sandboxing, and runtime monitoring. These are ordinary software supply-chain controls applied to a model-directed execution layer.

Build a reviewed tool registry

Dynamic discovery should not mean dynamic trust. Convert discovered tools into an internal registry entry that has been reviewed for the target environment.

json 
{
  "tool_id": "invoice.lookup",
  "version": "3.2.1",
  "artifact_digest": "sha256:...",
  "publisher": "billing-platform",
  "effects": ["read:invoice"],
  "network_allowlist": ["billing.internal"],
  "data_classes": ["customer_financial"],
  "approval": "none",
  "result_max_bytes": 65536
}

Material changes create a new review decision:

  • name or description;
  • parameter or result schema;
  • requested permissions;
  • network destinations;
  • package dependencies or artifact digest;
  • side-effect behavior;
  • data retention.

Treat an MCP server, plugin, skill, or package registry as an untrusted source of candidates. The internal registry is the execution authority.

Enforce least privilege at runtime

Permissions should be narrower than “the agent can call this tool.” Authorize the exact effect against trusted identity and current state.

typescript 
type ToolRequest = {
  tool: "invoice.lookup" | "invoice.refund";
  invoiceId: string;
  amount?: number;
};

async function authorize(
  request: ToolRequest,
  context: { userId: string; tenantId: string; roles: string[] },
) {
  const invoice = await loadInvoice(request.invoiceId);
  if (invoice.tenantId !== context.tenantId) throw new Error("tenant_denied");

  if (request.tool === "invoice.refund") {
    if (!context.roles.includes("refund_operator")) throw new Error("role_denied");
    if (!request.amount || request.amount > invoice.refundableAmount) {
      throw new Error("invalid_amount");
    }
  }
  return invoice;
}

The model must not supply userId, tenantId, role, price, approval state, or policy version. Derive them from authenticated runtime context and authoritative services.

Apply resource budgets:

  • maximum tool calls and repeated identical calls;
  • execution time and concurrency;
  • output bytes and rows;
  • network destinations and methods;
  • file-system paths and process capabilities;
  • spend, write volume, and retry count.

Design approval that actually constrains action

“Allow?” is not meaningful approval. Show the user the exact effect:

text 
Refund $84.20 for invoice inv_2817
Destination: original payment method
Reason: duplicate charge
Tool version: [email protected]
Expires: 2026-07-28T12:05:00Z

Bind the signed approval to:

  • tool and version;
  • normalized arguments;
  • target resource and tenant;
  • destination and side effect;
  • policy version;
  • expiration and idempotency key.

If any bound field changes, request approval again. A confirmation must not authorize a later broader call.

Treat tool results as untrusted

A successful tool call can still return malicious or unsafe content. Normalize results before they re-enter model context.

typescript 
function normalizeSearchResult(value: unknown) {
  if (!value || typeof value !== "object") throw new Error("invalid_result");
  const input = value as Record<string, unknown>;
  const text = typeof input.text === "string" ? input.text.slice(0, 8_000) : "";
  return {
    sourceId: String(input.sourceId ?? ""),
    text,
    trust: "external_untrusted",
  };
}

Keep source identity and trust labels attached through summarization and memory writes. Do not let returned text define new tools, change policy, select an external destination, or write persistent memory without a separate gate.

Sandbox tools and restrict egress

Isolation limits damage when authorization or model behavior fails.

Boundary Minimum control
Process non-root identity, syscall and resource limits
File system read-only base, explicit writable directories
Network deny by default, destination and protocol allowlist
Credentials short-lived, tool-specific, tenant-scoped
Data minimum fields, redaction, result-size bounds
Side effects idempotency keys, previews, transaction limits

Separate read tools from write tools. Do not expose a generic shell, SQL executor, HTTP client, or cloud credential when a narrow business operation can be implemented instead.

Test the complete attack chain

Test trajectories, not only single prompts:

  1. poisoned tool description attempts to add an undeclared prerequisite;
  2. search result requests data exfiltration;
  3. tool returns another tenant's identifier;
  4. model repeats a write after a timeout;
  5. tool version changes between planning and execution;
  6. delegated agent asks for a broader credential;
  7. output attempts to persist an instruction in memory.

For each case, assert the deterministic boundary that stops impact. A model that “usually refuses” is not a passing security control.

Record tool ID and version, registry digest, authenticated principal, policy decision, approval reference, normalized argument digest, destination class, side-effect result, duration, and bounded error code. Avoid storing raw secrets, prompts, and full private results.

Incident response and kill switches

Prepare controls before an incident:

  • disable a tool version or publisher globally;
  • revoke tool credentials independently;
  • block a destination or effect class;
  • quarantine memory written by affected runs;
  • identify executions by artifact digest;
  • replay policy decisions with redacted fixtures;
  • notify owners of externally visible side effects.

Audit logs must be useful without becoming a second sensitive-data store. Separate low-sensitivity correlation metadata from tightly controlled payload evidence.

Production checklist

  1. Inventory tools, effects, data classes, and external destinations.
  2. Convert discovery results into a reviewed allowlisted registry.
  3. Pin artifacts and diff every material update.
  4. Derive identity and tenant from trusted context.
  5. Validate and authorize exact effects in code.
  6. Sandbox execution and deny network egress by default.
  7. Bind approval to exact arguments and versions.
  8. Normalize, bound, and label every tool result.
  9. Test injection, poisoning, retries, delegation, and memory persistence.
  10. Maintain kill switches and incident queries.

The prompt injection defense guide covers the broader untrusted-content problem. The function calling guide explains the proposal-to-execution boundary.

FAQ

Should an agent be allowed to install tools itself?

Not into a production trust domain. It may discover candidates or prepare a change request, but installation, permissions, artifact verification, and activation need an independent reviewed workflow.

Are signed tools automatically safe?

No. A signature proves artifact origin and integrity relative to a key; it does not prove the publisher is trustworthy, the code is safe, permissions are minimal, or returned content is benign.

Can input and output classifiers stop tool attacks?

They can add useful signals but are probabilistic and bypassable. Use them to inform policy or review, never as the only barrier protecting data or side effects.

How should retries work for write tools?

Use idempotency keys and authoritative status checks. After an ambiguous timeout, query the operation result before retrying. Never let the model decide that a write is safe to repeat.

Is a human in the loop enough?

Only if the human sees an accurate, bounded action and the approval is cryptographically or transactionally bound to that exact action. Vague or overloaded confirmations create approval fatigue and do not reduce authority.

Sources