LLM Tool Calling: Production Architecture and Safety

Tool calling turns model output into a proposal for software execution. That proposal may request a weather lookup, but it may also request a refund, file deletion, database query, email, purchase, or code execution. The JSON is not the security boundary. The executor is.

A production system should use this mental model:

The model suggests a typed operation. Deterministic code authenticates the principal, validates arguments, authorizes the exact effect, executes under bounded capabilities, and returns an untrusted result.

This guide develops that contract, explains current provider interfaces without tying the architecture to one SDK, and provides a runnable executor core. Technical claims were reviewed against current OpenAI, Anthropic, and MCP documentation on July 16, 2026.

Key Takeaways

• Tool calls are untrusted model output, even when strict schema mode guarantees their shape.
• Derive user, tenant, permissions, prices, and ownership from trusted server state, never model arguments.
• Separate read tools from write tools and preview an effect before committing it.
• Make side effects idempotent and bind confirmation to the exact canonical operation.
• Retry reads selectively; never blindly retry an ambiguous write.
• Parallelize only independent, read-only or safely idempotent calls.
• Tool results may contain prompt injection, malformed data, secrets, or oversized payloads.
• Bound the loop by steps, calls, wall time, tokens, cost, and repeated-call detection.
• Evaluate selection, argument meaning, authorization, execution, recovery, and final answers separately.

Tool Calling Is a Protocol, Not Execution

The basic loop is:

user request
  -> model receives available tool contracts
  -> model emits zero or more call proposals
  -> application validates and authorizes
  -> executor invokes approved tools
  -> application returns typed results
  -> model continues or produces a final answer

The model does not call a Python function by itself. Provider APIs serialize a choice and arguments; your application decides what happens next.

This distinction creates three independent correctness questions:

1. Selection: Was the right tool chosen?
2. Arguments: Do the values mean what the user intended?
3. Authorization: Is this principal allowed to cause this exact effect?

A schema can help with the second question’s syntax. It cannot answer the third.

Structured Output, Tool Calling, and MCP

Structured Output

Use structured output when the desired product is data:

• extract invoice fields;
• classify a ticket;
• produce a typed plan for human review;
• transform text into a known schema.

No external action is implied. Schema validity still does not guarantee factual correctness.

Tool Calling

Use tool calling when the model may select a capability or provide arguments to an executor. It usually creates a loop because the model needs the result before continuing.

MCP

Model Context Protocol defines client-server interfaces for tools, resources, prompts, capability negotiation, and transport. It is not an upgraded form of model function calling. An MCP Host still needs to:

• decide which server and tools are trusted;
• expose a scoped subset to the model;
• preserve user and tenant identity;
• authorize each call;
• validate server output;
• manage consent, cancellation, timeout, and audit.

Provider tool formats and MCP solve adjacent layers.

Provider Interface Differences

Do not mix wire formats.

Read the provider documentation for the selected endpoint and model. Model names, supported JSON Schema subsets, streaming events, and state-continuation APIs change faster than the executor architecture.

Strict schema mode is valuable: it can guarantee that generated arguments conform to the supported schema. It does not guarantee:

• a city exists;
• an order belongs to the user;
• a product is in stock;
• a price or exchange rate is authoritative;
• a request is safe or authorized;
• a tool result is trustworthy.

Design Narrow Tool Contracts

Good tool contracts expose business intent, not infrastructure power.

Prefer:

get_my_order(order_reference)
request_refund(order_reference, reason_code)
preview_subscription_change(plan_id)

Avoid:

run_sql(query)
http_request(url, headers, body)
execute_shell(command)
update_record(table, id, fields)

The narrow contract lets server code derive the authenticated customer and enforce workflow rules.

Do Not Let the Model Supply Trusted Fields

For an order:

• derive customer_id from the authenticated session;
• load product price from the catalog;
• calculate tax and discounts in code;
• resolve order ownership in the service;
• generate the idempotency key server-side;
• never accept is_admin, approved, or role from model output.

Model arguments describe user intent. They are not identity or authority.

JSON Schema That Supports Reliability

Use:

• explicit required;
• additionalProperties: false where strict mode requires or supports it;
• bounded strings and arrays;
• enums for stable business concepts;
• integers instead of floating-point values for money where appropriate;
• one meaning per field;
• descriptions that state preconditions and non-goals.

Example:

{
  "type": "object",
  "properties": {
    "order_reference": {
      "type": "string",
      "pattern": "^[A-Z0-9-]{6,32}$",
      "description": "Customer-visible order reference. Never use an internal database ID."
    },
    "reason_code": {
      "type": "string",
      "enum": ["duplicate", "damaged", "not_received", "other"]
    }
  },
  "required": ["order_reference", "reason_code"],
  "additionalProperties": false
}

Avoid asking the model to reproduce values already held by the application. Inject them into executor context instead.

The Production Executor Pipeline

Every proposal should pass through:

parse -> schema validate -> resolve tool -> authenticate -> authorize object
      -> validate business state -> classify risk -> confirm if needed
      -> reserve idempotency -> execute with timeout -> normalize result
      -> audit -> return untrusted result

Parse and Resolve Safely

• reject malformed JSON;
• reject unknown tools without indexing a dictionary blindly;
• enforce a versioned registry;
• reject extra arguments;
• cap argument and result size;
• distinguish model errors from policy denials and tool failures.

Authenticate and Authorize

Pass a trusted execution context:

principal_id, tenant_id, roles, request_id, trace_id, locale

The model must not construct this context. Object authorization must occur after loading the resource, not merely at the endpoint level.

Preview Before Commit

For high-impact writes:

1. resolve authoritative values;
2. create a preview of the exact effect;
3. show it to the user;
4. bind approval to a digest of canonical arguments and resource version;
5. commit only if the digest still matches.

This prevents approval for one recipient, amount, or resource being reused for another.

Runnable Executor Core

The following standard-library example models a server-side order-note tool. The model may provide an order reference and note; identity and ownership come from trusted context.

from __future__ import annotations

from dataclasses import dataclass
from enum import Enum
from hashlib import sha256
import json


class Status(str, Enum):
    OK = "ok"
    DENIED = "denied"
    INVALID = "invalid"
    DUPLICATE = "duplicate"


@dataclass(frozen=True)
class ExecutionContext:
    principal_id: str
    tenant_id: str
    request_id: str


@dataclass(frozen=True)
class Order:
    reference: str
    tenant_id: str
    owner_id: str


@dataclass(frozen=True)
class ToolResult:
    status: Status
    code: str
    data: dict[str, object]


class OrderRepository:
    def __init__(self, orders: list[Order]) -> None:
        self.orders = {order.reference: order for order in orders}
        self.notes: dict[tuple[str, str], str] = {}

    def get(self, reference: str) -> Order | None:
        return self.orders.get(reference)

    def add_note_once(
        self,
        *,
        order: Order,
        note: str,
        idempotency_key: str,
    ) -> bool:
        key = (order.reference, idempotency_key)
        if key in self.notes:
            return False
        self.notes[key] = note
        return True


def make_idempotency_key(
    context: ExecutionContext,
    tool_name: str,
    arguments: dict[str, object],
) -> str:
    payload = {
        "arguments": arguments,
        "principal_id": context.principal_id,
        "request_id": context.request_id,
        "tenant_id": context.tenant_id,
        "tool_name": tool_name,
    }
    canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
    return sha256(canonical.encode()).hexdigest()


def add_order_note(
    arguments: dict[str, object],
    *,
    context: ExecutionContext,
    repository: OrderRepository,
) -> ToolResult:
    if set(arguments) != {"order_reference", "note"}:
        return ToolResult(Status.INVALID, "invalid_arguments", {})

    reference = arguments["order_reference"]
    note = arguments["note"]
    if not isinstance(reference, str) or not isinstance(note, str):
        return ToolResult(Status.INVALID, "invalid_types", {})
    if not 1 <= len(note) <= 500:
        return ToolResult(Status.INVALID, "invalid_note_length", {})

    order = repository.get(reference)
    if order is None:
        return ToolResult(Status.DENIED, "not_found_or_forbidden", {})
    if (
        order.tenant_id != context.tenant_id
        or order.owner_id != context.principal_id
    ):
        return ToolResult(Status.DENIED, "not_found_or_forbidden", {})

    key = make_idempotency_key(context, "add_order_note", arguments)
    created = repository.add_note_once(
        order=order,
        note=note,
        idempotency_key=key,
    )
    if not created:
        return ToolResult(Status.DUPLICATE, "already_applied", {})

    return ToolResult(
        Status.OK,
        "note_added",
        {"order_reference": order.reference},
    )

Notice the uniform not_found_or_forbidden response. It avoids revealing whether another user’s order exists.

Idempotency and Write Semantics

Network timeouts create ambiguity: the server may have committed a write even though the client saw no response. A blind retry can duplicate a payment, email, or order.

For every write tool:

• accept or derive a stable idempotency key;
• store result and status atomically with the effect;
• return the prior result for a duplicate key;
• define whether the operation is create, replace, append, or compare-and-set;
• use resource versions for concurrent updates;
• make cancellation semantics explicit.

Retries should depend on failure class:

Do not feed raw stack traces, secrets, or internal topology back to the model.

Parallel Calls and Dependencies

Multiple proposed calls are not automatically safe to run in parallel.

Parallelize only when:

• calls are independent;
• they are read-only or safely idempotent;
• they do not compete for the same quota or resource version;
• ordering has no business meaning;
• partial failure has a defined result.

Examples:

• weather for two cities: usually independent;
• reserve inventory then create order: dependent;
• update the same document twice: conflicting;
• send three emails: independent technically, but consequential and confirmation-sensitive.

Represent dependencies as a DAG, cap concurrency, propagate cancellation, and preserve each call ID. “The model returned an array” is not a concurrency strategy.

Tool Results Are Untrusted Input

Tool output can contain:

• prompt injection from a webpage or email;
• data belonging to another tenant due to an upstream bug;
• malformed or unexpectedly large values;
• active HTML, Markdown links, code, or file paths;
• secrets and personal data;
• stale records.

Validate result schemas, enforce response-size limits, redact fields, attach provenance and timestamps, and keep instructions from tool data at the lowest trust level. Never execute code found in a result merely because the model asks.

Bounded Tool Loops

An agent loop needs hard budgets:

• maximum model turns;
• total and per-tool calls;
• wall-clock deadline;
• input/output tokens;
• monetary cost;
• repeated identical call detection;
• maximum result bytes;
• user cancellation.

Stop with a structured reason such as budget_exhausted, repeated_call, or approval_required. Do not let the model decide whether its own loop limit applies.

Tool Discovery and Least Exposure

Large tool catalogs increase selection errors, prompt cost, and attack surface. Route to a task-specific subset before the model sees definitions.

Tool discovery must not bypass policy:

• registry metadata is not authorization;
• an MCP Server being installed does not make all its tools trusted;
• tool descriptions and server results are untrusted supply-chain inputs;
• use allowlisted server identities, pinned versions where possible, scoped credentials, and per-tool policy.

Observability Without Leaking Secrets

Record:

• model, prompt, and tool-schema versions;
• proposed tool name and arguments after redaction;
• policy decision and reason;
• principal, tenant, resource, call ID, and idempotency key hash;
• start/end time, retries, timeout, and result code;
• confirmation and cancellation events;
• final outcome and token/cost data.

Use distributed traces across model, executor, and downstream service. Do not log credentials, full personal data, or unrestricted tool results.

Evaluation

Evaluate layers independently:

Selection

• correct tool;
• correct decision not to call;
• confusion between overlapping tools;
• performance with irrelevant tools present.

Arguments

• schema conformance;
• field-level semantic accuracy;
• missing-information behavior;
• invented identifiers and values;
• locale, date, currency, and unit handling.

Execution and Security

• object authorization;
• cross-tenant denial;
• confirmation binding;
• idempotent duplicates;
• prompt injection through tool results;
• timeout, retry, and partial failure;
• loop termination.

End-to-End

• user task success;
• factual grounding in tool results;
• no claim of success after failure;
• p50/p95 latency;
• tool calls, tokens, and cost per successful task.

Test with real anonymized workflows, adversarial inputs, provider/model upgrades, and fault injection.

Production Checklist

• [ ] Tools expose narrow business operations, not generic infrastructure.
• [ ] Strict schema mode is enabled where supported.
• [ ] Arguments are validated again by the executor.
• [ ] Identity, tenant, ownership, price, and roles come from trusted systems.
• [ ] Object-level authorization occurs on every call.
• [ ] Writes have idempotency and explicit conflict semantics.
• [ ] High-impact actions use preview and digest-bound confirmation.
• [ ] Parallel calls are proven independent.
• [ ] Tool results are size-limited, validated, redacted, and untrusted.
• [ ] Loop, time, cost, and retry budgets are enforced outside the model.
• [ ] MCP servers and discovered tools are scoped and authorized.
• [ ] Traces support incident reconstruction without exposing secrets.

Frequently Asked Questions

Can the model generate SQL or shell commands as tool arguments?

Avoid it. Expose parameterized domain operations. If code execution is a real requirement, isolate it in a sandbox with no ambient credentials and strict resource and network controls.

Should a failed tool result be sent back to the model?

Send a normalized, redacted error code when the model can safely recover. Do not expose stack traces or invite retries for authorization failures.

Should every read be parallelized?

No. Respect downstream quotas, dependencies, consistency requirements, cancellation, and result budgets. Controlled concurrency is an executor decision.

Does strict schema remove the need for validation?

No. It removes a class of shape errors for supported schemas. Business, authorization, freshness, and cross-field constraints remain.

Conclusion

Reliable tool calling is an application-runtime discipline, not a clever prompt. The model provides flexible intent interpretation; the executor supplies identity, policy, transactions, limits, and evidence.

Design each tool as if an untrusted but persuasive caller will invoke it with perfectly valid JSON. If the backend remains safe and correct under that condition, the tool contract is ready for a model.

Primary Sources

• OpenAI Function Calling Guide
• OpenAI Structured Outputs
• Anthropic Tool Use Documentation
• Anthropic Strict Tool Use
• Model Context Protocol Specification