What is Callback Handler?
Callback Handler is a framework hook that receives typed lifecycle events around model, tool, retrieval, stream, workflow, or agent execution for observation or explicitly defined interception.
How It Works
Event identity and trace structure
Every event should carry run, span, and parent identifiers; component type and version; event kind; timestamps; attempt; and a terminal status. Start and end events must correlate even across nested agents and asynchronous tools. Map framework events to an observability schema such as OpenTelemetry at one adapter boundary rather than scattering vendor attribute names through business code.
Observer, interceptor, and middleware boundaries
An observer records an event without changing execution. An interceptor may validate, redact, short-circuit, or replace input or output. Middleware usually wraps the operation and owns before-and-after control. Frameworks use these labels differently, so document whether a handler can mutate data, cancel work, return a substitute result, or trigger an external effect. Do not rely on unspecified handler order for correctness.
Streaming and reliability
Streams may be single-consumer resources; a callback that reads them can starve the application unless the framework tees or copies the stream correctly. Handlers run on the critical path in many runtimes, so bound their work, propagate cancellation, isolate failures, and move expensive export to a queue. External writes should be idempotent because retries or resumed runs can deliver equivalent events more than once.
Privacy, cost, and testability
Prompts, retrieved documents, tool arguments, model output, and credentials can contain sensitive data. Default to metadata-only telemetry, allowlist attributes, redact before export, control retention and access, and avoid high-cardinality labels such as raw user IDs or full prompts. Unit-test handler logic with synthetic events, then integration-test nesting, stream completion, cancellation, duplicate delivery, exporter outage, and behavior when an interceptor rejects a call.
Key Characteristics
- Receives typed lifecycle events with run, span, parent, component, attempt, and status identity
- Separates observation or interception from the component's primary business logic
- Requires explicit mutation, cancellation, replacement, and ordering semantics
- Must preserve stream ownership and propagate cancellation and terminal errors
- Needs bounded latency, failure isolation, idempotency, and backpressure handling
- Defaults to metadata-only telemetry with attribute allowlists and retention controls
Common Use Cases
- Creating nested spans for model, tool, retrieval, workflow, and agent operations
- Recording usage, latency, finish states, retry attempts, and provider request IDs
- Exporting privacy-filtered trajectory events for debugging and evaluation
- Streaming progress events without consuming the application's result stream
- Intercepting a sensitive tool proposal before deterministic authorization
Example
Loading code...Frequently Asked Questions
How is a Callback Handler different from middleware?
A read-only callback usually observes a point-in-time event, while middleware wraps an operation and controls the before-and-after flow. Some frameworks let callbacks intercept or replace results, blurring the terms. Rely on the documented mutation, ordering, cancellation, and error contract rather than the name.
What identity should every callback event contain?
Include a run ID, span ID, optional parent span, component name and version, event kind, attempt, timestamp, and terminal status. Tool and model events also need stable call IDs and finish reasons. These fields let traces reconstruct nesting, retries, handoffs, and partial failures.
Can a Callback Handler safely read a streaming response?
Only if the framework supplies a copy, tee, or replayable event stream. Many stream readers are single-consumer; consuming them inside a handler can deprive the caller of data. Define who owns close and cancellation, preserve chunk order and IDs, and test errors before and after the first chunk.
Should a callback failure stop the agent run?
It depends on purpose. A failed best-effort metrics exporter should normally be isolated, while an authorization interceptor may need to fail closed. Classify handlers by criticality, set timeouts and backpressure policy, and test exporter outages so an observability problem does not silently become an application outage.
Should prompts and tool arguments be recorded in callbacks?
Not by default. They may contain personal data, credentials, proprietary documents, or malicious payloads. Prefer metadata-only events, explicitly allowlist required content, redact before export, restrict access and retention, and avoid raw values as metric labels. Capture content only for a documented purpose and approved data policy.