Multi-Agent Systems: When and How to Build Them

A multi-agent system is an operational architecture, not a prompt-writing technique. It can isolate permissions, parallelize independent research, and let separately owned capabilities cooperate. It also adds messages, model calls, partial failures, duplicated context, and new paths for prompt injection.

The practical rule is simple:

Start with one agent and a deterministic workflow. Add another agent only when you can name the bottleneck it removes and the metric that will prove the benefit.

Key Takeaways

• Multiple steps do not imply multiple agents. Stable sequences belong in ordinary workflow code.
• An agent boundary should buy isolation, parallelism, or independent ownership.
• Separate the control plane from worker execution; do not let workers invent topology or authority.
• Role prompts describe behavior, while credentials and tool allowlists enforce capability.
• Exchange typed artifacts with provenance instead of forwarding entire conversations.
• Put budgets on every worker and on the complete run.
• Treat agent output as untrusted input, even when it came from your own model.
• Compare the design against a single-agent baseline before shipping.

The Decision Test

Use a multi-agent design when at least one condition is material:

Do not split merely because personas sound natural. Prefer one agent or deterministic code when:

• the flow is a fixed extract -> validate -> store pipeline;
• workers need almost the same context and tools;
• every result must be serialized through one bottleneck;
• the task is short and the coordination overhead dominates;
• no team can own cross-agent failures.

Run an ablation: same model, same tools, same task set, first as one agent and then as multiple agents. Compare success, p50/p95 latency, tokens, tool calls, policy violations, and operator effort.

Architecture Patterns

Manager-Worker

A manager decomposes work, assigns bounded tasks, and aggregates results. This fits parallel research and specialist routing, but the manager becomes a bottleneck and a high-value attack target.

request -> manager -> [worker A, worker B, worker C] -> verifier -> response

The manager should issue explicit assignments, not forward an ambiguous conversation. Workers return evidence and status; they do not decide whether another worker may access a protected tool.

Peer Handoffs

One agent transfers control to another based on a routing rule. This fits support or triage flows where one specialist owns the next stage.

triage -> billing
       -> account recovery
       -> technical support

A handoff must carry a minimal typed state and a reason. Cap the handoff count and reject cycles such as triage -> billing -> triage.

Shared Blackboard

Workers read and write versioned artifacts in a shared store. This supports asynchronous or long-running work, but requires compare-and-set updates, ownership rules, and conflict resolution. The blackboard is a database, not an unstructured prompt transcript.

Debate or Voting

Several workers propose answers and a judge selects or synthesizes one. This can improve coverage only when errors are sufficiently diverse. Calling the same model with near-identical prompts often creates correlated mistakes, not independent evidence.

Control Plane and Data Plane

Keep orchestration deterministic:

control plane
  task decomposition, routing, budgets, authorization, retries,
  cancellation, checkpointing, human approval, audit

worker data plane
  model inference, scoped retrieval, approved tool calls,
  typed result plus provenance

The control plane owns:

• the run ID, task graph, and allowed transitions;
• principal and tenant identity;
• per-worker tool and data scopes;
• retry and timeout policy;
• global token, cost, and wall-clock budgets;
• confirmation for consequential actions;
• terminal states such as completed, failed, cancelled, and needs_review.

A worker must not grant itself tools, create unbounded subtasks, or mark a business action successful without executor evidence.

Message Contracts

A useful work message contains:

{
  "schema_version": "1",
  "run_id": "run-42",
  "task_id": "research-security",
  "attempt": 1,
  "sender": "manager",
  "recipient": "security-researcher",
  "deadline": "2026-10-04T12:00:00Z",
  "input_refs": ["artifact://brief/17"],
  "required_output": "evidence_bundle.v1"
}

The corresponding result should include status, evidence references, confidence or uncertainty where meaningful, and a bounded summary. Keep large documents in an artifact store and pass references. This reduces context cost, preserves provenance, and avoids repeatedly copying untrusted content.

Delivery semantics matter:

• use stable task and attempt IDs;
• make writes idempotent;
• expect duplicate delivery from queues;
• distinguish retryable infrastructure failures from invalid work;
• ignore late results after cancellation or supersession;
• never infer identity from a sender string in model-generated content.

Framework Selection

Frameworks encode different control models. Verify current release documentation and migration notes before pinning one.

Microsoft's current AutoGen documentation distinguishes AgentChat from the older pyautogen/classic API surface. Do not copy a classic autogen.GroupChat example into a new design without first choosing the intended generation. OpenAI's Swarm repository was educational; production work should use the maintained Agents SDK or another supported runtime.

Runnable Go Coordinator

The following standard-library program demonstrates the part frameworks cannot decide for you: bounded parallel assignments, least-privilege tool checks, typed results, cancellation, and deterministic aggregation. Replace mockRunner with your model adapter while preserving the contract.

package main

import (
	"context"
	"errors"
	"fmt"
	"sort"
	"sync"
	"time"
)

type Assignment struct {
	ID            string
	Brief         string
	RequiredTools []string
}

type Evidence struct {
	ID      string
	Source  string
	Summary string
}

type Result struct {
	TaskID   string
	AgentID  string
	Evidence []Evidence
	Err      error
}

type Agent struct {
	ID           string
	AllowedTools map[string]bool
	Run          func(context.Context, Assignment) ([]Evidence, error)
}

type WorkItem struct {
	Agent      Agent
	Assignment Assignment
}

func (a Agent) Execute(
	ctx context.Context,
	assignment Assignment,
) Result {
	for _, tool := range assignment.RequiredTools {
		if !a.AllowedTools[tool] {
			return Result{
				TaskID:  assignment.ID,
				AgentID: a.ID,
				Err:     fmt.Errorf("tool %q is not allowed", tool),
			}
		}
	}

	evidence, err := a.Run(ctx, assignment)
	return Result{
		TaskID:   assignment.ID,
		AgentID:  a.ID,
		Evidence: evidence,
		Err:      err,
	}
}

func runParallel(
	ctx context.Context,
	work []WorkItem,
) ([]Result, error) {
	ctx, cancel := context.WithTimeout(ctx, 2*time.Second)
	defer cancel()

	results := make(chan Result, len(work))
	var waitGroup sync.WaitGroup
	for _, item := range work {
		waitGroup.Add(1)
		go func(agent Agent, assignment Assignment) {
			defer waitGroup.Done()
			results <- agent.Execute(ctx, assignment)
		}(item.Agent, item.Assignment)
	}

	go func() {
		waitGroup.Wait()
		close(results)
	}()

	collected := make([]Result, 0, len(work))
	for result := range results {
		if result.Err != nil {
			cancel()
			return nil, fmt.Errorf(
				"%s failed: %w",
				result.AgentID,
				result.Err,
			)
		}
		if result.TaskID == "" || len(result.Evidence) == 0 {
			cancel()
			return nil, errors.New("invalid worker result")
		}
		collected = append(collected, result)
	}

	sort.Slice(collected, func(i, j int) bool {
		return collected[i].TaskID < collected[j].TaskID
	})
	return collected, nil
}

func mockRunner(
	agentID string,
	source string,
) func(context.Context, Assignment) ([]Evidence, error) {
	return func(ctx context.Context, assignment Assignment) ([]Evidence, error) {
		select {
		case <-ctx.Done():
			return nil, ctx.Err()
		default:
		}
		return []Evidence{{
			ID:      assignment.ID + ":1",
			Source:  source,
			Summary: agentID + " completed: " + assignment.Brief,
		}}, nil
	}
}

func main() {
	marketAgent := Agent{
		ID:           "market-researcher",
		AllowedTools: map[string]bool{"public_search": true},
		Run:          mockRunner("market-researcher", "public://market"),
	}
	riskAgent := Agent{
		ID:           "risk-reviewer",
		AllowedTools: map[string]bool{"policy_read": true},
		Run:          mockRunner("risk-reviewer", "policy://controls"),
	}

	work := []WorkItem{
		{
			Agent: marketAgent,
			Assignment: Assignment{
				ID:            "market",
				Brief:         "collect demand evidence",
				RequiredTools: []string{"public_search"},
			},
		},
		{
			Agent: riskAgent,
			Assignment: Assignment{
				ID:            "risk",
				Brief:         "identify launch controls",
				RequiredTools: []string{"policy_read"},
			},
		},
	}

	results, err := runParallel(context.Background(), work)
	if err != nil {
		panic(err)
	}
	for _, result := range results {
		fmt.Printf("%s: %s\n", result.TaskID, result.Evidence[0].Summary)
	}
}

The example deliberately keeps the agent runner behind a function. A production adapter should also meter model usage, validate structured output, attach trace IDs, redact prompts and results, and convert provider failures into a small internal error taxonomy.

Security Boundaries

Each worker needs a separate capability envelope:

• issue short-lived, audience-bound credentials;
• scope retrieval by tenant and data classification;
• expose only tools required by the assignment;
• authorize the resource after loading it;
• require human approval for externally visible or irreversible writes;
• sanitize tool results before another model sees them;
• prevent one worker from rewriting control-plane instructions.

Agent-to-agent messages are not trusted because both ends are internal. A compromised webpage can inject instructions into a research worker, which then forwards them to a worker with stronger tools. Preserve data/instruction separation and re-authorize every effect at the executor.

Failure Policy

Define behavior before deployment:

Avoid automatic retries for ambiguous writes. A timeout after a charge, email, or deletion may mean the operation succeeded and only the response was lost.

Evaluation and Observability

Evaluate three layers separately:

1. Worker: task completion, evidence precision/recall, tool argument quality, policy compliance.
2. Coordinator: routing accuracy, decomposition quality, duplicate work, retries, cancellation, and budget enforcement.
3. System: business outcome, human correction rate, p50/p95 latency, cost per successful task, and security incidents.

Trace the complete causal graph:

run -> assignment -> model call -> tool call -> artifact
    -> handoff/retry -> review -> final answer

Record versions for prompts, models, tool schemas, policies, and routing logic. Store redacted inputs and outputs under explicit retention rules. A final-answer score alone cannot reveal unnecessary tool use, leaked data, or a reviewer that rubber-stamped unsupported claims.

Use adversarial cases: malicious retrieved instructions, cross-tenant IDs, unavailable workers, duplicate queue delivery, stale evidence, conflicting specialists, and cancellation during a side effect.

Rollout Strategy

1. Establish a single-agent or workflow baseline.
2. Add one specialist at the measured bottleneck.
3. Run in shadow mode without external side effects.
4. Compare quality, latency, cost, and operator interventions.
5. Add durable checkpoints and failure drills.
6. Enable low-risk traffic with a rollback path.
7. Expand only when the observed benefit exceeds coordination cost.

Frequently Asked Questions

Is a manager itself an agent?

It can be, but decomposition and policy enforcement should remain constrained. Many systems use a model to propose a plan and deterministic code to validate task types, dependencies, recipients, tools, and budgets.

Does independent voting remove hallucinations?

No. Workers that share a model, prompt pattern, and source set can make correlated errors. Diversity must come from evidence, methods, or genuinely independent systems, and a verifier still needs a defined rubric.

Should agents share one conversation history?

Usually not. Send the smallest task-specific context and artifact references. Shared full transcripts increase cost, expose unrelated sensitive data, and make responsibility unclear.

Is MCP a multi-agent protocol?

No. MCP standardizes context and capability exchange between hosts, clients, and servers. A coordinator may use MCP tools, but task decomposition, worker scheduling, authority, and multi-agent state remain application concerns.

Summary

Multi-agent architecture is justified by measurable isolation, parallelism, or ownership, not by the number of steps in a prompt. Keep topology and authority in a deterministic control plane, give workers narrow capabilities, exchange typed evidence, and evaluate the full trajectory against a simpler baseline.

For the single-agent foundation, read the production AI agent architecture guide. For deeper coordination patterns, see multi-agent orchestration patterns.

Primary Sources

• Anthropic: Building Effective Agents
• Anthropic: How We Built Our Multi-Agent Research System
• OpenAI Agents SDK: Multi-Agent Orchestration
• Microsoft AutoGen AgentChat Teams
• CrewAI Crews
• LangGraph Overview