TL;DR: An MCP Gateway is an optional proxy or aggregation layer, not an official MCP participant. Under MCP 2026-07-28, modern requests are stateless and can reach any compatible replica; session affinity is a Legacy concern. A production design must validate routing headers against the JSON-RPC body, preserve OAuth audience boundaries, bound concurrency, retry only safe operations, and prove capacity with workload-specific tests.

Key Takeaways

  • The official MCP architecture defines Host, Client, and Server; Gateway is an optional deployment pattern
  • A transparent proxy preserves one Server relationship, while an aggregator becomes a Server downstream and separate Clients upstream
  • MCP 2026-07-28 removes Protocol Sessions from the modern path and adds Mcp-Method and Mcp-Name for infrastructure routing
  • Gateway denial is only one policy layer: every upstream Server still authorizes the exact Principal, Tenant, object, arguments, purpose, and side effect
  • Bounded queues, per-upstream bulkheads, cancellation, idempotency-aware retries, and measured failure tests matter more than a headline connection count

Why You Need an MCP Gateway

An MCP Gateway is justified when shared controls reduce more risk and operational work than the extra hop creates. Direct MCP Client-to-MCP Server connections remain valid and are often simpler for one Host, one Server, or local stdio.

Fleet policy duplication: Multiple Hosts otherwise repeat Server inventory, trust review, protocol-version policy, egress rules, quotas, and telemetry configuration.

Capability collisions: Aggregating tools/list, resources/list, or prompts/list requires stable namespaces and provenance. A bare Tool name is unique only within one Server.

Uneven failure pressure: One slow or expensive Server can consume shared queues, sockets, memory, and retry budgets unless the Gateway isolates each upstream.

Security boundaries across hops: A Gateway can centralize coarse admission and denial, but downstream Servers still need object- and argument-level authorization. It must not pass an inbound bearer token to a different upstream Resource.

Do not deploy a Gateway merely because there are two Servers. The decision should follow measured policy duplication, blast radius, latency budget, availability target, and ownership. The MCP Gateway glossary provides the concise role definition; this guide focuses on the production data path.

Core MCP Gateway Architecture

The safest architecture separates a reviewed control plane from a horizontally scalable data plane. The control plane manages Server identity, policy, descriptor revisions, and rollout; the data plane validates and routes Requests using immutable snapshots.

graph TB subgraph clients_group["Clients"] C1["Claude Desktop"] C2["Cursor IDE"] C3["Custom Agent"] end subgraph Gateway["MCP Gateway"] direction TB Auth["Authentication + coarse policy"] Validate["Version + header/body validation"] Router["Namespace + route lookup"] Limits["Queue + per-upstream bulkhead"] Monitor["Observability"] CB["Circuit Breaker"] end subgraph ServerPool["MCP Server Pool"] S1["Server A: search, fetch"] S2["Server B: db_query, db_write"] S3["Server C: code_run, code_lint"] end C1 -->|"Selected MCP transport"| Auth C2 -->|"Selected MCP transport"| Auth C3 -->|"Selected MCP transport"| Auth Auth --> Validate Validate --> Router Router --> Limits Limits --> CB CB -->|"POST /mcp"| S1 CB -->|"POST /mcp"| S2 CB -->|"POST /mcp"| S3 CB -->|"Health Check"| S1 CB -->|"Health Check"| S2 CB -->|"Health Check"| S3 Monitor -.->|"Collect"| Validate Monitor -.->|"Collect"| Router Monitor -.->|"Collect"| Limits Monitor -.->|"Collect"| CB

For an aggregating Gateway, the downstream connection terminates at the Gateway. It validates the request, resolves a configured upstream, applies admission policy, and creates a separate upstream MCP Request. That is a protocol boundary, not byte-for-byte forwarding:

sequenceDiagram participant Client as MCP Client participant GW as MCP Gateway participant Auth as Auth Module participant Valid as Protocol Validator participant Router as Tool Router participant CB as Circuit Breaker participant Server as MCP Server Client->>GW: POST /mcp
Mcp-Method + Mcp-Name GW->>Auth: Validate JWT Token Auth-->>GW: Principal + Tenant GW->>Valid: Compare headers with JSON-RPC body Valid-->>GW: Version + capability + name GW->>Router: Resolve namespace and policy Router->>CB: Check circuit state alt Circuit Open CB-->>GW: Reject with bounded error GW-->>Client: HTTP / JSON-RPC failure else Circuit Closed/Half-Open CB->>Server: New upstream MCP request Server-->>CB: JSON or request-scoped SSE CB-->>GW: Response GW-->>Client: Validated result end

Route Modern Stateless Requests First

The MCP 2026-07-28 Streamable HTTP specification makes the normal remote path an independent HTTP POST, not a pooled Protocol Session. Every Request carries its protocol version and relevant Client Capabilities in _meta. server/discover is available when a Client wants the Server's versions and capabilities before another operation, but it is not a connection handshake.

Streamable HTTP mirrors selected routing fields into headers:

http
POST /mcp HTTP/1.1
MCP-Protocol-Version: 2026-07-28
Mcp-Method: tools/call
Mcp-Name: finance.payments.create
Content-Type: application/json
Accept: application/json,text/event-stream

{"jsonrpc":"2.0","id":41,"method":"tools/call","params":{"name":"finance.payments.create","arguments":{"invoiceId":"inv_8f2","idempotencyKey":"op_7b16f3a0"},"_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{}}}}

The Gateway may route or meter from Mcp-Method and Mcp-Name, but the JSON-RPC body remains authoritative. A terminating Gateway must reject a mismatched version, method, or name rather than trusting whichever copy is convenient. Mcp-Param-* headers are valid only for primitive Tool properties explicitly marked with x-mcp-header; do not mirror arbitrary arguments or secrets.

Ordinary Requests can use standard keep-alive connection reuse underneath HTTP, but the connection is not a protocol identity, conversation, or authorization context. Round-robin routing is valid when compatible replicas share no hidden application state. If a workflow needs continuity, expose a scoped business handle in the protocol data and authorize it on every Request.

Isolate Legacy Connection Handling

Legacy MCP revisions may still require Initialize, Mcp-Session-Id, GET/SSE, affinity, or stream resumption. Keep that behavior on an explicit versioned route so modern traffic never inherits Legacy state assumptions.

The following Go-shaped pool illustrates the old compatibility problem only. It omits handshake, authentication, parser, heartbeat, cancellation, and shutdown behavior; it is not a current MCP implementation or production-ready component:

go
package gateway

import (
	"context"
	"fmt"
	"net/http"
	"sync"
	"time"
)

type ConnState int

const (
	ConnIdle ConnState = iota
	ConnActive
	ConnDraining
)

type SSEConn struct {
	ID        string
	ServerURL string
	State     ConnState
	CreatedAt time.Time
	LastUsed  time.Time
	mu        sync.Mutex
	client    *http.Client
	eventCh   chan []byte
	closeCh   chan struct{}
}

type ConnPool struct {
	mu          sync.RWMutex
	conns       map[string][]*SSEConn // serverURL -> connections
	maxPerHost  int
	idleTimeout time.Duration
	maxLifetime time.Duration
}

func NewConnPool(maxPerHost int, idleTimeout, maxLifetime time.Duration) *ConnPool {
	pool := &ConnPool{
		conns:       make(map[string][]*SSEConn),
		maxPerHost:  maxPerHost,
		idleTimeout: idleTimeout,
		maxLifetime: maxLifetime,
	}
	go pool.evictLoop()
	return pool
}

func (p *ConnPool) Acquire(ctx context.Context, serverURL string) (*SSEConn, error) {
	p.mu.Lock()
	defer p.mu.Unlock()

	conns := p.conns[serverURL]
	for _, conn := range conns {
		conn.mu.Lock()
		if conn.State == ConnIdle && time.Since(conn.CreatedAt) < p.maxLifetime {
			conn.State = ConnActive
			conn.LastUsed = time.Now()
			conn.mu.Unlock()
			return conn, nil
		}
		conn.mu.Unlock()
	}

	if len(conns) >= p.maxPerHost {
		return nil, fmt.Errorf("connection pool exhausted for %s", serverURL)
	}

	conn, err := p.dial(ctx, serverURL)
	if err != nil {
		return nil, err
	}
	conn.State = ConnActive
	p.conns[serverURL] = append(p.conns[serverURL], conn)
	return conn, nil
}

func (p *ConnPool) Release(conn *SSEConn) {
	conn.mu.Lock()
	defer conn.mu.Unlock()
	conn.State = ConnIdle
	conn.LastUsed = time.Now()
}

func (p *ConnPool) evictLoop() {
	ticker := time.NewTicker(30 * time.Second)
	defer ticker.Stop()

	for range ticker.C {
		p.mu.Lock()
		for url, conns := range p.conns {
			alive := conns[:0]
			for _, conn := range conns {
				conn.mu.Lock()
				expired := conn.State == ConnIdle &&
					(time.Since(conn.LastUsed) > p.idleTimeout ||
						time.Since(conn.CreatedAt) > p.maxLifetime)
				if expired {
					close(conn.closeCh)
					conn.mu.Unlock()
					continue
				}
				conn.mu.Unlock()
				alive = append(alive, conn)
			}
			p.conns[url] = alive
		}
		p.mu.Unlock()
	}
}

func (p *ConnPool) dial(ctx context.Context, serverURL string) (*SSEConn, error) {
	conn := &SSEConn{
		ID:        fmt.Sprintf("conn-%d", time.Now().UnixNano()),
		ServerURL: serverURL,
		CreatedAt: time.Now(),
		LastUsed:  time.Now(),
		client:    &http.Client{Timeout: 0},
		eventCh:   make(chan []byte, 256),
		closeCh:   make(chan struct{}),
	}

	req, err := http.NewRequestWithContext(ctx, "GET", serverURL+"/sse", nil)
	if err != nil {
		return nil, err
	}
	req.Header.Set("Accept", "text/event-stream")

	go conn.readLoop(req)
	return conn, nil
}

func (c *SSEConn) readLoop(req *http.Request) {
	resp, err := c.client.Do(req)
	if err != nil {
		return
	}
	defer resp.Body.Close()

	buf := make([]byte, 4096)
	for {
		select {
		case <-c.closeCh:
			return
		default:
			n, err := resp.Body.Read(buf)
			if err != nil {
				return
			}
			if n > 0 {
				data := make([]byte, n)
				copy(data, buf[:n])
				select {
				case c.eventCh <- data:
				case <-c.closeCh:
					return
				}
			}
		}
	}
}

The pool scans and removes idle Legacy connections. Its values are placeholders, not recommendations. A real compatibility adapter must key state by authenticated Principal, configured Server, negotiated Legacy revision, and authorization context; it must also bound parser memory, event queues, descriptors, redirects, and shutdown time.

The http.Client{Timeout: 0} line demonstrates why this is not copy-paste code. A Legacy stream needs explicit context cancellation, heartbeat and idle limits, connection caps, a tested drain deadline, and the selected revision's reconnect semantics. Modern request-scoped SSE ends with the final response; long-lived change notifications use a separate subscriptions/listen Request and do not support Last-Event-ID resumption.

Request Routing and Load Balancing

The Gateway routes only against a reviewed registry keyed by configured Server identity, not against whichever Server most recently claimed a Tool name. For transparent proxying, the configured endpoint already determines the upstream. For aggregation, the Gateway must namespace duplicate Tool names, build a deterministic composite list, and map every exposed descriptor revision back to one upstream.

The routing strategy operates in three tiers:

  1. Protocol validation: Match MCP-Protocol-Version, Mcp-Method, and Mcp-Name against the body
  2. Namespace and policy lookup: Resolve the exposed capability revision to an allowed upstream Server
  3. Replica selection: Choose a healthy compatible instance within that Server cluster
go
package gateway

import (
	"fmt"
	"hash/crc32"
	"sort"
	"sync"
)

type ServerInfo struct {
	URL     string
	Weight  int
	Tools   []string
	Healthy bool
}

type ToolRouter struct {
	mu       sync.RWMutex
	toolMap  map[string][]*ServerInfo // toolName -> servers
	hashRing *ConsistentHash
}

type ConsistentHash struct {
	ring     map[uint32]*ServerInfo
	keys     []uint32
	replicas int
}

func NewConsistentHash(replicas int) *ConsistentHash {
	return &ConsistentHash{
		ring:     make(map[uint32]*ServerInfo),
		replicas: replicas,
	}
}

func (ch *ConsistentHash) Add(server *ServerInfo) {
	for i := 0; i < ch.replicas; i++ {
		key := crc32.ChecksumIEEE([]byte(fmt.Sprintf("%s-%d", server.URL, i)))
		ch.ring[key] = server
		ch.keys = append(ch.keys, key)
	}
	sort.Slice(ch.keys, func(i, j int) bool { return ch.keys[i] < ch.keys[j] })
}

func (ch *ConsistentHash) Get(key string) *ServerInfo {
	if len(ch.keys) == 0 {
		return nil
	}
	hash := crc32.ChecksumIEEE([]byte(key))
	idx := sort.Search(len(ch.keys), func(i int) bool { return ch.keys[i] >= hash })
	if idx >= len(ch.keys) {
		idx = 0
	}
	return ch.ring[ch.keys[idx]]
}

func NewToolRouter() *ToolRouter {
	return &ToolRouter{
		toolMap:  make(map[string][]*ServerInfo),
		hashRing: NewConsistentHash(150),
	}
}

func (r *ToolRouter) Register(server *ServerInfo) {
	r.mu.Lock()
	defer r.mu.Unlock()

	for _, tool := range server.Tools {
		r.toolMap[tool] = append(r.toolMap[tool], server)
	}
	r.hashRing.Add(server)
}

func (r *ToolRouter) Route(toolName, routingKey string) (*ServerInfo, error) {
	r.mu.RLock()
	defer r.mu.RUnlock()

	servers, ok := r.toolMap[toolName]
	if !ok || len(servers) == 0 {
		return nil, fmt.Errorf("no server registered for tool: %s", toolName)
	}

	healthy := make([]*ServerInfo, 0, len(servers))
	for _, s := range servers {
		if s.Healthy {
			healthy = append(healthy, s)
		}
	}
	if len(healthy) == 0 {
		return nil, fmt.Errorf("all servers for tool %s are unhealthy", toolName)
	}

	if len(healthy) == 1 {
		return healthy[0], nil
	}

	target := r.hashRing.Get(routingKey + ":" + toolName)
	if target != nil && target.Healthy {
		return target, nil
	}
	return healthy[0], nil
}

Consistent hashing is optional. It can reduce remapping for an explicit Tenant, shard, or business-state handle, but a connection or self-reported Client name is not a safe routing identity. Modern Stateless Requests should use ordinary healthy-replica selection unless the application contract actually requires affinity.

Aggregate Capabilities Without Losing Provenance

Aggregation changes the protocol surface and therefore requires its own contract. The Gateway should never concatenate upstream lists and hope names remain unique.

Concern Required Gateway behavior
Server identity Bind routes to configured endpoint or immutable deployment identity; treat serverInfo as display metadata
Name collisions Prefix or map names with a stable Server namespace and reject ambiguous calls
Version support Advertise only revisions the Gateway can validate and faithfully relay or translate
Capabilities Expose only the intersection the full downstream-to-upstream path can support
Discovery cache Key by Principal, Tenant, Server, protocol revision, policy revision, cacheScope, and TTL
Descriptor changes Compare Tool, Resource, and Prompt hashes; re-review consequential changes before exposure
Fan-out Bound parallel discovery, deadlines, bytes, and partial-failure behavior

Notifications are cache invalidation signals, not permission grants. A modern Gateway opens explicit subscriptions/listen streams for the filters it needs, correlates events by Subscription ID, invalidates affected entries, and re-subscribes after disconnect.

Preserve Authorization Across Both Hops

An aggregating Gateway terminates two separate security relationships. On the inbound hop it may act as the protected MCP Resource Server. On each upstream hop it acts as an MCP Client with a token intended for that specific Server.

The MCP authorization specification and security guidance prohibit treating one bearer token as valid across Resource boundaries. Never pass the inbound token through to an upstream MCP Server. Validate issuer and Audience, bind credentials to the RFC 8707 Resource Indicator, request the challenged least-privilege Scope, and keep tokens isolated by Principal, Tenant, issuer, and Server. The Gateway may apply an allowlist or contextual deny decision, but the upstream Server still authorizes the exact object, arguments, purpose, and side effect.

For a tools/call, bind any user approval to the effective Server, descriptor revision, material arguments, destination, and policy revision. Tool annotations and natural-language descriptions are untrusted hints. This division aligns the Gateway with OAuth transport security without turning it into the only authorization authority.

Concurrency Control and Backpressure

In high-concurrency scenarios, without rate control, downstream MCP Servers can easily be overwhelmed by traffic spikes. The Gateway needs to implement two layers of protection: semaphore-based concurrency control + token bucket rate limiting.

go
package gateway

import (
	"context"
	"fmt"
	"sync"
	"time"
)

type RateLimiter struct {
	tokens     chan struct{}
	maxTokens  int
	refillRate time.Duration
	stopCh     chan struct{}
}

func NewRateLimiter(maxTokens int, refillRate time.Duration) *RateLimiter {
	rl := &RateLimiter{
		tokens:     make(chan struct{}, maxTokens),
		maxTokens:  maxTokens,
		refillRate: refillRate,
		stopCh:     make(chan struct{}),
	}
	for i := 0; i < maxTokens; i++ {
		rl.tokens <- struct{}{}
	}
	go rl.refill()
	return rl
}

func (rl *RateLimiter) refill() {
	ticker := time.NewTicker(rl.refillRate)
	defer ticker.Stop()

	for {
		select {
		case <-rl.stopCh:
			return
		case <-ticker.C:
			select {
			case rl.tokens <- struct{}{}:
			default:
			}
		}
	}
}

func (rl *RateLimiter) Allow(ctx context.Context) bool {
	select {
	case <-rl.tokens:
		return true
	case <-ctx.Done():
		return false
	}
}

type BackpressureController struct {
	semaphore   chan struct{}
	rateLimiter *RateLimiter
	queueSize   int64
	mu          sync.Mutex
	metrics     *BackpressureMetrics
}

type BackpressureMetrics struct {
	Accepted int64
	Rejected int64
	Queued   int64
}

func NewBackpressureController(maxConcurrent, maxRPS int) *BackpressureController {
	return &BackpressureController{
		semaphore:   make(chan struct{}, maxConcurrent),
		rateLimiter: NewRateLimiter(maxRPS, time.Second/time.Duration(maxRPS)),
		metrics:     &BackpressureMetrics{},
	}
}

func (bp *BackpressureController) Execute(
	ctx context.Context,
	fn func(context.Context) (any, error),
) (any, error) {
	if !bp.rateLimiter.Allow(ctx) {
		bp.mu.Lock()
		bp.metrics.Rejected++
		bp.mu.Unlock()
		return nil, fmt.Errorf("rate limit exceeded")
	}

	select {
	case bp.semaphore <- struct{}{}:
		defer func() { <-bp.semaphore }()
	case <-ctx.Done():
		bp.mu.Lock()
		bp.metrics.Rejected++
		bp.mu.Unlock()
		return nil, ctx.Err()
	}

	bp.mu.Lock()
	bp.metrics.Accepted++
	bp.mu.Unlock()

	return fn(ctx)
}

The token bucket limits admission rate, while the semaphore caps work already admitted to one protected pool. This excerpt is intentionally small: production code must reject invalid limits, stop the refill goroutine, expose queue deadlines, partition limits by Tenant and upstream, and avoid one global semaphore that lets a slow Server block unrelated traffic. A finite queue may absorb a measured burst, but an unbounded queue only converts overload into latency and memory pressure.

Separate Stateless Routing from Application State

Modern MCP horizontal scaling does not require a shared Protocol Session store. Each 2026-07-28 Request carries the version and relevant Client Capabilities needed to process it, so any compatible Gateway replica can validate and route it. HTTP keep-alive, request-scoped SSE, and a subscriptions/listen connection are transport resources, not durable conversation identity.

Some applications still need continuity, but that state must be explicit:

State Correct owner and key Scaling rule
Workflow or job Application or upstream Server, keyed by an opaque scoped handle Authorize the handle on every Request; define TTL, concurrency, and replay semantics
MRTR continuation Upstream Server, represented by opaque requestState Preserve Principal and Server binding; never interpret it or use it as permission
Discovery cache Gateway, keyed by identity, authorization context, Server, version, policy revision, and cacheScope Honor TTL and invalidation; private entries must never cross Principals or Tenants
Subscription stream The Gateway replica that opened subscriptions/listen Re-subscribe after disconnect; deduplicate by Subscription and Event ID
Legacy Session Isolated compatibility adapter Follow that revision's Session ID, affinity, expiry, ordering, and reconnect contract

Redis can back explicit application state or a Legacy registry, but it is not a modern MCP requirement. Redis Pub/Sub alone is also not durable delivery: if loss, ordering, or replay matters, choose a store or broker whose guarantees match the application contract. Never key authorization or cache access solely by a connection ID, Request ID, self-reported clientInfo, or caller-provided business handle.

Observability and Monitoring

When debugging MCP requests, validate JSON-RPC framing with protocol tests and the selected SDK. A production gateway should expose bounded Metrics, structured Logs, and distributed Traces without recording raw credentials or unbounded tool payloads.

For core metrics collection, prefer bounded-cardinality labels. A raw Principal, Request ID, URL, argument, or unbounded Tool name belongs in sampled and redacted traces or logs, not in a Prometheus label:

Metric Name Type Description
mcp_gateway_requests_total Counter Accepted, queued, rejected, cancelled, retried, and completed Requests by version, method class, route, and outcome
mcp_gateway_queue_duration_seconds Histogram Time waiting for admission by route and upstream class
mcp_gateway_service_duration_seconds Histogram Gateway processing time excluding queue and upstream work
mcp_gateway_upstream_duration_seconds Histogram Upstream latency by stable Server identity and outcome
mcp_gateway_streams_active Gauge Active request-scoped response streams and subscription streams by stream class
mcp_gateway_cache_operations_total Counter Hit, miss, bypass, expiry, and invalidation decisions by cache class
mcp_gateway_circuit_state Gauge Circuit state by stable upstream identity
mcp_gateway_effect_outcomes_total Counter Known success, known failure, and unknown write effect status

For distributed tracing, use OpenTelemetry to continue valid inbound context or create a new trace, then create separate admission, routing, authorization, upstream, and streaming spans. Record the Gateway and policy revision, protocol version, stable route and Server identity, descriptor hash, Request correlation, retry and idempotency decisions, and final effect status. Do not assume every upstream preserves trace context; correlate MCP Request IDs separately.

Audit logs should capture the authenticated Principal and Tenant, target Server and namespaced capability, descriptor and policy revision, authorization decision, redacted argument digest, response type, bytes, cancellation, and outcome. Record identity from validated credentials, not merely by decoding a JWT. Access tokens, raw secrets, complete prompts, and sensitive Tool or Resource payloads must remain absent.

Production-Grade Fault Tolerance

MCP Servers can become temporarily unavailable due to deployment updates, resource exhaustion, or network partitions. The Gateway must implement the Circuit Breaker pattern to isolate failures and prevent cascading outages.

go
package gateway

import (
	"fmt"
	"sync"
	"time"
)

type CircuitState int

const (
	StateClosed   CircuitState = iota
	StateOpen
	StateHalfOpen
)

type CircuitBreaker struct {
	mu               sync.Mutex
	state            CircuitState
	failureCount     int
	successCount     int
	failureThreshold int
	successThreshold int
	timeout          time.Duration
	lastFailureTime  time.Time
	onStateChange    func(from, to CircuitState)
}

type CircuitBreakerConfig struct {
	FailureThreshold int
	SuccessThreshold int
	Timeout          time.Duration
	OnStateChange    func(from, to CircuitState)
}

func NewCircuitBreaker(cfg CircuitBreakerConfig) *CircuitBreaker {
	return &CircuitBreaker{
		state:            StateClosed,
		failureThreshold: cfg.FailureThreshold,
		successThreshold: cfg.SuccessThreshold,
		timeout:          cfg.Timeout,
		onStateChange:    cfg.OnStateChange,
	}
}

func (cb *CircuitBreaker) Allow() (bool, error) {
	cb.mu.Lock()
	defer cb.mu.Unlock()

	switch cb.state {
	case StateClosed:
		return true, nil
	case StateOpen:
		if time.Since(cb.lastFailureTime) > cb.timeout {
			cb.transitionTo(StateHalfOpen)
			return true, nil
		}
		return false, fmt.Errorf("circuit breaker is open")
	case StateHalfOpen:
		return true, nil
	}
	return false, fmt.Errorf("unknown circuit state")
}

func (cb *CircuitBreaker) RecordSuccess() {
	cb.mu.Lock()
	defer cb.mu.Unlock()

	switch cb.state {
	case StateClosed:
		cb.failureCount = 0
	case StateHalfOpen:
		cb.successCount++
		if cb.successCount >= cb.successThreshold {
			cb.transitionTo(StateClosed)
		}
	}
}

func (cb *CircuitBreaker) RecordFailure() {
	cb.mu.Lock()
	defer cb.mu.Unlock()

	cb.lastFailureTime = time.Now()

	switch cb.state {
	case StateClosed:
		cb.failureCount++
		if cb.failureCount >= cb.failureThreshold {
			cb.transitionTo(StateOpen)
		}
	case StateHalfOpen:
		cb.transitionTo(StateOpen)
	}
}

func (cb *CircuitBreaker) transitionTo(newState CircuitState) {
	oldState := cb.state
	cb.state = newState
	cb.failureCount = 0
	cb.successCount = 0

	if cb.onStateChange != nil {
		cb.onStateChange(oldState, newState)
	}
}

func (cb *CircuitBreaker) State() CircuitState {
	cb.mu.Lock()
	defer cb.mu.Unlock()
	return cb.state
}

The circuit breaker state machine contains three states: Closed (normal pass-through while measuring failures) → Open (fail fast for that upstream) → Half-Open (admit a bounded probe budget). The compact example illustrates transitions but does not limit concurrent half-open probes, classify failures, persist state, or coordinate replicas; those behaviors must be added before production use. Never let one upstream circuit become a fleet-wide global circuit.

Retry policy follows MCP operation semantics, not HTTP POST alone:

  • Retry discovery and reads only within their deadline, freshness, authorization, and consistency contract
  • Retry a side-effecting tools/call only when the upstream enforces an idempotency key or transactional deduplication
  • Preserve the same logical operation key across attempts, but use correct per-attempt transport correlation
  • Treat timeout, cancellation, connection loss, and Circuit Open as an unknown effect unless the Server can prove the outcome
  • Use exponential backoff with jitter and a total attempt budget; retries must re-enter admission control

A cached Resource or read result may be returned only when its cacheScope, TTL, authorization context, and freshness policy allow it, and the response must disclose that it is cached. Never present historical Tool output as if a new Tool execution succeeded. If the effect is unknown, return a bounded error that preserves uncertainty; choosing an alternative Tool is a new, separately authorized operation and may duplicate the original side effect.

FAQ

Q: What is the difference between an MCP Gateway and an API Gateway? A: Both can terminate TLS, authenticate callers, route HTTP, rate-limit, and emit telemetry. An MCP-aware Gateway additionally validates MCP versions and routing headers, understands JSON-RPC methods, capability discovery, result envelopes, MRTR, and subscription streams. An existing API Gateway may be enough when those controls are unnecessary.

Q: How many concurrent connections can a single MCP Gateway node support? A: There is no portable number. Ordinary 2026-07-28 Requests are independent POSTs, while request-scoped SSE and subscriptions consume longer-lived resources. Measure request rate, concurrent streams, bytes, queue time, p95/p99 latency, downstream saturation, cancellation, and recovery on the target hardware.

Q: Does MCP 2026-07-28 require sticky sessions or a shared Session store? A: No. The modern Core has no Protocol Session, Initialize Handshake, GET Stream, or stream resumption. A self-describing Request can reach any compatible instance. Affinity or shared state is needed only for an isolated Legacy route or explicit application state identified by a scoped handle.

Q: How much latency does the Gateway layer add? A: There is no portable latency number. Separate queue, Gateway service, upstream, and stream duration; include token validation, policy lookup, serialization, payload size, observability, retries, and cache decisions. Report p50/p95/p99 on the target transport and hardware rather than assuming the extra hop is negligible.

Q: How do you smoothly migrate existing MCP Servers to the Gateway architecture? A: Start by choosing transparent or aggregating behavior. Register immutable Server identity, verify revision and transport support, namespace and hash descriptors, configure separate OAuth audiences and credentials, and test authorization and failure behavior. Move a small Client cohort through an explicit route, keep Legacy traffic isolated, and retire direct access only after rollback, duplicate-effect, cancellation, and recovery tests pass.

Summary

An MCP Gateway is justified when shared routing, capability governance, OAuth mediation, backpressure, and observability outweigh an extra hop and a larger blast radius. For MCP 2026-07-28, keep the normal data plane Stateless, isolate Legacy Session behavior, preserve Server authorization, and treat aggregation as a new protocol surface with explicit identity and namespace rules. Capacity, retry safety, and recovery must be demonstrated with workload-specific tests.

For the underlying participant and transport model, see the MCP Protocol Complete Guide. The Advanced MCP Protocol Practice covers Server-side implementation concerns that remain authoritative behind a Gateway.