TL;DR: An MCP Gateway is an optional proxy or aggregation layer, not an official MCP participant. Under MCP 2026-07-28, modern requests are stateless and can reach any compatible replica; session affinity is a Legacy concern. A production design must validate routing headers against the JSON-RPC body, preserve OAuth audience boundaries, bound concurrency, retry only safe operations, and prove capacity with workload-specific tests.
Key Takeaways
- The official MCP architecture defines Host, Client, and Server; Gateway is an optional deployment pattern
- A transparent proxy preserves one Server relationship, while an aggregator becomes a Server downstream and separate Clients upstream
- MCP 2026-07-28 removes Protocol Sessions from the modern path and adds
Mcp-MethodandMcp-Namefor infrastructure routing - Gateway denial is only one policy layer: every upstream Server still authorizes the exact Principal, Tenant, object, arguments, purpose, and side effect
- Bounded queues, per-upstream bulkheads, cancellation, idempotency-aware retries, and measured failure tests matter more than a headline connection count
Why You Need an MCP Gateway
An MCP Gateway is justified when shared controls reduce more risk and operational work than the extra hop creates. Direct MCP Client-to-MCP Server connections remain valid and are often simpler for one Host, one Server, or local stdio.
Fleet policy duplication: Multiple Hosts otherwise repeat Server inventory, trust review, protocol-version policy, egress rules, quotas, and telemetry configuration.
Capability collisions: Aggregating tools/list, resources/list, or prompts/list requires stable namespaces and provenance. A bare Tool name is unique only within one Server.
Uneven failure pressure: One slow or expensive Server can consume shared queues, sockets, memory, and retry budgets unless the Gateway isolates each upstream.
Security boundaries across hops: A Gateway can centralize coarse admission and denial, but downstream Servers still need object- and argument-level authorization. It must not pass an inbound bearer token to a different upstream Resource.
Do not deploy a Gateway merely because there are two Servers. The decision should follow measured policy duplication, blast radius, latency budget, availability target, and ownership. The MCP Gateway glossary provides the concise role definition; this guide focuses on the production data path.
Core MCP Gateway Architecture
The safest architecture separates a reviewed control plane from a horizontally scalable data plane. The control plane manages Server identity, policy, descriptor revisions, and rollout; the data plane validates and routes Requests using immutable snapshots.
For an aggregating Gateway, the downstream connection terminates at the Gateway. It validates the request, resolves a configured upstream, applies admission policy, and creates a separate upstream MCP Request. That is a protocol boundary, not byte-for-byte forwarding:
Mcp-Method + Mcp-Name GW->>Auth: Validate JWT Token Auth-->>GW: Principal + Tenant GW->>Valid: Compare headers with JSON-RPC body Valid-->>GW: Version + capability + name GW->>Router: Resolve namespace and policy Router->>CB: Check circuit state alt Circuit Open CB-->>GW: Reject with bounded error GW-->>Client: HTTP / JSON-RPC failure else Circuit Closed/Half-Open CB->>Server: New upstream MCP request Server-->>CB: JSON or request-scoped SSE CB-->>GW: Response GW-->>Client: Validated result end
Route Modern Stateless Requests First
The MCP 2026-07-28 Streamable HTTP specification makes the normal remote path an independent HTTP POST, not a pooled Protocol Session. Every Request carries its protocol version and relevant Client Capabilities in _meta. server/discover is available when a Client wants the Server's versions and capabilities before another operation, but it is not a connection handshake.
Streamable HTTP mirrors selected routing fields into headers:
POST /mcp HTTP/1.1
MCP-Protocol-Version: 2026-07-28
Mcp-Method: tools/call
Mcp-Name: finance.payments.create
Content-Type: application/json
Accept: application/json,text/event-stream
{"jsonrpc":"2.0","id":41,"method":"tools/call","params":{"name":"finance.payments.create","arguments":{"invoiceId":"inv_8f2","idempotencyKey":"op_7b16f3a0"},"_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{}}}}
The Gateway may route or meter from Mcp-Method and Mcp-Name, but the JSON-RPC body remains authoritative. A terminating Gateway must reject a mismatched version, method, or name rather than trusting whichever copy is convenient. Mcp-Param-* headers are valid only for primitive Tool properties explicitly marked with x-mcp-header; do not mirror arbitrary arguments or secrets.
Ordinary Requests can use standard keep-alive connection reuse underneath HTTP, but the connection is not a protocol identity, conversation, or authorization context. Round-robin routing is valid when compatible replicas share no hidden application state. If a workflow needs continuity, expose a scoped business handle in the protocol data and authorize it on every Request.
Isolate Legacy Connection Handling
Legacy MCP revisions may still require Initialize, Mcp-Session-Id, GET/SSE, affinity, or stream resumption. Keep that behavior on an explicit versioned route so modern traffic never inherits Legacy state assumptions.
The following Go-shaped pool illustrates the old compatibility problem only. It omits handshake, authentication, parser, heartbeat, cancellation, and shutdown behavior; it is not a current MCP implementation or production-ready component:
package gateway
import (
"context"
"fmt"
"net/http"
"sync"
"time"
)
type ConnState int
const (
ConnIdle ConnState = iota
ConnActive
ConnDraining
)
type SSEConn struct {
ID string
ServerURL string
State ConnState
CreatedAt time.Time
LastUsed time.Time
mu sync.Mutex
client *http.Client
eventCh chan []byte
closeCh chan struct{}
}
type ConnPool struct {
mu sync.RWMutex
conns map[string][]*SSEConn // serverURL -> connections
maxPerHost int
idleTimeout time.Duration
maxLifetime time.Duration
}
func NewConnPool(maxPerHost int, idleTimeout, maxLifetime time.Duration) *ConnPool {
pool := &ConnPool{
conns: make(map[string][]*SSEConn),
maxPerHost: maxPerHost,
idleTimeout: idleTimeout,
maxLifetime: maxLifetime,
}
go pool.evictLoop()
return pool
}
func (p *ConnPool) Acquire(ctx context.Context, serverURL string) (*SSEConn, error) {
p.mu.Lock()
defer p.mu.Unlock()
conns := p.conns[serverURL]
for _, conn := range conns {
conn.mu.Lock()
if conn.State == ConnIdle && time.Since(conn.CreatedAt) < p.maxLifetime {
conn.State = ConnActive
conn.LastUsed = time.Now()
conn.mu.Unlock()
return conn, nil
}
conn.mu.Unlock()
}
if len(conns) >= p.maxPerHost {
return nil, fmt.Errorf("connection pool exhausted for %s", serverURL)
}
conn, err := p.dial(ctx, serverURL)
if err != nil {
return nil, err
}
conn.State = ConnActive
p.conns[serverURL] = append(p.conns[serverURL], conn)
return conn, nil
}
func (p *ConnPool) Release(conn *SSEConn) {
conn.mu.Lock()
defer conn.mu.Unlock()
conn.State = ConnIdle
conn.LastUsed = time.Now()
}
func (p *ConnPool) evictLoop() {
ticker := time.NewTicker(30 * time.Second)
defer ticker.Stop()
for range ticker.C {
p.mu.Lock()
for url, conns := range p.conns {
alive := conns[:0]
for _, conn := range conns {
conn.mu.Lock()
expired := conn.State == ConnIdle &&
(time.Since(conn.LastUsed) > p.idleTimeout ||
time.Since(conn.CreatedAt) > p.maxLifetime)
if expired {
close(conn.closeCh)
conn.mu.Unlock()
continue
}
conn.mu.Unlock()
alive = append(alive, conn)
}
p.conns[url] = alive
}
p.mu.Unlock()
}
}
func (p *ConnPool) dial(ctx context.Context, serverURL string) (*SSEConn, error) {
conn := &SSEConn{
ID: fmt.Sprintf("conn-%d", time.Now().UnixNano()),
ServerURL: serverURL,
CreatedAt: time.Now(),
LastUsed: time.Now(),
client: &http.Client{Timeout: 0},
eventCh: make(chan []byte, 256),
closeCh: make(chan struct{}),
}
req, err := http.NewRequestWithContext(ctx, "GET", serverURL+"/sse", nil)
if err != nil {
return nil, err
}
req.Header.Set("Accept", "text/event-stream")
go conn.readLoop(req)
return conn, nil
}
func (c *SSEConn) readLoop(req *http.Request) {
resp, err := c.client.Do(req)
if err != nil {
return
}
defer resp.Body.Close()
buf := make([]byte, 4096)
for {
select {
case <-c.closeCh:
return
default:
n, err := resp.Body.Read(buf)
if err != nil {
return
}
if n > 0 {
data := make([]byte, n)
copy(data, buf[:n])
select {
case c.eventCh <- data:
case <-c.closeCh:
return
}
}
}
}
}
The pool scans and removes idle Legacy connections. Its values are placeholders, not recommendations. A real compatibility adapter must key state by authenticated Principal, configured Server, negotiated Legacy revision, and authorization context; it must also bound parser memory, event queues, descriptors, redirects, and shutdown time.
The http.Client{Timeout: 0} line demonstrates why this is not copy-paste code. A Legacy stream needs explicit context cancellation, heartbeat and idle limits, connection caps, a tested drain deadline, and the selected revision's reconnect semantics. Modern request-scoped SSE ends with the final response; long-lived change notifications use a separate subscriptions/listen Request and do not support Last-Event-ID resumption.
Request Routing and Load Balancing
The Gateway routes only against a reviewed registry keyed by configured Server identity, not against whichever Server most recently claimed a Tool name. For transparent proxying, the configured endpoint already determines the upstream. For aggregation, the Gateway must namespace duplicate Tool names, build a deterministic composite list, and map every exposed descriptor revision back to one upstream.
The routing strategy operates in three tiers:
- Protocol validation: Match
MCP-Protocol-Version,Mcp-Method, andMcp-Nameagainst the body - Namespace and policy lookup: Resolve the exposed capability revision to an allowed upstream Server
- Replica selection: Choose a healthy compatible instance within that Server cluster
package gateway
import (
"fmt"
"hash/crc32"
"sort"
"sync"
)
type ServerInfo struct {
URL string
Weight int
Tools []string
Healthy bool
}
type ToolRouter struct {
mu sync.RWMutex
toolMap map[string][]*ServerInfo // toolName -> servers
hashRing *ConsistentHash
}
type ConsistentHash struct {
ring map[uint32]*ServerInfo
keys []uint32
replicas int
}
func NewConsistentHash(replicas int) *ConsistentHash {
return &ConsistentHash{
ring: make(map[uint32]*ServerInfo),
replicas: replicas,
}
}
func (ch *ConsistentHash) Add(server *ServerInfo) {
for i := 0; i < ch.replicas; i++ {
key := crc32.ChecksumIEEE([]byte(fmt.Sprintf("%s-%d", server.URL, i)))
ch.ring[key] = server
ch.keys = append(ch.keys, key)
}
sort.Slice(ch.keys, func(i, j int) bool { return ch.keys[i] < ch.keys[j] })
}
func (ch *ConsistentHash) Get(key string) *ServerInfo {
if len(ch.keys) == 0 {
return nil
}
hash := crc32.ChecksumIEEE([]byte(key))
idx := sort.Search(len(ch.keys), func(i int) bool { return ch.keys[i] >= hash })
if idx >= len(ch.keys) {
idx = 0
}
return ch.ring[ch.keys[idx]]
}
func NewToolRouter() *ToolRouter {
return &ToolRouter{
toolMap: make(map[string][]*ServerInfo),
hashRing: NewConsistentHash(150),
}
}
func (r *ToolRouter) Register(server *ServerInfo) {
r.mu.Lock()
defer r.mu.Unlock()
for _, tool := range server.Tools {
r.toolMap[tool] = append(r.toolMap[tool], server)
}
r.hashRing.Add(server)
}
func (r *ToolRouter) Route(toolName, routingKey string) (*ServerInfo, error) {
r.mu.RLock()
defer r.mu.RUnlock()
servers, ok := r.toolMap[toolName]
if !ok || len(servers) == 0 {
return nil, fmt.Errorf("no server registered for tool: %s", toolName)
}
healthy := make([]*ServerInfo, 0, len(servers))
for _, s := range servers {
if s.Healthy {
healthy = append(healthy, s)
}
}
if len(healthy) == 0 {
return nil, fmt.Errorf("all servers for tool %s are unhealthy", toolName)
}
if len(healthy) == 1 {
return healthy[0], nil
}
target := r.hashRing.Get(routingKey + ":" + toolName)
if target != nil && target.Healthy {
return target, nil
}
return healthy[0], nil
}
Consistent hashing is optional. It can reduce remapping for an explicit Tenant, shard, or business-state handle, but a connection or self-reported Client name is not a safe routing identity. Modern Stateless Requests should use ordinary healthy-replica selection unless the application contract actually requires affinity.
Aggregate Capabilities Without Losing Provenance
Aggregation changes the protocol surface and therefore requires its own contract. The Gateway should never concatenate upstream lists and hope names remain unique.
| Concern | Required Gateway behavior |
|---|---|
| Server identity | Bind routes to configured endpoint or immutable deployment identity; treat serverInfo as display metadata |
| Name collisions | Prefix or map names with a stable Server namespace and reject ambiguous calls |
| Version support | Advertise only revisions the Gateway can validate and faithfully relay or translate |
| Capabilities | Expose only the intersection the full downstream-to-upstream path can support |
| Discovery cache | Key by Principal, Tenant, Server, protocol revision, policy revision, cacheScope, and TTL |
| Descriptor changes | Compare Tool, Resource, and Prompt hashes; re-review consequential changes before exposure |
| Fan-out | Bound parallel discovery, deadlines, bytes, and partial-failure behavior |
Notifications are cache invalidation signals, not permission grants. A modern Gateway opens explicit subscriptions/listen streams for the filters it needs, correlates events by Subscription ID, invalidates affected entries, and re-subscribes after disconnect.
Preserve Authorization Across Both Hops
An aggregating Gateway terminates two separate security relationships. On the inbound hop it may act as the protected MCP Resource Server. On each upstream hop it acts as an MCP Client with a token intended for that specific Server.
The MCP authorization specification and security guidance prohibit treating one bearer token as valid across Resource boundaries. Never pass the inbound token through to an upstream MCP Server. Validate issuer and Audience, bind credentials to the RFC 8707 Resource Indicator, request the challenged least-privilege Scope, and keep tokens isolated by Principal, Tenant, issuer, and Server. The Gateway may apply an allowlist or contextual deny decision, but the upstream Server still authorizes the exact object, arguments, purpose, and side effect.
For a tools/call, bind any user approval to the effective Server, descriptor revision, material arguments, destination, and policy revision. Tool annotations and natural-language descriptions are untrusted hints. This division aligns the Gateway with OAuth transport security without turning it into the only authorization authority.
Concurrency Control and Backpressure
In high-concurrency scenarios, without rate control, downstream MCP Servers can easily be overwhelmed by traffic spikes. The Gateway needs to implement two layers of protection: semaphore-based concurrency control + token bucket rate limiting.
package gateway
import (
"context"
"fmt"
"sync"
"time"
)
type RateLimiter struct {
tokens chan struct{}
maxTokens int
refillRate time.Duration
stopCh chan struct{}
}
func NewRateLimiter(maxTokens int, refillRate time.Duration) *RateLimiter {
rl := &RateLimiter{
tokens: make(chan struct{}, maxTokens),
maxTokens: maxTokens,
refillRate: refillRate,
stopCh: make(chan struct{}),
}
for i := 0; i < maxTokens; i++ {
rl.tokens <- struct{}{}
}
go rl.refill()
return rl
}
func (rl *RateLimiter) refill() {
ticker := time.NewTicker(rl.refillRate)
defer ticker.Stop()
for {
select {
case <-rl.stopCh:
return
case <-ticker.C:
select {
case rl.tokens <- struct{}{}:
default:
}
}
}
}
func (rl *RateLimiter) Allow(ctx context.Context) bool {
select {
case <-rl.tokens:
return true
case <-ctx.Done():
return false
}
}
type BackpressureController struct {
semaphore chan struct{}
rateLimiter *RateLimiter
queueSize int64
mu sync.Mutex
metrics *BackpressureMetrics
}
type BackpressureMetrics struct {
Accepted int64
Rejected int64
Queued int64
}
func NewBackpressureController(maxConcurrent, maxRPS int) *BackpressureController {
return &BackpressureController{
semaphore: make(chan struct{}, maxConcurrent),
rateLimiter: NewRateLimiter(maxRPS, time.Second/time.Duration(maxRPS)),
metrics: &BackpressureMetrics{},
}
}
func (bp *BackpressureController) Execute(
ctx context.Context,
fn func(context.Context) (any, error),
) (any, error) {
if !bp.rateLimiter.Allow(ctx) {
bp.mu.Lock()
bp.metrics.Rejected++
bp.mu.Unlock()
return nil, fmt.Errorf("rate limit exceeded")
}
select {
case bp.semaphore <- struct{}{}:
defer func() { <-bp.semaphore }()
case <-ctx.Done():
bp.mu.Lock()
bp.metrics.Rejected++
bp.mu.Unlock()
return nil, ctx.Err()
}
bp.mu.Lock()
bp.metrics.Accepted++
bp.mu.Unlock()
return fn(ctx)
}
The token bucket limits admission rate, while the semaphore caps work already admitted to one protected pool. This excerpt is intentionally small: production code must reject invalid limits, stop the refill goroutine, expose queue deadlines, partition limits by Tenant and upstream, and avoid one global semaphore that lets a slow Server block unrelated traffic. A finite queue may absorb a measured burst, but an unbounded queue only converts overload into latency and memory pressure.
Separate Stateless Routing from Application State
Modern MCP horizontal scaling does not require a shared Protocol Session store. Each 2026-07-28 Request carries the version and relevant Client Capabilities needed to process it, so any compatible Gateway replica can validate and route it. HTTP keep-alive, request-scoped SSE, and a subscriptions/listen connection are transport resources, not durable conversation identity.
Some applications still need continuity, but that state must be explicit:
| State | Correct owner and key | Scaling rule |
|---|---|---|
| Workflow or job | Application or upstream Server, keyed by an opaque scoped handle | Authorize the handle on every Request; define TTL, concurrency, and replay semantics |
| MRTR continuation | Upstream Server, represented by opaque requestState |
Preserve Principal and Server binding; never interpret it or use it as permission |
| Discovery cache | Gateway, keyed by identity, authorization context, Server, version, policy revision, and cacheScope |
Honor TTL and invalidation; private entries must never cross Principals or Tenants |
| Subscription stream | The Gateway replica that opened subscriptions/listen |
Re-subscribe after disconnect; deduplicate by Subscription and Event ID |
| Legacy Session | Isolated compatibility adapter | Follow that revision's Session ID, affinity, expiry, ordering, and reconnect contract |
Redis can back explicit application state or a Legacy registry, but it is not a modern MCP requirement. Redis Pub/Sub alone is also not durable delivery: if loss, ordering, or replay matters, choose a store or broker whose guarantees match the application contract. Never key authorization or cache access solely by a connection ID, Request ID, self-reported clientInfo, or caller-provided business handle.
Observability and Monitoring
When debugging MCP requests, validate JSON-RPC framing with protocol tests and the selected SDK. A production gateway should expose bounded Metrics, structured Logs, and distributed Traces without recording raw credentials or unbounded tool payloads.
For core metrics collection, prefer bounded-cardinality labels. A raw Principal, Request ID, URL, argument, or unbounded Tool name belongs in sampled and redacted traces or logs, not in a Prometheus label:
| Metric Name | Type | Description |
|---|---|---|
mcp_gateway_requests_total |
Counter | Accepted, queued, rejected, cancelled, retried, and completed Requests by version, method class, route, and outcome |
mcp_gateway_queue_duration_seconds |
Histogram | Time waiting for admission by route and upstream class |
mcp_gateway_service_duration_seconds |
Histogram | Gateway processing time excluding queue and upstream work |
mcp_gateway_upstream_duration_seconds |
Histogram | Upstream latency by stable Server identity and outcome |
mcp_gateway_streams_active |
Gauge | Active request-scoped response streams and subscription streams by stream class |
mcp_gateway_cache_operations_total |
Counter | Hit, miss, bypass, expiry, and invalidation decisions by cache class |
mcp_gateway_circuit_state |
Gauge | Circuit state by stable upstream identity |
mcp_gateway_effect_outcomes_total |
Counter | Known success, known failure, and unknown write effect status |
For distributed tracing, use OpenTelemetry to continue valid inbound context or create a new trace, then create separate admission, routing, authorization, upstream, and streaming spans. Record the Gateway and policy revision, protocol version, stable route and Server identity, descriptor hash, Request correlation, retry and idempotency decisions, and final effect status. Do not assume every upstream preserves trace context; correlate MCP Request IDs separately.
Audit logs should capture the authenticated Principal and Tenant, target Server and namespaced capability, descriptor and policy revision, authorization decision, redacted argument digest, response type, bytes, cancellation, and outcome. Record identity from validated credentials, not merely by decoding a JWT. Access tokens, raw secrets, complete prompts, and sensitive Tool or Resource payloads must remain absent.
Production-Grade Fault Tolerance
MCP Servers can become temporarily unavailable due to deployment updates, resource exhaustion, or network partitions. The Gateway must implement the Circuit Breaker pattern to isolate failures and prevent cascading outages.
package gateway
import (
"fmt"
"sync"
"time"
)
type CircuitState int
const (
StateClosed CircuitState = iota
StateOpen
StateHalfOpen
)
type CircuitBreaker struct {
mu sync.Mutex
state CircuitState
failureCount int
successCount int
failureThreshold int
successThreshold int
timeout time.Duration
lastFailureTime time.Time
onStateChange func(from, to CircuitState)
}
type CircuitBreakerConfig struct {
FailureThreshold int
SuccessThreshold int
Timeout time.Duration
OnStateChange func(from, to CircuitState)
}
func NewCircuitBreaker(cfg CircuitBreakerConfig) *CircuitBreaker {
return &CircuitBreaker{
state: StateClosed,
failureThreshold: cfg.FailureThreshold,
successThreshold: cfg.SuccessThreshold,
timeout: cfg.Timeout,
onStateChange: cfg.OnStateChange,
}
}
func (cb *CircuitBreaker) Allow() (bool, error) {
cb.mu.Lock()
defer cb.mu.Unlock()
switch cb.state {
case StateClosed:
return true, nil
case StateOpen:
if time.Since(cb.lastFailureTime) > cb.timeout {
cb.transitionTo(StateHalfOpen)
return true, nil
}
return false, fmt.Errorf("circuit breaker is open")
case StateHalfOpen:
return true, nil
}
return false, fmt.Errorf("unknown circuit state")
}
func (cb *CircuitBreaker) RecordSuccess() {
cb.mu.Lock()
defer cb.mu.Unlock()
switch cb.state {
case StateClosed:
cb.failureCount = 0
case StateHalfOpen:
cb.successCount++
if cb.successCount >= cb.successThreshold {
cb.transitionTo(StateClosed)
}
}
}
func (cb *CircuitBreaker) RecordFailure() {
cb.mu.Lock()
defer cb.mu.Unlock()
cb.lastFailureTime = time.Now()
switch cb.state {
case StateClosed:
cb.failureCount++
if cb.failureCount >= cb.failureThreshold {
cb.transitionTo(StateOpen)
}
case StateHalfOpen:
cb.transitionTo(StateOpen)
}
}
func (cb *CircuitBreaker) transitionTo(newState CircuitState) {
oldState := cb.state
cb.state = newState
cb.failureCount = 0
cb.successCount = 0
if cb.onStateChange != nil {
cb.onStateChange(oldState, newState)
}
}
func (cb *CircuitBreaker) State() CircuitState {
cb.mu.Lock()
defer cb.mu.Unlock()
return cb.state
}
The circuit breaker state machine contains three states: Closed (normal pass-through while measuring failures) → Open (fail fast for that upstream) → Half-Open (admit a bounded probe budget). The compact example illustrates transitions but does not limit concurrent half-open probes, classify failures, persist state, or coordinate replicas; those behaviors must be added before production use. Never let one upstream circuit become a fleet-wide global circuit.
Retry policy follows MCP operation semantics, not HTTP POST alone:
- Retry discovery and reads only within their deadline, freshness, authorization, and consistency contract
- Retry a side-effecting
tools/callonly when the upstream enforces an idempotency key or transactional deduplication - Preserve the same logical operation key across attempts, but use correct per-attempt transport correlation
- Treat timeout, cancellation, connection loss, and Circuit Open as an unknown effect unless the Server can prove the outcome
- Use exponential backoff with jitter and a total attempt budget; retries must re-enter admission control
A cached Resource or read result may be returned only when its cacheScope, TTL, authorization context, and freshness policy allow it, and the response must disclose that it is cached. Never present historical Tool output as if a new Tool execution succeeded. If the effect is unknown, return a bounded error that preserves uncertainty; choosing an alternative Tool is a new, separately authorized operation and may duplicate the original side effect.
FAQ
Q: What is the difference between an MCP Gateway and an API Gateway? A: Both can terminate TLS, authenticate callers, route HTTP, rate-limit, and emit telemetry. An MCP-aware Gateway additionally validates MCP versions and routing headers, understands JSON-RPC methods, capability discovery, result envelopes, MRTR, and subscription streams. An existing API Gateway may be enough when those controls are unnecessary.
Q: How many concurrent connections can a single MCP Gateway node support? A: There is no portable number. Ordinary 2026-07-28 Requests are independent POSTs, while request-scoped SSE and subscriptions consume longer-lived resources. Measure request rate, concurrent streams, bytes, queue time, p95/p99 latency, downstream saturation, cancellation, and recovery on the target hardware.
Q: Does MCP 2026-07-28 require sticky sessions or a shared Session store? A: No. The modern Core has no Protocol Session, Initialize Handshake, GET Stream, or stream resumption. A self-describing Request can reach any compatible instance. Affinity or shared state is needed only for an isolated Legacy route or explicit application state identified by a scoped handle.
Q: How much latency does the Gateway layer add? A: There is no portable latency number. Separate queue, Gateway service, upstream, and stream duration; include token validation, policy lookup, serialization, payload size, observability, retries, and cache decisions. Report p50/p95/p99 on the target transport and hardware rather than assuming the extra hop is negligible.
Q: How do you smoothly migrate existing MCP Servers to the Gateway architecture? A: Start by choosing transparent or aggregating behavior. Register immutable Server identity, verify revision and transport support, namespace and hash descriptors, configure separate OAuth audiences and credentials, and test authorization and failure behavior. Move a small Client cohort through an explicit route, keep Legacy traffic isolated, and retire direct access only after rollback, duplicate-effect, cancellation, and recovery tests pass.
Summary
An MCP Gateway is justified when shared routing, capability governance, OAuth mediation, backpressure, and observability outweigh an extra hop and a larger blast radius. For MCP 2026-07-28, keep the normal data plane Stateless, isolate Legacy Session behavior, preserve Server authorization, and treat aggregation as a new protocol surface with explicit identity and namespace rules. Capacity, retry safety, and recovery must be demonstrated with workload-specific tests.
For the underlying participant and transport model, see the MCP Protocol Complete Guide. The Advanced MCP Protocol Practice covers Server-side implementation concerns that remain authoritative behind a Gateway.