Eino Orchestration Engine: Chain, Graph, and Workflow in Practice

TL;DR

Building LLM applications requires more than single model calls — you need to compose prompt templates, model inference, tool calling, and result processing into coherent execution flows. Eino provides three orchestration APIs: Chain (linear pipelines), Graph (directed graphs with cycles and branching), and Workflow (DAGs with field-level data mapping). This article covers all three with complete Go code, architecture diagrams, and a practical Tool Calling Agent implementation.

Table of Contents

1. Key Takeaways
2. Why Orchestration Matters
3. Chain: Linear Orchestration
4. Graph: Graph-Based Orchestration
5. Workflow: Field-Level Mapping
6. Core Orchestration Capabilities
7. Comparison of Three APIs
8. Practice: Building a Tool Calling Agent
9. Best Practices
10. FAQ
11. Summary
12. Related Resources

Key Takeaways

• Chain: Linear DAG with minimal boilerplate — ideal for "template → model → post-process" pipelines
• Graph: Directed graph (supports cycles) with Branch for conditional routing — the go-to for building Agentic Workflows
• Workflow: DAG with field-level MapFields mapping — best for complex multi-input data transformations
• Type Safety: All three APIs leverage Go generics for compile-time type checking
• Built-in Streaming: The orchestration engine automatically handles stream concatenation, splitting, and merging

Why Orchestration Matters

Modern LLM applications are rarely single API calls. They're multi-step execution flows:

User Input → Prompt Assembly → Model Inference → Tool Calling → Result Parsing → Response

Consider a typical AI assistant: it needs to format user queries into prompts, send them to a model, check if the model wants to call external tools, execute those tools, feed results back to the model, and finally format the response. Each step has different input/output types, some can run in parallel, and the entire flow needs to support both synchronous and streaming modes.

Manually wiring these steps together creates several challenges:

• Type mismatches: Upstream outputs don't match downstream input formats, requiring tedious conversion code
• Stream propagation: Manual handling of stream forwarding, concatenation, and fan-out to multiple consumers
• Concurrency management: Independent nodes can't auto-parallelize without explicit goroutine orchestration
• Observability: No unified injection point for logging, tracing, and metrics across the entire pipeline
• Error handling: Failures in one node need proper propagation and cleanup across the graph

Eino's orchestration engine solves these fundamentally. It abstracts components as Nodes, defines execution topology through orchestration APIs, and compiles everything into an efficient Runnable instance via Compile(). The compiler analyzes the topology at build time, validates type compatibility, identifies parallelism opportunities, and produces an optimized execution plan.

Chain: Linear Orchestration

Chain is the simplest orchestration pattern — a straight line from start to finish with no branching or cycles.

Use Cases

• Prompt template → ChatModel → output parsing
• Text preprocessing → vectorization → storage
• Any purely sequential pipeline

Code Example

package main

import (
    "context"
    "fmt"

    "github.com/cloudwego/eino/compose"
    "github.com/cloudwego/eino/schema"
)

func buildChain(ctx context.Context) {
    // Create Chain: input map[string]any, output *schema.Message
    chain, _ := compose.NewChain[map[string]any, *schema.Message]().
        AppendChatTemplate(prompt).    // Node 1: Assemble prompt
        AppendChatModel(model).        // Node 2: Call model
        AppendLambda(extractContent).  // Node 3: Extract content
        Compile(ctx)

    // Execute
    result, err := chain.Invoke(ctx, map[string]any{
        "query": "what's your name?",
    })
    if err != nil {
        panic(err)
    }
    fmt.Println(result.Content)
}

Chain's design philosophy is simplicity: append nodes one by one with Append methods, making orchestration code as intuitive as the business logic itself.

Graph: Graph-Based Orchestration

When your execution flow requires conditional branching or cycles, Chain falls short. Graph uses three primitives — Node, Edge, and Branch — to construct arbitrary directed graphs.

Core Concepts

Agent Graph Structure (Mermaid Diagram)

This is a classic ReAct Agent loop: the model decides whether to call tools, results feed back into the model for further reasoning, and the cycle continues until no more tool calls are needed.

Code Example

func buildAgentGraph(ctx context.Context) {
    graph := compose.NewGraph[map[string]any, *schema.Message]()

    // Add nodes
    _ = graph.AddChatTemplateNode("node_template", chatTpl)
    _ = graph.AddChatModelNode("node_model", chatModel)
    _ = graph.AddToolsNode("node_tools", toolsNode)
    _ = graph.AddLambdaNode("node_converter", takeOne)

    // Define edges
    _ = graph.AddEdge(compose.START, "node_template")
    _ = graph.AddEdge("node_template", "node_model")

    // Conditional branch: does model output contain tool calls?
    branch := compose.NewBranch(func(ctx context.Context, msg *schema.Message) (string, error) {
        if len(msg.ToolCalls) > 0 {
            return "node_tools", nil
        }
        return compose.END, nil
    })
    _ = graph.AddBranch("node_model", branch)

    // Tool calling loop-back
    _ = graph.AddEdge("node_tools", "node_converter")
    _ = graph.AddEdge("node_converter", "node_model")

    // Compile and execute
    compiledGraph, err := graph.Compile(ctx)
    if err != nil {
        panic(err)
    }

    out, err := compiledGraph.Invoke(ctx, map[string]any{
        "query": "What's the weather in Beijing this weekend?",
    })
    fmt.Println(out.Content)
}

Graph's power lies in its support for cycles — the node_converter → node_model back-edge enables the model to repeatedly call tools, forming an autonomous Agent decision loop.

Workflow: Field-Level Mapping

Workflow is a DAG (no cycles), but adds field-level data mapping on top of Graph's capabilities. When multiple downstream nodes need different fields from an upstream output, Workflow is the optimal choice.

Data Flow Structure (Mermaid Diagram)

Code Example

func buildWorkflow(ctx context.Context) {
    wf := compose.NewWorkflow[[]*schema.Message, *schema.Message]()

    // Add nodes with input mappings
    wf.AddChatModelNode("model", m).AddInput(compose.START)

    wf.AddLambdaNode("lambda1", compose.InvokableLambda(lambda1)).
        AddInput("model", compose.MapFields("Content", "Input"))

    wf.AddLambdaNode("lambda2", compose.InvokableLambda(lambda2)).
        AddInput("model", compose.MapFields("Role", "Role"))

    wf.AddLambdaNode("lambda3", compose.InvokableLambda(lambda3)).
        AddInput("lambda1", compose.MapFields("Output", "Query")).
        AddInput("lambda2", compose.MapFields("Output", "MetaData"))

    wf.End().AddInput("lambda3")

    // Compile and execute
    runnable, err := wf.Compile(ctx)
    if err != nil {
        panic(err)
    }

    result, err := runnable.Invoke(ctx, messages)
    fmt.Println(result.Content)
}

MapFields("Content", "Input") means: map the upstream model node's Content field to the downstream lambda1 node's Input field. This precise field mapping eliminates the coupling that comes with passing entire objects between nodes.

Core Orchestration Capabilities

Eino's orchestration engine goes far beyond topology definition. It provides a comprehensive set of runtime capabilities:

Type Safety

Go generics enable compile-time type checking between nodes:

// Compiler verifies the map[string]any → *schema.Message type chain
chain, _ := compose.NewChain[map[string]any, *schema.Message]()

Stream Handling

The engine has three built-in stream processing mechanisms that handle the complexity of streaming data through a graph:

This means you can freely mix streaming and non-streaming nodes in the same graph — the engine handles all conversions transparently.

Concurrency Management

The Compile phase analyzes graph topology to automatically identify parallelizable node paths — no manual goroutine management required.

Aspect Injection

Through the Callback mechanism, you can inject cross-cutting concerns without modifying business code:

• Logging: Record each node's inputs and outputs
• Tracing: Auto-generate Spans for distributed tracing
• Metrics: Track node latency, token consumption, etc.

Option Assignment

Configure runtime parameters per-node or per-type:

// Set options for a specific node
compiledGraph.Invoke(ctx, input,
    compose.WithNodeOption("node_model", model.WithTemperature(0.7)),
)

Comparison of Three APIs

Selection Guide:

• Is your flow a straight line? → Use Chain
• Need branching or cycles? → Use Graph
• Need precise field-level data flow? → Use Workflow

Practice: Building a Tool Calling Agent

Here's a complete Tool Calling Agent example, building on the ChatModel and Tool components covered earlier in this series:

package main

import (
    "context"
    "fmt"

    "github.com/cloudwego/eino/compose"
    "github.com/cloudwego/eino/components/model"
    "github.com/cloudwego/eino/components/tool"
    "github.com/cloudwego/eino/schema"
)

// Define tool: get weather
func getWeather(ctx context.Context, params map[string]string) (string, error) {
    city := params["city"]
    return fmt.Sprintf("%s: Sunny this weekend, 22-28°C", city), nil
}

func main() {
    ctx := context.Background()

    // Initialize components
    chatModel := initChatModel()       // ChatModel instance
    chatTpl := initChatTemplate()      // Prompt template
    toolsNode := initToolsNode()       // Tools node
    converter := initConverter()       // Tool result converter

    // Build Graph
    graph := compose.NewGraph[map[string]any, *schema.Message]()

    _ = graph.AddChatTemplateNode("template", chatTpl)
    _ = graph.AddChatModelNode("model", chatModel)
    _ = graph.AddToolsNode("tools", toolsNode)
    _ = graph.AddLambdaNode("converter", converter)

    // Define execution topology
    _ = graph.AddEdge(compose.START, "template")
    _ = graph.AddEdge("template", "model")

    // Conditional branch: does the model need to call tools?
    branch := compose.NewBranch(func(ctx context.Context, msg *schema.Message) (string, error) {
        if len(msg.ToolCalls) > 0 {
            return "tools", nil
        }
        return compose.END, nil
    })
    _ = graph.AddBranch("model", branch)

    // Tool calling loop-back
    _ = graph.AddEdge("tools", "converter")
    _ = graph.AddEdge("converter", "model")

    // Compile
    agent, err := graph.Compile(ctx)
    if err != nil {
        panic(fmt.Sprintf("compile error: %v", err))
    }

    // Execute
    result, err := agent.Invoke(ctx, map[string]any{
        "query": "What's the weather in Beijing this weekend?",
    })
    if err != nil {
        panic(err)
    }
    fmt.Println(result.Content)
    // Output: Beijing: Sunny this weekend, 22-28°C, great for outdoor activities...
}

This code demonstrates the core advantage of Eino Graph orchestration: replacing imperative control flow with declarative topology definitions, making Agent logic clear and maintainable.

Best Practices

Start simple, upgrade as needed

Don't over-engineer — if Chain meets your requirements, don't reach for Graph. The orchestration API choice should match actual complexity. Many production systems start with Chain and graduate to Graph only when they need conditional branching or tool calling loops.

Leverage Compile for validation

Compile() doesn't just produce a runnable instance — it validates graph integrity, including orphan node detection, type chain verification, and edge completeness. Execute compilation at application startup to surface graph definition errors early, before any user traffic hits the system.

Use Options for runtime tuning

Don't hardcode model parameters. Use WithNodeOption to dynamically adjust Temperature, MaxTokens, and other parameters at call time, enabling A/B testing and gradual rollouts. This pattern separates graph structure (static) from execution parameters (dynamic).

Add Callbacks to critical nodes

In production, always attach Callbacks to ChatModel nodes to track token consumption and response latency — this is the foundation of cost control. Callbacks can also implement circuit breakers, rate limiting, and fallback logic without polluting the graph definition.

Leverage Go's type system defensively

Define clear input/output structs and let generics catch type mismatches at compile time. When working with JSON data structures, use the JSON to Go tool to quickly generate type definitions.

Design graphs for testability

Keep node implementations as pure functions where possible. Since each node is a separate component, you can unit test it in isolation and then integration test the compiled graph. Mock individual nodes by swapping implementations at compile time.

FAQ

Can Chain have conditional branches?

No. Chain is strictly limited to linear DAGs. If you need conditional branching, switch to Graph.

Can Graph cycles cause infinite loops?

The Branch routing function is responsible for deciding when to exit the loop. We recommend setting a maximum iteration count as a safety valve to prevent the model from falling into infinite tool calling.

Does Workflow support cycles?

No. Workflow is a strict DAG (Directed Acyclic Graph). If you need cycles with field mapping, consider nesting a Workflow as a subgraph within a Graph.

Is the orchestration overhead significant?

The orchestration engine's overhead is minimal (microsecond-level), completely negligible compared to LLM inference latency (milliseconds to seconds). The Compile phase pre-computes the execution plan, resulting in near-zero runtime overhead.

How do I debug complex Graphs?

Use the Callback mechanism to log inputs and outputs at each node's entry and exit points. Eino also supports OpenTelemetry integration, allowing you to visualize each node's execution in distributed tracing systems.

Summary

Eino's orchestration engine provides Go developers with a complete, type-safe component composition solution. The three APIs cover the full spectrum from simple pipelines to complex Agents:

• Chain handles 80% of simple sequential scenarios
• Graph handles Agent scenarios requiring conditional branching and cycles
• Workflow handles complex transformation scenarios requiring precise data flow control

Combined with the design philosophy introduced in the Eino Framework Overview and the core components detailed in the previous article, you now have all the foundational knowledge needed to build production-grade AI applications with Eino.

Related Resources

• Eino Official Repository - GitHub
• Eino Framework Overview: Why Go for AI Applications
• Eino Core Components: ChatModel, Tool, and Retriever
• AI Agent Glossary
• Agentic Workflow Glossary
• JSON Formatter Tool
• JSON to Go Tool