Eino RAG Pipeline: Production Indexing, Retrieval, and Evaluation

TL;DR

A production RAG system is two versioned pipelines: an offline path that turns governed source documents into traceable index entries, and an online path that retrieves authorized evidence before generation. Eino v0.9.21 supplies stable component interfaces for loading, transforming, embedding, indexing, and retrieving. Reliability comes from preserving provenance, pinning index compatibility, evaluating retrieval separately from generation, and failing closed when evidence or authorization is insufficient.

Table of Contents

1. The Two RAG Pipelines
2. Eino v0.9.21 Component Contracts
3. Design the Index Contract
4. Chunking and Provenance
5. Online Retrieval and Reranking
6. Runnable Eino Reference Implementation
7. Authorization and Prompt-Injection Boundaries
8. Evaluation and Release Gates
9. Failure Diagnosis
10. Frequently Asked Questions
11. Primary Sources

The Two RAG Pipelines

Retrieval-augmented generation combines model parameters with external, retrievable evidence. The original RAG paper framed this as parametric memory plus non-parametric memory; production systems must also make the memory current, authorized, attributable, and measurable.

The offline path owns corpus correctness. The online path owns query correctness. Mixing them into one function makes retries, access control, and rollback much harder.

Eino v0.9.21 Component Contracts

Eino core defines narrow interfaces rather than one universal RAG object.

schema.Document contains ID, Content, and MetaData; helper methods store retrieval score and sub-index information in metadata. Eino's Retriever documentation states that results should be ordered by relevance. WithScoreThreshold filters results and does not itself sort them.

These are core contracts. Concrete vector databases, loaders, parsers, and model clients are separate modules or application implementations. Pin every module independently and do not copy an import path from an old tutorial without checking that the package exists for your chosen revision.

Design the Index Contract

An index is compatible only when the complete encoding contract matches.

Persist this manifest with each index version:

Changing an embedding model in place is unsafe even when vector dimensions happen to match. Build a new index, run shadow evaluation, switch an alias or routing key, and retain rollback until the new version is proven.

Write semantics

Use stable document and chunk IDs so re-indexing is idempotent. A practical identity is derived from:

source identity + source revision + structural location + normalized content hash

Treat delete propagation as part of indexing. A source document removed or revoked upstream must not survive indefinitely in the retrieval index.

Chunking and Provenance

Chunking is a retrieval decision, not a fixed token number.

Structure-aware boundaries usually preserve meaning better than arbitrary windows. Split headings, paragraphs, code blocks, tables, and list groups without breaking their internal semantics. Add overlap only when the evaluation set proves it recovers cross-boundary evidence; overlap also increases duplicate candidates, storage, and prompt cost.

Every chunk should carry:

• immutable chunk ID and parent document ID
• canonical source URI and source revision
• title and structural path
• byte, line, page, or timestamp location when available
• tenant and object-level authorization fields
• language and content type
• ingestion timestamp and index version

Provenance is required for citations, deletion, incident analysis, and freshness checks. A generated answer cannot repair provenance that was discarded during ingestion.

Online Retrieval and Reranking

Retrieval should be designed as a sequence of testable stages.

1. Normalize the query without removing meaningful identifiers.
2. Resolve tenant, user, and object filters from trusted identity.
3. Retrieve a wider candidate set.
4. Optionally fuse lexical and dense rankings.
5. Rerank a bounded set.
6. Deduplicate near-identical chunks.
7. Build context with source IDs and a token budget.
8. Generate an answer or abstain when evidence is insufficient.

Dense, sparse, and hybrid retrieval

Dense retrieval captures semantic similarity but may miss exact error codes, product IDs, or names. Sparse retrieval preserves lexical matches but may miss paraphrases. Hybrid retrieval can combine both, but it is not automatically better.

Reciprocal Rank Fusion is a simple score-independent baseline:

RRF(document) = sum(1 / (k + rank_in_list))

Choose the fusion constant and candidate depths on a frozen query set. Do not compare systems using different corpora, filters, or relevance labels.

Reranking

Reranking spends more computation on a small candidate set. In Eino, it can be represented by a document.Transformer or an application stage. The exact model or backend API is not part of the core Retriever interface.

BEIR found that retrieval approaches vary significantly across domains and that reranking can improve zero-shot quality at higher computational cost. That supports evaluation across representative domains, not a claim that one retriever is universally best.

Runnable Eino Reference Implementation

This deterministic program implements Eino's Indexer and Retriever interfaces with an in-memory lexical score. It is not a production search engine; it provides a compileable contract test for ID assignment, metadata preservation, relevance ordering, TopK, ScoreThreshold, and cancellation.

package main

import (
	"context"
	"fmt"
	"log"
	"sort"
	"strconv"
	"strings"
	"sync"
	"unicode"

	"github.com/cloudwego/eino/components/indexer"
	"github.com/cloudwego/eino/components/retriever"
	"github.com/cloudwego/eino/schema"
)

type memoryStore struct {
	mu   sync.RWMutex
	docs map[string]*schema.Document
}

func newMemoryStore() *memoryStore {
	return &memoryStore{docs: make(map[string]*schema.Document)}
}

func cloneDocument(doc *schema.Document) *schema.Document {
	metadata := make(map[string]any, len(doc.MetaData))
	for key, value := range doc.MetaData {
		metadata[key] = value
	}
	return &schema.Document{
		ID:       doc.ID,
		Content:  doc.Content,
		MetaData: metadata,
	}
}

func (s *memoryStore) Store(
	ctx context.Context,
	docs []*schema.Document,
	_ ...indexer.Option,
) ([]string, error) {
	s.mu.Lock()
	defer s.mu.Unlock()

	ids := make([]string, 0, len(docs))
	for i, doc := range docs {
		if err := ctx.Err(); err != nil {
			return nil, err
		}
		copy := cloneDocument(doc)
		if copy.ID == "" {
			copy.ID = "doc-" + strconv.Itoa(i+1)
		}
		s.docs[copy.ID] = copy
		ids = append(ids, copy.ID)
	}
	return ids, nil
}

func terms(text string) map[string]struct{} {
	result := make(map[string]struct{})
	for _, token := range strings.FieldsFunc(
		strings.ToLower(text),
		func(r rune) bool {
			return !unicode.IsLetter(r) && !unicode.IsDigit(r)
		},
	) {
		result[token] = struct{}{}
	}
	return result
}

func overlapScore(query, content string) float64 {
	queryTerms := terms(query)
	if len(queryTerms) == 0 {
		return 0
	}
	contentTerms := terms(content)
	matches := 0
	for term := range queryTerms {
		if _, ok := contentTerms[term]; ok {
			matches++
		}
	}
	return float64(matches) / float64(len(queryTerms))
}

func (s *memoryStore) Retrieve(
	ctx context.Context,
	query string,
	opts ...retriever.Option,
) ([]*schema.Document, error) {
	options := retriever.GetCommonOptions(nil, opts...)
	topK := 3
	if options.TopK != nil && *options.TopK > 0 {
		topK = *options.TopK
	}
	threshold := 0.0
	if options.ScoreThreshold != nil {
		threshold = *options.ScoreThreshold
	}

	s.mu.RLock()
	defer s.mu.RUnlock()

	results := make([]*schema.Document, 0, len(s.docs))
	for _, doc := range s.docs {
		if err := ctx.Err(); err != nil {
			return nil, err
		}
		score := overlapScore(query, doc.Content)
		if score >= threshold {
			results = append(results, cloneDocument(doc).WithScore(score))
		}
	}
	sort.Slice(results, func(i, j int) bool {
		return results[i].Score() > results[j].Score()
	})
	if len(results) > topK {
		results = results[:topK]
	}
	return results, nil
}

func main() {
	ctx := context.Background()
	store := newMemoryStore()

	_, err := store.Store(ctx, []*schema.Document{
		{
			ID:      "timeouts",
			Content: "Propagate one context deadline to model and tool calls.",
			MetaData: map[string]any{
				"source": "runbook.md",
				"tenant": "acme",
			},
		},
		{
			ID:      "retries",
			Content: "Retry only transient and idempotent operations.",
			MetaData: map[string]any{
				"source": "reliability.md",
				"tenant": "acme",
			},
		},
	})
	if err != nil {
		log.Fatal(err)
	}

	docs, err := store.Retrieve(
		ctx,
		"How should tool calls use a deadline?",
		retriever.WithTopK(2),
		retriever.WithScoreThreshold(0.2),
	)
	if err != nil {
		log.Fatal(err)
	}
	for _, doc := range docs {
		fmt.Printf(
			"id=%s score=%.2f source=%s\n",
			doc.ID,
			doc.Score(),
			doc.MetaData["source"],
		)
	}
	// Output: id=timeouts score=0.43 source=runbook.md
}

Use this module pin:

require github.com/cloudwego/eino v0.9.21

A production backend should also honor index and sub-index options, apply authorization filters before scoring, define consistency behavior, and batch network operations. Contract tests can reuse the same fixture across implementations.

Authorization and Prompt-Injection Boundaries

Retrieval is a data-access operation. The model must never decide which tenant or object filters apply.

Use this order:

authenticated identity
  -> server-side policy decision
  -> mandatory retrieval filters
  -> candidate documents
  -> content safety and injection handling
  -> model context

Documents are untrusted input even when they come from an internal knowledge base. Retrieved text can contain instructions that attempt to override system policy or exfiltrate data. Preserve the distinction between evidence and instructions, restrict tools independently, and never grant access based on text retrieved from the corpus.

Cache keys must include every value that changes visibility or output: tenant, authorization scope, locale, corpus/index version, retrieval configuration, and query normalization version. A shared cache without policy dimensions is a cross-tenant leak.

Evaluation and Release Gates

Measure retrieval before measuring generation. Otherwise a fluent model can hide a weak retriever, and a poor answer can be blamed on retrieval without evidence.

Retrieval set

For each query, store:

• relevant document and chunk IDs
• relevance grades when partial relevance matters
• expected tenant and policy filters
• unanswerable or no-access label
• important exact identifiers

Report Recall@K, MRR, and nDCG by language, domain, document age, and query type. Keep a lexical baseline; BEIR shows why dense retrieval should not be assumed to dominate every domain.

End-to-end set

Measure:

• answer correctness
• citation precision and completeness
• whether every material claim is supported
• abstention when evidence is missing
• resistance to instructions inside retrieved documents
• latency, model tokens, candidate count, and reranker cost

Release a new chunker, embedding, retriever, or reranker only after shadow evaluation and canary checks. Change one major retrieval variable at a time when possible.

Failure Diagnosis

Save retrieval candidates and scores in a protected diagnostic store using bounded retention. Do not rely only on the final answer when investigating a bad case.

Frequently Asked Questions

Should an Eino RAG pipeline be a Chain or a Graph?

Use a Chain for a fixed linear path. Use a Graph when the workflow branches by query type, performs parallel retrieval, or needs loops and state. The component interfaces remain useful in either form.

Is a vector database required?

No. Small or highly structured corpora may work better with SQL, inverted indexes, or exact lookup. The runnable example deliberately starts with a lexical implementation so the contract and baseline are visible.

How large should chunks be?

There is no universal token count. Test structural and fixed-window strategies against real questions, then compare retrieval quality, duplicate rate, context cost, and citation precision. Keep the tokenizer and overlap in the index manifest.

When should reranking be added?

Add it when first-stage retrieval has adequate recall but poor ordering, and only if the quality gain justifies latency and cost. Reranking cannot recover documents that were never retrieved.

Summary

Eino's value in RAG is the separation of contracts: load, transform, embed, index, retrieve, then generate. Production quality depends on what flows through those contracts: versioned corpus data, preserved provenance, enforced authorization, compatible embeddings, measured retrieval, and safe rollout. Start with a reproducible baseline and add complexity only when the evaluation set proves its value.

Related Resources

• Eino Components: ChatModel, Tool, and Retriever
• RAG vs Fine-Tuning
• Hybrid Search Glossary
• Embedding Glossary
• RAG Glossary

Primary Sources

• Eino v0.9.21 Retriever API
• Eino v0.9.21 Indexer API
• Eino v0.9.21 Document API
• CloudWeGo Retriever guide
• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• BEIR: A Heterogeneous Benchmark for Information Retrieval