AI Search Architecture: Retrieval, Citations, and Safety

TL;DR

An AI search engine is a retrieval, evidence, and answer-verification system, not merely an LLM connected to a search API. A production architecture must plan queries, retrieve only authorized sources, fetch web content safely, preserve evidence provenance, generate atomic claims, verify claim-level citations, handle missing or conflicting evidence, and pass quality, security, latency, and cost gates as one versioned release.

Table of Contents

• What an AI Search Engine Is
• The Production Architecture
• Define a Query Contract
• Retrieve and Fetch Evidence Safely
• Rank and Assemble Evidence
• Generate Claims, Then Verify Citations
• A Runnable Citation Contract
• Evaluate the Pipeline by Layer
• Design Vertical AI Search
• Operate a Versioned Release
• Frequently Asked Questions
• Summary

Key Takeaways

• Retrieval success is not answer success. A relevant document can be ignored, misquoted, or attached to the wrong claim.
• Citations are structured evidence links. URL presence alone does not establish support or completeness.
• Retrieved content is untrusted data. Web pages can contain prompt injection, poisoned facts, scripts, and malicious URLs.
• Authorization happens before model context. An LLM must never decide which tenant or document a requester may access.
• One score is insufficient. Retriever, evidence, answer, citation, abstention, security, and operations need separate gates.

What an AI Search Engine Is

An AI search engine accepts an information need and returns a synthesized response tied to retrievable evidence. It may search the public web, a private corpus, structured databases, or several sources, but its defining product responsibility is to make the answer verifiable and policy-compliant.

The neighboring concepts have narrower scope:

The original RAG paper combined parametric generation with explicit non-parametric memory. Modern answer engines extend that pattern with search routing, source acquisition, policy enforcement, citation, and interaction. Not every query needs generation: navigational queries, exact records, calculations, or weak-evidence cases may be better served by links, structured results, clarification, or abstention.

Product internals are usually proprietary. External behavior does not prove that a named service uses a specific web index, model, number of subqueries, ranking formula, or citation algorithm. Build and evaluate an explicit architecture instead of reverse-engineering assumptions into design requirements.

The Production Architecture

A production AI search architecture separates trust boundaries and preserves a typed evidence path from source to claim.

The stages should exchange contracts, not free-form strings:

1. Request contract: authenticated principal, locale, time sensitivity, answer mode, source policy, and budget.
2. Query plan: subquestions, retrieval routes, required source classes, and termination limits.
3. Evidence records: immutable IDs, canonical source, snapshot, text offsets, timestamps, policy scope, and retrieval scores.
4. Answer object: atomic claims and citation references, not citation markers embedded in an opaque paragraph.
5. Verification result: supported, unsupported, conflicting, unauthorized, stale, or unverifiable.
6. Release trace: model, prompts, retrievers, indexes, parsers, policies, corpus revisions, costs, and timings.

This architecture allows each failure to be located. A fluent but unsupported answer is not mislabeled as a retriever error, and a retrieval miss is not “fixed” by asking the generator to be more confident.

Define a Query Contract

The query contract determines what the system is allowed and required to search before an LLM expands the request.

{
  "requestId": "req-1842",
  "principal": {
    "tenantId": "tenant-a",
    "subjectId": "user-7",
    "roles": ["support"]
  },
  "query": "Which refund policy applies to annual plans?",
  "locale": "en-US",
  "asOf": "2026-08-23T12:00:00Z",
  "answerMode": "cited-summary",
  "sourcePolicy": {
    "classes": ["approved-public", "tenant-private"],
    "domains": ["docs.example.com"],
    "maxAgeSeconds": 86400
  },
  "budget": {
    "maxSubqueries": 4,
    "maxFetchedBytes": 2000000,
    "maxEvidenceTokens": 12000,
    "deadlineMs": 5000
  }
}

The planner can then decide:

• whether search is needed;
• whether the query is navigational, factual, comparative, exploratory, or unsafe;
• which claims require fresh evidence;
• which private repositories the principal can access;
• whether to decompose the request;
• when to stop searching.

Query rewriting is a recall hypothesis, not a guaranteed improvement. Expansion can drift from the user’s intent, amplify a false premise, or leak sensitive terms to an external provider. Keep the original query, record every generated subquery, classify sensitive data before external calls, and compare single-query and multi-query baselines.

See the Query Rewriting glossary for the retrieval-level pattern.

Retrieve and Fetch Evidence Safely

Retrieval must produce evidence that is relevant, authorized, current enough, and safe to process. Dense vectors are only one candidate-generation method.

Route across complementary sources

Fuse incompatible retriever scores by rank unless calibration evidence supports score fusion. Apply authorization before private content reaches rerankers, caches, prompts, or logs. A post-generation filter cannot reliably retract information already exposed to the model.

Isolate web fetching

A web-search result is a URL suggestion, not trusted content. The fetcher should:

• allow only intended schemes;
• resolve and validate destination addresses;
• block loopback, private, link-local, metadata, and internal ranges;
• revalidate every redirect;
• enforce network egress policy;
• limit redirects, time, bytes, decompressed size, and content types;
• sandbox HTML, PDF, office, image, and archive parsers;
• remove active content while preserving source text and offsets;
• record final URL, canonical URL, retrieval time, headers, digest, and parser revision.

These controls follow the defense-in-depth model in the OWASP SSRF Prevention Cheat Sheet. An open-web product may not be able to allowlist every domain, which makes network isolation and destination validation even more important.

Crawlers should implement RFC 9309 Robots Exclusion Protocol. The RFC explicitly states that robots rules are not access authorization. Compliance with robots.txt also does not resolve copyright, privacy, contract, paywall, or data-use obligations.

Treat content as untrusted data

Retrieved pages can contain indirect Prompt Injection such as instructions to ignore policy, reveal secrets, call tools, or prefer a poisoned source. The OWASP RAG Security Cheat Sheet covers document poisoning, access-control inheritance, source provenance, chunk isolation, cache risks, output validation, and fail-closed behavior.

Delimiters and prompt wording can reduce confusion, but they do not create a security boundary. The generator should have no authority to expand its own data access, modify filters, follow arbitrary links, or invoke consequential tools.

Rank and Assemble Evidence

Ranking should optimize evidence coverage and quality under a budget, not simply choose the highest similarity values.

Start with:

1. Retrieve broad candidate sets from approved routes.
2. Normalize stable document and passage identities.
3. Collapse exact and near duplicates.
4. Rerank for query or subquestion relevance.
5. Apply source-class, freshness, language, and authority policies.
6. Preserve materially conflicting evidence.
7. Select a diverse set covering the required information units.
8. Stop when the evidence-sufficiency rule or budget is reached.

An evidence record should retain enough information for verification:

{
  "evidenceId": "ev-7",
  "documentId": "policy-annual-refunds",
  "sourceUrl": "https://docs.example.com/policies/refunds",
  "canonicalUrl": "https://docs.example.com/policies/refunds",
  "snapshotSha256": "sha256:replace-with-real-digest",
  "retrievedAt": "2026-08-23T12:00:01Z",
  "validFrom": "2026-07-01",
  "textStart": 1480,
  "textEnd": 1612,
  "passage": "Annual plans may be refunded within 14 days of the initial purchase.",
  "policyScope": {
    "tenantId": "tenant-a",
    "classification": "internal",
    "authorized": true
  },
  "scores": {
    "retrieval": 0.73,
    "reranker": 0.91
  }
}

Scores are release-specific ranking signals, not probabilities of truth. The snapshot digest and offsets let a reviewer inspect what the model saw even after the live page changes.

Context assembly should also preserve contradictions. If current policy says 14 days and an older source says 30 days, silently dropping one can hide a data-lifecycle failure. Pass version and authority signals to the verifier and state unresolved conflicts to the user.

For retrieval mechanics, use the Semantic Search guide. For iterative retrieval, see Agentic RAG.

Generate Claims, Then Verify Citations

Citation generation should produce an explicit claim-to-evidence graph. Formatting [1] after a sentence is not validation.

The ALCE benchmark separates fluency, correctness, and citation quality. TREC 2024 RAG publishes distinct retrieval judgments, information-nugget assessments, and citation-support assessments. These designs establish a useful production rule: answer quality and citation quality are related but separate.

For every factual claim, verify:

1. Validity: Does the evidence ID map to a retained snapshot?
2. Relevance: Is the cited passage about the claim?
3. Entailment: Does the passage actually support the claim as written?
4. Completeness: Are all externally verifiable claims supported?
5. Authority: Is the source acceptable for this domain and claim type?
6. Freshness: Was the source valid for the requested time?
7. Authorization: Was the evidence allowed for this principal and response?
8. Conflict: Does another authoritative source materially disagree?

Post-hoc citation attachment can find a plausible passage after generation while concealing that the model did not use it. Prefer generating atomic claims against evidence IDs, then run an independent verifier. Remove, narrow, regenerate, or abstain when support fails.

Do not expose hidden chain-of-thought as proof. A concise verification trace should show the final claim, evidence IDs, decision, and reason without revealing private reasoning tokens.

A Runnable Citation Contract

This Python example verifies structural citation integrity before semantic entailment review. It fails on unknown evidence, unauthorized evidence, missing quotes, uncited claims, and quotations absent from the retained passage.

from dataclasses import dataclass
from typing import Iterable


@dataclass(frozen=True)
class Evidence:
    evidence_id: str
    passage: str
    authorized: bool


@dataclass(frozen=True)
class Citation:
    evidence_id: str
    quote: str


@dataclass(frozen=True)
class Claim:
    claim_id: str
    text: str
    citations: tuple[Citation, ...]


def validate_citations(
    claims: Iterable[Claim],
    evidence: Iterable[Evidence],
) -> list[str]:
    evidence_by_id = {item.evidence_id: item for item in evidence}
    errors: list[str] = []

    for claim in claims:
        if not claim.citations:
            errors.append(f"{claim.claim_id}: missing citation")
            continue

        for citation in claim.citations:
            item = evidence_by_id.get(citation.evidence_id)
            if item is None:
                errors.append(
                    f"{claim.claim_id}: unknown evidence {citation.evidence_id}"
                )
                continue
            if not item.authorized:
                errors.append(
                    f"{claim.claim_id}: unauthorized evidence {citation.evidence_id}"
                )
            if not citation.quote.strip():
                errors.append(f"{claim.claim_id}: empty supporting quote")
            elif citation.quote not in item.passage:
                errors.append(
                    f"{claim.claim_id}: quote is absent from {citation.evidence_id}"
                )

    return errors


evidence = [
    Evidence(
        evidence_id="ev-7",
        passage=(
            "Annual plans may be refunded within 14 days "
            "of the initial purchase."
        ),
        authorized=True,
    )
]
claims = [
    Claim(
        claim_id="c-1",
        text="Annual plans have a 14-day initial refund window.",
        citations=(
            Citation(
                evidence_id="ev-7",
                quote="within 14 days of the initial purchase",
            ),
        ),
    )
]

assert validate_citations(claims, evidence) == []
print("structural citation checks passed")

Expected output:

structural citation checks passed

Exact quotation proves that cited text exists in the retained passage; it does not prove entailment. Add a calibrated natural-language-inference or LLM verifier, test it against human labels, and route high-risk or ambiguous claims to human review.

Evaluate the Pipeline by Layer

Evaluation should identify which contract failed and prevent average scores from hiding risky slices.

The RAGChecker paper motivates fine-grained retriever and generator diagnostics. A production AI search suite can extend that separation:

Build negative controls:

• answer absent from all sources;
• correct answer present only in an unauthorized source;
• two authoritative sources conflict;
• retrieved page contains indirect prompt instructions;
• result URL redirects to a private address;
• source changes after retrieval;
• citation ID exists but does not support the claim;
• cache entry belongs to another tenant or policy revision.

Version human relevance and citation judgments with a frozen corpus snapshot. The TREC RAG data demonstrates why retrieval, nugget coverage, and citation support need different judgments.

Release thresholds should include worst-slice results, not only global averages. A system that performs well on public English fact lookup but leaks private documents or fails on no-answer cases is not production-ready.

Design Vertical AI Search

Vertical AI search changes the evidence and policy contract, not merely the prompt or model.

A domain-adapted model does not automatically make vertical search better. The advantage comes only when curated evidence, schema, ranking signals, policies, and expert evaluation improve the target workload. Compare against lexical-only, semantic-only, non-generative, and general AI search baselines.

For high-risk legal or medical search, distinguish discovery from professional decision-making. Preserve the authoritative source, edition, jurisdiction, date, and caveat; do not let synthesis erase provenance or scope.

Operate a Versioned Release

An AI search result belongs to a complete release identity because every layer can change behavior.

release:
  answerModel: provider/model@immutable-revision
  plannerPrompt: planner-v5
  synthesisPrompt: cited-answer-v8
  verifier: claim-support-v3
retrieval:
  lexicalIndex: web-2026-08-23
  embeddingModel: provider/embed@revision
  vectorIndex: corpus-v14
  reranker: provider/reranker@revision
fetch:
  policy: web-fetch-v6
  parser: document-parser-v11
policy:
  sourcePolicy: sources-v4
  authorizationPolicy: authz-v9
  cachePolicy: evidence-cache-v3
evaluation:
  dataset: ai-search-eval-v7
  judgmentRevision: judgments-2026-08-20

Use that identity in traces and cache keys. A safe cache key includes normalized query, locale, answer mode, principal policy scope, corpus or web snapshot, source policy, and all relevant component revisions. Never share a response cache across tenants merely because the text query matches.

Invalidate or re-evaluate when:

• a source is deleted, retracted, deauthorized, or materially changed;
• the index, embedding, reranker, parser, prompt, model, or verifier changes;
• source policy or user authorization changes;
• a new query class, locale, or high-risk domain launches;
• evaluation detects citation, abstention, security, latency, or cost regression.

Observability should retain query plans, source decisions, evidence IDs, verification outcomes, timings, and release versions while minimizing sensitive content. Logs themselves require authorization, retention, and deletion controls.

Frequently Asked Questions

Is every AI search engine a RAG system?

RAG is a useful architectural family for combining retrieval and generation, but a product may mix generated answers with direct links, structured data, calculators, cached responses, or model memory. “Every engine uses the same RAG pipeline” is not a verifiable design assumption.

How many subqueries and sources should an AI search engine use?

There is no universal number. Choose limits from evidence coverage, rewrite drift, latency, cost, and source diversity on representative query classes. A simple lookup may need one authoritative source; a comparative research question may need multiple independent sources and iterative retrieval.

Are inline citations enough to prevent hallucinations?

No. Citations improve inspectability, but a model can cite an irrelevant passage or omit support for part of a sentence. Evaluate answer correctness, citation entailment, citation completeness, source authority, and abstention independently.

Should AI search use vectors instead of BM25?

Not by default. Dense retrieval helps with paraphrases, while lexical retrieval remains strong for names, identifiers, quotations, and new vocabulary. Compare lexical-only, dense-only, hybrid, and reranked pipelines on the same labeled dataset.

What should happen when sources disagree?

Preserve the conflict, apply explicit authority and time rules where the domain has them, and show unresolved disagreement. The generator should not silently average contradictory facts or select the most convenient source.

Summary

AI search is a production evidence system. Its architecture must control query planning, source access, safe fetching, retrieval, ranking, context assembly, claim generation, citation verification, abstention, caching, and observability. Build each stage around versioned contracts, evaluate it separately and end to end, and fail closed when evidence or authorization is insufficient.

Related Resources

• Semantic Search engineering guide
• RAG guide
• Agentic RAG architecture
• RAG hallucination controls
• Prompt Injection defense guide
• Semantic Search glossary
• Citation glossary
• Grounding glossary
• ALCE citation evaluation paper
• TREC 2024 RAG evaluation data
• OWASP RAG Security Cheat Sheet