What is Cross-Encoder?
Cross-Encoder is a pair-scoring model architecture that jointly encodes a query and candidate in one forward pass and returns a task-specific relevance, similarity, or classification score.
How It Works
Joint Encoding and Score Semantics
A common text model serializes [CLS] query [SEP] candidate [SEP], applies Transformer layers to the joint sequence, and maps a pooled representation to a scalar. Other model families and modalities can use different layouts. The original [BERT passage reranking](https://arxiv.org/abs/1901.04085) work showed the strength of this approach under MS MARCO and TREC-CAR conditions. A sigmoid can map a logit into [0,1] without making it calibrated probability; ordering, thresholding and explanations remain model- and workload-specific.
Candidate Budget and Runtime Boundary
Every candidate requires pair construction, tokenization and model computation. Batch size can improve throughput but may worsen queueing or tail latency, and long candidates compete for the shared sequence limit. Truncation policy must preserve the evidence region rather than silently scoring only the beginning. Tune candidate depth against pre-rerank Recall@K, post-rerank nDCG or MRR, p95 latency, cost and timeout rate. The current [Sentence Transformers documentation](https://www.sbert.net/examples/sentence_transformer/applications/retrieve_rerank/README.html) presents retrieve-then-rerank as a two-stage pattern; its example candidate counts are demonstrations, not defaults.
Failure Modes and Release Evaluation
A Cross-Encoder cannot recover missing candidates, repair unauthorized retrieval, or prove that a passage supports the final answer. It can overfit source-specific artifacts, prefer verbose passages, mishandle negation or language shifts, and assign high scores to plausible but unsupported candidates. Evaluate by query group with frozen candidates, labels and truncation; report nDCG, MRR, Precision@K, evidence support, latency and cost. Research on scaling reranker inference shows that scoring more documents can eventually stop helping or degrade quality, so candidate depth is an empirical release parameter.
Key Characteristics
- Jointly processes both inputs so attention can model pair-level token interactions
- Produces task-specific scores rather than reusable standalone document vectors
- Usually offers richer ranking signals at higher per-candidate compute cost
- Operates on a bounded candidate set supplied by an earlier retrieval stage
- Depends on serialization, truncation, model revision, activation and label semantics
- Requires workload-level calibration and ranking evaluation before thresholds are used
Common Use Cases
- Reranking lexical, dense or hybrid candidates for a RAG pipeline
- Resolving fine-grained constraints, negation and entity distinctions in search
- Scoring question-passage pairs for a domain-specific relevance task
- Auditing whether a larger rerank candidate budget justifies its latency and cost
- Providing teacher scores for distilling a cheaper retrieval model
Example
Loading code...Frequently Asked Questions
Is a Cross-Encoder score a relevance probability?
Not automatically. Many rerankers emit unbounded logits; applying sigmoid only transforms the number and does not calibrate it. Thresholds need held-out labels from the target language, corpus, candidate source and model revision.
Why can a Cross-Encoder not search the whole corpus directly?
It must run a joint forward pass for each query-candidate pair and cannot reuse one standalone document vector for arbitrary queries. Large corpora therefore need a cheaper first-stage retriever to bound the number of pairs.
How many candidates should a Cross-Encoder rerank?
There is no universal top-k. Increase depth while measuring pre-rerank Recall@K, post-rerank nDCG or MRR, truncation, p95 latency, cost and timeouts. Stop when marginal quality no longer justifies the operational cost or begins to regress.
Are all rerankers Cross-Encoders?
No. Rerankers can use late interaction, listwise LLM scoring, learned features, business rules or combinations. Record the actual model, input grouping and score semantics rather than inferring architecture from the word reranker.
Can Cross-Encoder scores be cached?
Only for the same normalized query, candidate content, model revision, serialization and policy context. Candidate updates, permissions, truncation or model changes invalidate the pair score; caching must not bypass access checks.