What is Bi-Encoder?
Bi-Encoder is a model architecture that encodes two inputs independently into reusable vectors and compares those vectors with a declared similarity function.
How It Works
Architecture and Similarity Contract
The query and document towers need compatible output spaces, but they do not have to be the same model. Asymmetric retrieval may use different prompts, maximum lengths, pooling rules or weights for short questions and long passages. The serving contract records both tower revisions, tokenizer, prefixes, truncation, pooling, dimension, normalization and metric. Document vectors are reusable only while that complete contract remains compatible.
Contrastive Training and Negative Data
Training commonly raises the score of a labeled query-positive pair relative to in-batch, mined or curated negatives using a softmax or margin objective. Temperature changes the scale and concentration of the distribution; it is not merely a serving threshold. Easy negatives may teach little, while false negatives punish genuinely relevant documents. Hard-negative mining therefore needs provenance, deduplication, graded labels and slices for near-duplicate, multilingual and domain-specific cases. The original [DPR](https://aclanthology.org/2020.emnlp-main.550/) work demonstrates dual-encoder retrieval for open-domain QA, but its reported gains belong to its datasets and training protocol.
Serving, Migration, and Evaluation
Store model and preprocessing identity beside every document vector and reject mixed revisions. During migration, dual-write or rebuild a shadow index, compare candidate overlap and labeled Recall@K, then validate ANN Recall, latency and downstream evidence before switching. Evaluate candidate recall before reranking, because ranking metrics after a Cross-Encoder can hide first-stage omissions. [Sentence-BERT](https://aclanthology.org/D19-1410/) established efficient independently computed sentence representations, but current production quality still depends on task-aligned training, chunking, language and index behavior.
Key Characteristics
- Encodes the two sides independently before any similarity comparison
- Allows document representations to be precomputed, indexed and reused across queries
- May use shared, partially shared or asymmetric query and document towers
- Is commonly trained with contrastive objectives and carefully selected negatives
- Trades pair-level interaction for scalable first-stage candidate generation
- Requires both tower and preprocessing identities to remain compatible with the index
Common Use Cases
- Retrieving candidates from large document or product collections
- Creating reusable document embeddings for a vector index
- Serving low-latency semantic search before a more expensive reranker
- Training asymmetric question-to-passage or text-to-code retrieval
- Distilling pairwise ranking behavior into a cheaper first-stage model
Example
Loading code...Frequently Asked Questions
Are Bi-Encoder, dual encoder, and Siamese encoder identical terms?
They are often used interchangeably, but the emphasis differs. Bi- or dual encoder means two inputs are encoded independently. Siamese usually implies shared weights, while a two-tower retrieval system may use asymmetric or only partially shared towers.
Do the query and document towers have to share weights?
No. Shared weights are useful when both inputs have similar distributions. Asymmetric tasks can benefit from different prompts, pooling, lengths or parameters, provided training places both outputs in a compatible scoring space.
Why do Bi-Encoders need hard negatives?
Random negatives can be trivially unrelated and produce little ranking signal. Hard negatives expose distinctions the current model confuses, but mislabeled or false negatives can damage training, so they need provenance and human or rule-based quality checks.
Can a Bi-Encoder score be used as a probability threshold?
Not without calibration. Dot products and cosine similarities depend on model, normalization, task and candidate distribution. Use them for ranking within the declared contract, and calibrate any threshold on held-out workload data.
When should a Cross-Encoder follow a Bi-Encoder?
Add a reranker when first-stage Recall@K is adequate but final ordering remains weak and the latency budget permits pairwise scoring. Tune candidate depth jointly with quality and p95 cost; a fixed top-k copied from a tutorial is not a production rule.