What is Zero-Shot Learning?
Zero-Shot Learning is a family of transfer and evaluation settings in which a model predicts target classes or performs a target task without labeled target examples under a declared protocol.
Quick Facts
| Created | Concept originated in 2000s, LLM context from 2020 |
|---|---|
| Specification | Official Specification |
How It Works
The word zero is meaningful only after the protocol names what is absent. In classical Zero-Shot Learning (ZSL), labeled examples exist for Seen Classes but not for target Unseen Classes. Attributes, class descriptions, language embeddings, knowledge graphs, or a pretrained multimodal representation provide Semantic Side Information that connects the two sets. A model can learn a compatibility function between inputs and class representations, then score an input against declared Unseen Candidate Labels. Conventional ZSL assumes every evaluation input belongs to an Unseen Class; Generalized Zero-Shot Learning (GZSL) uses a joint Seen-and-Unseen candidate space and exposes the common bias toward Seen Classes. Inductive ZSL does not use unlabeled evaluation instances during optimization, whereas Transductive ZSL may use unlabeled target data; those results are not directly comparable. LLM Zero-Shot Prompting is a different protocol: the current request contains an instruction but no Input-Output Demonstration, and no task-specific Gradient Update happens during the call. A System Prompt, label descriptions, retrieved context, Tools, and prior pretraining still supply information. Therefore, zero-shot prompting does not prove that the model never encountered the task, label, benchmark, or answer during Pretraining or Instruction Tuning. CLIP-style zero-shot transfer similarly means no Dataset-Specific Training for the evaluated task, not an absence of relevant concepts from web-scale pretraining. Zero-Shot Learning also differs from Open-Set Recognition, which may only reject an unknown input rather than assign a specific Unseen Label, and from Open-Vocabulary Classification, which describes a flexible label interface rather than one universal training protocol. Reproducible evaluation must freeze the model, data and class splits, semantic side information, Candidate Labels, label order, Prompt or Hypothesis Template, decoding, and any transductive access. It must audit overlap across raw examples, classes, sources, semantic descriptions, and the pretrained backbone. Classical ZSL should report per-class Unseen accuracy; GZSL should report Seen and Unseen per-class accuracy plus their Harmonic Mean or a calibrated trade-off. LLM evaluations need task-specific metrics, chance and few-shot or supervised baselines, calibration or abstention, prompt-variation tests, repeated runs where sampling is stochastic, and an explicit contamination status. A zero-shot score is evidence about that frozen protocol, not proof of universal generalization.
Key Characteristics
- Defines zero relative to a declared unit such as target-class labels, task examples, Prompt Demonstrations, or Dataset-Specific Training
- Uses Semantic Side Information or pretrained representations to transfer knowledge from Seen to Unseen Classes
- Distinguishes Conventional ZSL from GZSL, where Seen-Class bias must be measured in a joint candidate space
- Separates Inductive evaluation from Transductive access to unlabeled target instances
- Treats LLM Zero-Shot Prompting as no in-prompt Demonstrations, not proof of no Pretraining or Instruction-Tuning exposure
- Requires immutable splits, templates, labels, model identity, contamination evidence, calibrated metrics, and meaningful baselines
Common Use Cases
- Classifying newly introduced product, document, or visual categories from stable label descriptions before labeled examples exist
- Using vision-language representations for open-vocabulary retrieval or classification under a frozen candidate-label protocol
- Establishing a no-Demonstration LLM baseline before investing in few-shot examples, retrieval, or Fine-Tuning
- Testing cross-lingual or cross-domain transfer when target labels have Semantic Side Information but no labeled target examples
- Comparing Conventional ZSL, GZSL, few-shot, and supervised systems on contamination-audited evaluation splits
Example
Loading code...Frequently Asked Questions
What is the difference between classical Zero-Shot Learning and LLM zero-shot prompting?
Classical ZSL withholds labeled examples for target classes and uses Semantic Side Information to transfer from Seen to Unseen Classes. LLM zero-shot prompting gives a pretrained model an instruction but no Input-Output Demonstrations in the current Prompt and performs no task-specific Gradient Update during that call. They share the word zero but remove different evidence from different stages.
Does zero-shot mean the model never saw the task or label during training?
No. It means a specific protocol provides zero target-class labels, zero Prompt Demonstrations, or zero Dataset-Specific Training. A foundation model may still have encountered related concepts, task formats, labels, or benchmark material during Pretraining or Instruction Tuning. Audit known overlap and mark unverifiable training exposure as unknown rather than claiming the task was unseen.
How do Conventional ZSL and Generalized ZSL differ?
Conventional ZSL evaluates only inputs from Unseen Classes and predicts among Unseen Labels. GZSL evaluates a mixture of Seen and Unseen inputs and predicts over their union, so it reveals bias toward familiar classes. Report Seen and Unseen per-class accuracy separately and combine them with a Harmonic Mean or calibrated trade-off instead of reporting only aggregate accuracy.
Is Zero-Shot Learning the same as Open-Set Recognition or Few-Shot Learning?
No. Open-Set Recognition can reject an input as unknown without identifying its unseen class. Classical ZSL predicts one of the declared Unseen Labels using side information. Few-Shot Learning supplies a small labeled support set or in-prompt Demonstrations for the target, while a valid zero-shot protocol supplies none for the declared zero-shot unit.
How should Zero-Shot Learning be evaluated?
Freeze the model, class and data splits, semantic descriptions, Candidate Labels, Prompt Template, label order, decoding, and transductive access. Audit duplicates, source families, class overlap, side-information leakage, and pretrained-backbone contamination. Report per-class and slice metrics, chance and few-shot or supervised baselines, calibration or abstention, prompt sensitivity, repeated-run uncertainty, and failures rather than one context-free accuracy score.