What is SFT?
SFT (Supervised Fine-Tuning) is a fine-tuning method that continues training a pretrained model on labeled input-target examples by maximizing the likelihood of designated target tokens.
Quick Facts
| Full Name | Supervised Fine-Tuning |
|---|---|
| Specification | Official Specification |
How It Works
SFT describes the supervised objective, not one mandatory post-training pipeline or parameter-update method. Fine-tuning is the parent concept; Instruction Tuning is a common SFT use aimed at instruction following across tasks, while SFT also covers labeled completion, classification, extraction, tool-use, multimodal, and other target mappings.
A causal language model commonly minimizes token-level negative log-likelihood, -sum over t in M of log p(y_t | input, y before t), using ground-truth previous tokens during training. The target mask M is part of the experiment: full-sequence, completion-only, and assistant-only loss optimize different positions, and Assistant-only Loss is not universally correct. An Attention Mask controls which tokens participate in attention, while a Loss Mask controls which predictions contribute to the objective.
The same run must bind the base model, parameter-update strategy, Tokenizer or Processor, Chat Template, Special Tokens, truncation and Packing policy, immutable data splits, label provenance, and synthetic Teacher identity. Training or validation Loss alone does not prove task quality, factuality, safety, preserved capabilities, or serving parity.
Release evidence compares prompt-only and base-model baselines on held-out task, format, language, domain, safety, refusal, regression, latency, and exact serving-artifact slices. SFT can teach represented behavior, but it does not guarantee knowledge freshness, preference alignment, authorization, or safe tool use.
Key Characteristics
- Optimizes labeled targets with token-level or task-specific supervised loss rather than pairwise preferences or scalar rewards
- Separates the supervised objective from the parameter-update strategy, which may be full-parameter, selective, or PEFT-based
- Makes full-sequence, completion-only, or assistant-only Loss Mask policy explicit and distinct from the Attention Mask
- Binds Tokenizer or Processor, Chat Template, Special Tokens, truncation, Packing, and label construction to the run identity
- Requires licensed and deduplicated data, immutable splits, label provenance, contamination checks, and task-mixture records
- Evaluates held-out behavior, capability regressions, safety, serving parity, resource use, and rollback readiness
Common Use Cases
- Training a model to produce a stable extraction, classification, transformation, or structured-output contract
- Teaching instruction-following, conversational, tool-use, or multimodal behavior from reviewed demonstrations
- Adapting response style and domain workflows while evaluating factual knowledge separately
- Building a supervised baseline before deciding whether preference optimization adds measurable value
- Correcting recurrent behaviors that remain unreliable under a maintainable prompt or retrieval baseline
Example
Loading code...Frequently Asked Questions
How are SFT, Fine-tuning, and Instruction Tuning different?
Fine-tuning is the parent process of continuing optimization from a pretrained checkpoint. SFT specifies supervised input-target labels and an objective. Instruction Tuning is an SFT pattern that expresses tasks as instructions and desired responses, often across task families. SFT can also cover labeled completion, classification, extraction, tool-use, or multimodal mappings that are not designed for broad instruction following.
Should SFT always compute loss only on Assistant responses?
No. Full-sequence, completion-only, and Assistant-only loss are different policies. Current TRL defaults depend on the dataset format, and Assistant-only Loss requires a compatible conversational template with generation masks. Choose the mask from the intended behavior, inspect generated labels and non-masked Token counts, version the policy, and compare held-out results instead of treating one setting as universal.
How much SFT data is required?
There is no portable example-count threshold. The requirement depends on task diversity, label quality and disagreement, target behavior, base model, Tokenizer, sequence-length distribution, synthetic-data correlation, and acceptable error rates. Use immutable splits and learning curves, inspect per-slice failures and near-duplicates, and add data only when it produces reliable held-out gains.
Does lower SFT Loss mean the model is ready to deploy?
No. Loss measures fit only on labels selected by the mask and can fall while the model overfits, memorizes contaminated examples, loses general capabilities, or fails under its serving template. A release gate should compare base and prompt-only baselines on held-out target, format, language, domain, safety, refusal, regression, latency, and exact serving-artifact tests.
When should a team use SFT instead of prompting, RAG, or preference optimization?
Start with measurable prompting and retrieval baselines. Use SFT when reviewed demonstrations need to change stable task behavior or output patterns in model weights. Use retrieval for current or attributable knowledge, and preference methods when relative judgments among plausible answers provide the needed signal. These methods can be combined, but SFT alone does not guarantee fresh facts, human preference alignment, safety, or tool authorization.