What is SFT?
Updated on
SFT (Supervised Fine-Tuning) is a fine-tuning method that continues training a pretrained model on labeled input-target examples by maximizing the likelihood of designated target tokens.
Quick Facts
| Full Name | Supervised Fine-Tuning |
|---|---|
| Specification | Official Specification |
How It Works
Inspect the Labels That Actually Receive Loss
Render representative examples with the pinned Tokenizer and Chat Template, shift labels once for next-token prediction, and inspect which positions remain after Padding and Loss Masks. Full-sequence, completion-only and assistant-only policies optimize different targets. Current [TRL SFTTrainer documentation](https://huggingface.co/docs/trl/main/en/sft_trainer) supports several dataset formats and loss policies, and may compute the same NLL objective through a memory-saving chunked implementation. Pin the library revision instead of treating its defaults as the SFT definition.
Build Data Around Production Failure Slices
Define the expected output and rejection behavior first, then curate examples across real intents, languages, lengths, difficulty and edge cases. Record provenance, license, consent, label policy, reviewer disagreement, deduplication, contamination checks and any Synthetic Teacher. Keep train, validation and test identities separate. AWS's current [SFT data guidance](https://aws.amazon.com/blogs/machine-learning/preparing-data-for-supervised-fine-tuning-part-1-formatting-and-quality/) emphasizes formatting, quality, diversity and consistency; its product-specific schemas and sample-count guidance are not universal thresholds.
Test the Same Template and Artifact Used in Serving
Training and inference must agree on role markers, Special Tokens, system content, tool schemas and termination behavior. The [Transformers chat-template guide](https://huggingface.co/docs/transformers/main/en/chat_templating) warns that models derived from the same base can require different control tokens and that duplicated Special Tokens can hurt behavior. Export or compose the intended serving artifact, rerun target, regression, safety and latency gates, compare with the base and prompt-only baselines, and retain a tested rollback.
Key Characteristics
- Optimizes labeled targets with token-level or task-specific supervised loss rather than pairwise preferences or scalar rewards
- Separates the supervised objective from the parameter-update strategy, which may be full-parameter, selective, or PEFT-based
- Makes full-sequence, completion-only, or assistant-only Loss Mask policy explicit and distinct from the Attention Mask
- Binds Tokenizer or Processor, Chat Template, Special Tokens, truncation, Packing, and label construction to the run identity
- Requires licensed and deduplicated data, immutable splits, label provenance, contamination checks, and task-mixture records
- Evaluates held-out behavior, capability regressions, safety, serving parity, resource use, and rollback readiness
Common Use Cases
- Training a model to produce a stable extraction, classification, transformation, or structured-output contract
- Teaching instruction-following, conversational, tool-use, or multimodal behavior from reviewed demonstrations
- Adapting response style and domain workflows while evaluating factual knowledge separately
- Building a supervised baseline before deciding whether preference optimization adds measurable value
- Correcting recurrent behaviors that remain unreliable under a maintainable prompt or retrieval baseline
Example
Loading code...Frequently Asked Questions
How are SFT, Fine-tuning, and Instruction Tuning different?
Fine-tuning is the parent process of continuing optimization from a pretrained checkpoint. SFT specifies supervised input-target labels and an objective. Instruction Tuning is an SFT pattern that expresses tasks as instructions and desired responses, often across task families. SFT can also cover labeled completion, classification, extraction, tool-use, or multimodal mappings that are not designed for broad instruction following.
Should SFT always compute loss only on Assistant responses?
No. Full-sequence, completion-only, and Assistant-only loss are different policies. Current TRL defaults depend on the dataset format, and Assistant-only Loss requires a compatible conversational template with generation masks. Choose the mask from the intended behavior, inspect generated labels and non-masked Token counts, version the policy, and compare held-out results instead of treating one setting as universal.
How much SFT data is required?
There is no portable example-count threshold. The requirement depends on task diversity, label quality and disagreement, target behavior, base model, Tokenizer, sequence-length distribution, synthetic-data correlation, and acceptable error rates. Use immutable splits and learning curves, inspect per-slice failures and near-duplicates, and add data only when it produces reliable held-out gains.
Does lower SFT Loss mean the model is ready to deploy?
No. Loss measures fit only on labels selected by the mask and can fall while the model overfits, memorizes contaminated examples, loses general capabilities, or fails under its serving template. A release gate should compare base and prompt-only baselines on held-out target, format, language, domain, safety, refusal, regression, latency, and exact serving-artifact tests.
When should a team use SFT instead of prompting, RAG, or preference optimization?
Start with measurable prompting and retrieval baselines. Use SFT when reviewed demonstrations need to change stable task behavior or output patterns in model weights. Use retrieval for current or attributable knowledge, and preference methods when relative judgments among plausible answers provide the needed signal. These methods can be combined, but SFT alone does not guarantee fresh facts, human preference alignment, safety, or tool authorization.