LLM Fine-Tuning【2026】: SFT, LoRA, QLoRA, and Evaluation

TL;DR

Fine-tuning changes model parameters or adds trainable adapters so a base model behaves differently on a target distribution. It is useful for stable behavior, output structure, tool-call conventions, or domain-specific task patterns. It is a poor substitute for a frequently changing knowledge base, and it does not automatically make private data safe, factual, authorized, or removable.

Start with a baseline prompt and retrieval system. Fine-tune only when a measured failure persists, the training data is licensed and representative, and the team can run held-out capability, safety, privacy, and regression evaluations. Treat every memory, quality, cost, and benchmark number as an experiment result tied to a model revision, tokenizer, sequence length, optimizer, hardware, and date.

What Fine-Tuning Changes

Supervised fine-tuning (SFT) minimizes a loss on selected target tokens. Full fine-tuning updates all or most base-model weights. Parameter-efficient fine-tuning (PEFT) freezes the base model and trains an adapter, such as a LoRA update. Preference optimization methods such as DPO use preference data and a different objective; they are not interchangeable with SFT.

Fine-tuning can improve:

• a stable response format or style;
• classification, extraction, routing, or tool-call patterns;
• behavior on a well-defined domain distribution;
• a smaller model's task performance when the data and evaluation are strong.

It does not reliably provide:

• a current database of facts;
• authorization or tenant isolation;
• deletion of a memorized example;
• immunity to prompt injection or unsafe outputs;
• a guarantee that a model will cite or follow every training example.

Choose the Intervention

RAG and fine-tuning can coexist: training can teach format and retrieval-use behavior, while retrieval supplies current, permission-filtered evidence. Retrieval content remains untrusted data, not instructions.

Full Fine-Tuning, LoRA, and QLoRA

Full Fine-Tuning

Updating all weights can provide high capacity but requires memory for weights, gradients, optimizer states, activations, temporary buffers, and sometimes multiple checkpoints. A “7B model needs X GB” statement is incomplete without precision, optimizer, sequence length, micro-batch, checkpointing, parallelism, and framework details. Estimate and measure the complete training job instead of copying a fixed number.

LoRA

LoRA adds a low-rank update to selected modules:

W' = W + scale × B × A

The rank, target modules, scaling, dropout, and initialization are design choices. The fraction of trainable parameters and quality change depend on architecture and configuration; ranges such as “0.1–1%” are not universal guarantees. Adapters are convenient to version and compose, but multiple adapters can conflict and merging changes numerical behavior.

QLoRA

QLoRA combines an adapter with quantized frozen base weights. NF4, double quantization, paged optimizers, compute dtype, kernel support, and hardware affect the result. Quantization reduces memory in many setups, but it does not imply a fixed quality or memory outcome. Validate the quantized model against the same baseline and task slices.

Dataset Engineering

Provenance and Consent

For every record, retain source, license or permission, collection date, transformation steps, annotator/reviewer, and deletion lineage. Remove secrets, unnecessary personal data, credentials, and data whose use is not authorized. A model checkpoint or adapter is not automatically easy to scrub when a source is withdrawn.

Split Before Transformation

Create train, validation, and test identities before paraphrasing, chunking, augmentation, or conversation templating. Deduplicate near-duplicates across splits. Keep a contamination set for benchmark and evaluation prompts. Never tune hyperparameters on the final test set.

Format and Quality

Use the model's documented chat template and preserve role boundaries. Validate JSONL records, token lengths, empty turns, language, labels, tool-call arguments, and refusal examples. Review both ordinary and adversarial cases. Do not use a fixed rule such as “100–10,000 examples” as a target; sample efficiency depends on task entropy, base-model capability, label noise, and coverage.

import json
from pathlib import Path


def read_instruction_jsonl(path: str):
    for line_number, line in enumerate(Path(path).read_text(encoding="utf-8").splitlines(), 1):
        if not line.strip():
            continue
        record = json.loads(line)
        if not isinstance(record.get("instruction"), str):
            raise ValueError(f"line {line_number}: instruction must be text")
        if not isinstance(record.get("output"), str) or not record["output"].strip():
            raise ValueError(f"line {line_number}: output must be non-empty text")
        yield {
            "instruction": record["instruction"].strip(),
            "input": str(record.get("input", "")).strip(),
            "output": record["output"].strip(),
        }

This validates shape only. It does not validate factuality, licensing, privacy, or task correctness.

A Reproducible Training Run

Framework APIs and model licenses change. Pin the base-model revision, tokenizer revision, Python/CUDA/framework versions, dataset hash, chat template, random seeds, hardware, precision, and training configuration. Record them in an experiment manifest.

The following is an illustrative SFT skeleton. It intentionally avoids claiming a universal model ID, target-module list, sequence length, or learning rate:

# Illustrative skeleton: pin and test exact package APIs before production use.
from dataclasses import dataclass


@dataclass(frozen=True)
class RunConfig:
    base_revision: str
    dataset_revision: str
    seed: int
    max_sequence_length: int
    learning_rate: float
    effective_batch_size: int


def build_training_record(sample, tokenizer):
    messages = [
        {"role": "user", "content": sample["instruction"] + "\n" + sample["input"]},
        {"role": "assistant", "content": sample["output"]},
    ]
    return tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=False,
    )


# Select SFTTrainer/SFTConfig or an equivalent framework API matching the
# pinned versions. Use a train/validation split, seed, checkpoint policy,
# gradient accumulation, and label masking appropriate to the task.

For a real run, confirm that the tokenizer's chat template, padding side, EOS behavior, label mask, truncation policy, quantization compute dtype, and target modules match the base model. trust_remote_code=True executes repository code and should require a reviewed, pinned source rather than being a default.

Use gradient accumulation deliberately:

effective batch = devices × per-device micro-batch × accumulation steps

It changes optimization dynamics and does not make a job equivalent to unlimited memory. Monitor tokens per second, peak memory, loss, validation loss, gradient anomalies, checkpoint integrity, and training interruptions.

Evaluation Before Deployment

Compare the untouched base model, prompt-only baseline, RAG baseline when relevant, and each fine-tuned candidate on the same frozen test set. Include:

• task accuracy or exact-match metrics appropriate to the task;
• structured-output validity and schema compliance;
• citation/evidence correctness when retrieval is used;
• calibration or abstention behavior;
• safety, privacy, jailbreak, prompt-injection, and tool-use slices;
• multilingual, long-context, rare-case, and out-of-distribution slices;
• latency, throughput, memory, cost, and failure rates.

BLEU and ROUGE can be useful for narrow generation tasks, but they are not general truth, helpfulness, or safety metrics. Use multiple references or task-specific rubrics where appropriate, blind human review, inter-rater agreement, and uncertainty intervals. Report model revision, prompts, data, baseline, metric definition, hardware, precision, sampling policy, date, and sample count.

def exact_match_rate(predictions, references):
    if len(predictions) != len(references) or not references:
        raise ValueError("predictions and references must have equal non-zero length")
    normalized = lambda value: " ".join(value.split()).casefold()
    return sum(
        normalized(prediction) == normalized(reference)
        for prediction, reference in zip(predictions, references)
    ) / len(references)

Do not reuse training examples for evaluation, report only a cherry-picked prompt, or infer factuality from loss alone. A lower validation loss can coexist with worse instruction following, safety, or memorization.

Hyperparameters and Ablations

Learning rate, rank, alpha, dropout, epochs, packing, sequence length, batch size, warmup, optimizer, and quantization are interacting variables. Start with a small, documented sweep and change one meaningful factor at a time. Save the exact configuration and compare against a baseline.

Useful ablations include:

• base model versus adapter;
• different data subsets and deduplication policies;
• target modules and rank;
• full precision versus quantized loading;
• prompt-only versus RAG versus SFT;
• with and without potentially contaminated or synthetic records.

Do not copy a paper's learning rate or hardware result without matching its model, data, objective, and evaluation protocol. A paper's result is evidence for its setup, not a universal recipe.

Deployment and Governance

Keep the base model, adapter, tokenizer, chat template, prompt, safety policy, dependency lockfile, and experiment manifest versioned together. Verify model and adapter compatibility before loading. Scan artifacts and logs for secrets and personal data.

At serving time, the runtime owns:

• authentication, tenant/object authorization, and tool allowlists;
• input/output schema validation and content controls;
• rate limits, budgets, timeouts, cancellation, and audit logging;
• retrieval permissions and deletion propagation;
• rollback and traffic shadowing.

An adapter is not a license to use its training data, and a private checkpoint is not proof that data is inaccessible. Define retention, deletion, access, export, and incident procedures before training.

Common Failure Modes

Frequently Asked Questions

How much data is enough?

There is no universal count. Begin with a representative, licensed pilot and an untouched test set. Learning curves, error slices, annotation quality, and baseline gaps provide stronger evidence than a 100/1,000/10,000 rule.

What does the LoRA rank control?

Rank controls the capacity of the low-rank update, not a guaranteed quality level. Higher rank can increase capacity, trainable parameters, and memory, while also increasing overfitting risk. Compare ranks in a controlled ablation.

Does fine-tuning add reliable domain knowledge?

It can improve behavior on the training distribution, but it is not a queryable, current knowledge store. Use permission-filtered retrieval for changing facts and evaluate factuality rather than assuming memorization is correct.

Can fine-tuning remove private data?

Not reliably. Deleting a record from the dataset does not prove it has been removed from checkpoints, adapters, optimizer states, caches, logs, or derivatives. Plan data lineage and deletion testing before training.

Should fine-tuning and RAG be combined?

Often, yes. Fine-tuning can teach stable formats or retrieval behavior; RAG can supply current evidence. Keep retrieval authorization and prompt-injection defenses in the runtime.

How should a fine-tuned model be evaluated?

Use frozen task tests, baseline comparisons, safety/privacy and out-of-distribution slices, structured-output checks, human review, and cost/latency measurements. Publish the protocol and uncertainty, not only an attractive score.

Primary Sources

• Hu et al.: LoRA
• Dettmers et al.: QLoRA
• Rafailov et al.: Direct Preference Optimization
• Hugging Face PEFT documentation
• Hugging Face TRL documentation
• NIST AI Risk Management Framework
• OWASP Top 10 for Large Language Model Applications

Conclusion

Fine-tuning is an experimental intervention, not a universal upgrade. Establish a baseline, use data with clear provenance, pin the complete environment, measure the full resource path, and evaluate quality, safety, privacy, generalization, cost, and uncertainty on held-out slices. Use LoRA or QLoRA when their operational trade-offs fit the experiment, and keep authorization and governance outside the model weights.