What is Adapter?
Adapter is base-dependent trainable state that changes a pretrained model's behavior while most base parameters remain frozen. The term can mean an inserted bottleneck module, a broader PEFT component such as LoRA, or the checkpoint artifact that stores the learned adaptation; these meanings should not be treated as identical.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
A canonical bottleneck Adapter inserts a residual module into a frozen network: for hidden state x, a common form is x + W_up σ(W_down x), where W_down projects from hidden width d to bottleneck width m and W_up projects back to d. This adds sequential forward computation and is mechanically different from LoRA, which represents a weight update with low-rank factors and may be mergeable. Frameworks also use Adapter as an umbrella name for LoRA, IA3, soft prompts, and other PEFT state, and as a name for their saved checkpoint. An Adapter artifact is not a standalone model: it depends on the exact base weights, architecture, tokenizer or processor, input template, method configuration, and runtime implementation. Multiple approved Adapters may share one base, but activation, composition, batching, cache use, authorization, and rollback are serving concerns rather than automatic consequences of small checkpoint size.
Key Characteristics
- Uses learned state attached to or composed with a frozen or mostly frozen base model
- May be a bottleneck module, low-rank update, learned prompt, scaling vector, or framework-specific PEFT component
- Has method-specific inference cost and mergeability; a canonical bottleneck module is not equivalent to mergeable LoRA
- Requires exact Base-Adapter compatibility across weights, architecture, tokenizer or processor, template, and implementation
- Creates a separately versioned behavior change that needs task, regression, safety, latency, and parity evaluation
- Supports shared-base variants only when serving limits, cache policy, authorization, observability, and rollback are explicit
Common Use Cases
- Adding a task or domain specialization while retaining an immutable shared base
- Testing multiple adaptation mechanisms under the same data and evaluation protocol
- Shipping compact, independently versioned model variants with explicit base dependencies
- Serving a controlled catalog of tenant, language, or workflow variants on compatible infrastructure
- Rolling back an adapted behavior without misidentifying the underlying base, tokenizer, or template
Example
Loading code...Frequently Asked Questions
Is LoRA an adapter method?
Many libraries use Adapter as an umbrella term that includes LoRA because LoRA adds separately trainable state to a frozen base. Mechanically, however, LoRA's low-rank weight update differs from a canonical bottleneck Adapter inserted into the forward path. State which meaning is intended when discussing architecture, latency, or merging.
What is the difference between an Adapter and PEFT?
PEFT is the broader family of parameter-efficient adaptation methods. Adapter may refer specifically to an inserted bottleneck module, generically to a PEFT component, or to a saved adaptation checkpoint. LoRA, soft prompts, selective parameter updates, and bottleneck modules can all belong to PEFT, but they do not have the same computation or artifact format.
Can every Adapter be merged into the base model?
No. LoRA-style delta weights are often mergeable when the framework, dtype, quantization, and configuration support it. Bottleneck modules and soft prompts change the forward path and generally remain separate. Even a supported merge creates a new full-model artifact that needs finite-output, parity, held-out, safety, and serving validation.
Can an Adapter checkpoint run without its base model?
Usually not. The checkpoint stores learned adaptation state and configuration, not all base weights. Loading requires a compatible base revision and architecture and often the same tokenizer or processor, input template, framework behavior, and target-module names. Record immutable revisions and hashes instead of relying only on a repository name.
Is switching between Adapters automatically safe?
No. A serving system must validate compatibility, authorize catalog entries, bound loaded and active Adapter counts, manage CPU and accelerator caches, isolate tenants, observe latency and quality, and provide rollback. User-controlled local or remote paths can also create a supply-chain risk; expose approved names rather than arbitrary artifact locations.