What is VAE?
VAE (Variational Autoencoder) is a latent-variable generative model that trains an encoder to approximate the posterior over latent variables and a decoder to model observations conditioned on those variables.
Quick Facts
| Full Name | Variational Autoencoder |
|---|---|
| Created | 2013 (proposed by Diederik P. Kingma and Max Welling) |
| Specification | Official Specification |
How It Works
Latent-variable model
The prior p(z) defines how latent variables are sampled before an observation is known, while the decoder p(x|z) defines a distribution over observations. The true posterior p(z|x) is usually intractable, so the encoder q(z|x) approximates it with shared neural-network parameters. A common diagonal Gaussian encoder outputs a mean and log variance for each latent dimension, but that is a modeling choice rather than the definition of every VAE.
ELBO and reparameterization
The ELBO contains an expected reconstruction term and a KL divergence from q(z|x) to the prior. Maximizing it improves a lower bound on log p(x), not reconstruction alone. Reparameterization expresses a sample as z = mu + sigma * epsilon, with epsilon drawn from a fixed noise distribution, so gradients can reach the encoder parameters. The reconstruction likelihood, prior, KL weighting, and estimator must match the data and objective.
Failure modes and diagnosis
Posterior collapse occurs when q(z|x) approaches the prior and the decoder ignores z; a near-zero aggregate KL can reveal full collapse but can hide dimension-level behavior. Other failures include inactive units, poor aggregate-prior match, oversmoothed reconstructions from an unsuitable likelihood, and latent features that do not support the intended task. Diagnose per-dimension KL, active units, mutual-information estimates, reconstruction slices, prior samples, and downstream probes.
Evaluation and modern use
Choose metrics from the purpose: held-out likelihood estimates for density modeling, reconstruction and perceptual measures for codecs, prior-sample quality and coverage for generation, and labeled downstream tests for representations or anomaly detection. Latent diffusion systems often use a separately trained autoencoder to reduce spatial cost, but the exact architecture and objective vary and are not always a textbook VAE. Record the codec, scaling, and reconstruction contract when interoperating with a diffusion model.
Key Characteristics
- Combines a decoder-defined generative model with an amortized inference network
- Represents each input with an approximate posterior rather than one fixed code
- Optimizes an ELBO containing reconstruction and prior-divergence terms
- Uses reparameterized sampling to estimate gradients through stochastic latents
- Can generate from the prior and encode observations for representation tasks
- Can suffer posterior collapse, inactive units, prior mismatch, and oversmoothing
Common Use Cases
- Learning compact probabilistic representations with measurable reconstruction trade-offs
- Building latent codecs for downstream generative or compression systems
- Conditional generation when latent uncertainty is part of the model
- Anomaly-detection research with calibrated real holdouts and failure analysis
- Semi-supervised or scientific models that require explicit latent variables
Example
Loading code...Frequently Asked Questions
How is a VAE different from a standard autoencoder?
A standard autoencoder usually maps an input to one deterministic code and optimizes reconstruction. A VAE defines a prior, an approximate posterior q(z|x), and a conditional observation model p(x|z), then optimizes an ELBO. Sampling and likelihood modeling are therefore part of the contract, although a trained latent space is not automatically interpretable.
Is VAE loss just reconstruction loss plus KL divergence?
That shorthand describes a common negative-ELBO implementation, but the reconstruction term is the negative log-likelihood under a chosen observation model, and the KL term depends on the approximate posterior and prior. Reduction, scaling, beta weights, and data likelihood change the objective, so report them rather than comparing raw loss values across implementations.
What does the reparameterization trick do?
It moves randomness into an auxiliary variable with a fixed distribution. For a Gaussian encoder, z = mu + sigma times epsilon with epsilon sampled from a standard normal. This allows pathwise gradients to update mu and sigma. It does not make arbitrary discrete sampling differentiable or guarantee a good posterior approximation.
What is posterior collapse in a VAE?
Posterior collapse means the approximate posterior becomes close to the prior and the decoder obtains little useful information from z. It is common with expressive decoders or unfavorable training dynamics. Inspect per-dimension KL, active units, reconstructions, prior samples, and downstream information; a plausible reconstruction alone can hide collapsed latents.
Is the autoencoder in every latent diffusion model a VAE?
No. Latent diffusion requires an encoder-decoder representation, but implementations may use KL-regularized continuous latents, vector-quantized codebooks, adversarial or perceptual losses, or hybrids. Check the exact checkpoint and training objective. Codec scale, channel count, and reconstruction behavior are implementation contracts, not universal VAE constants.