What is Truncated SVD?
Truncated SVD (Truncated Singular Value Decomposition) is a matrix-factorization method that computes or retains only the leading singular values and vectors to form a rank-limited approximation.
Quick Facts
| Full Name | Truncated Singular Value Decomposition |
|---|---|
| Created | 1936 low-rank approximation theorem by Eckart and Young; LSA application published in 1990 |
| Specification | Official Specification |
How It Works
Separate the mathematical target from the solver
The target is a rank-k approximation formed from leading singular triplets. An exact dense SVD, a Krylov eigensolver such as ARPACK, and randomized range finding are different ways to estimate that target. The randomized matrix decomposition framework first captures an approximate range, compresses the matrix, and finishes the small factorization deterministically.
Record solver, seed, oversampling, power iterations, tolerance, component signs, and rank. A slowly decaying spectrum can require more iterations or oversampling, while nearly tied singular values make individual axes unstable even when their joint subspace is reliable.
Keep sparse text geometry explicit
Current scikit-learn TruncatedSVD documentation emphasizes that it does not center input and can therefore operate efficiently on sparse matrices. Applied to term-count or TF-IDF matrices, this use is commonly called Latent Semantic Analysis.
The original LSA paper used SVD to approximate term-document associations. The learned factors remain linear combinations determined by vocabulary, weighting, and corpus composition; they are not automatically coherent topics or modern contextual embeddings.
Choose rank with reconstruction and task evidence
Larger k lowers training reconstruction error but increases memory, latency, and exposure to weak or noisy directions. Explained-variance fields are implementation diagnostics on uncentered transformed data and must not be interpreted as PCA's centered covariance result without checking the definition.
Fit vocabulary, weighting, and decomposition on training data only. Evaluate held-out reconstruction, retrieval or clustering metrics, stability across seeds and corpus samples, and performance on rare terms. Keep the fitted projection with the vocabulary and preprocessing version so new rows enter the same space.
Key Characteristics
- Retains only the leading singular values and left and right singular vectors
- Provides an optimal exact rank-limited approximation under standard matrix norms
- Can process sparse matrices efficiently because common transformers do not center input
- Supports iterative or randomized solvers with distinct accuracy and reproducibility controls
- Produces linear latent factors whose signs and near-tied axes are not uniquely identified
- Requires rank selection against held-out reconstruction and downstream objectives
Common Use Cases
- Compressing sparse TF-IDF document matrices for Latent Semantic Analysis
- Building a low-rank baseline before clustering or retrieval experiments
- Reducing large user-item or interaction matrices when missingness semantics are understood
- Approximating dense matrices for storage or downstream numerical work
- Inspecting spectral decay and effective low-rank structure
Example
Loading code...Frequently Asked Questions
How is Truncated SVD different from PCA?
PCA applies SVD or eigendecomposition after centering features. Common Truncated SVD transformers factor the input without centering, which preserves sparse storage but changes the components. They coincide only under compatible centering and projection conventions.
Why is Truncated SVD used for sparse text matrices?
Subtracting a feature mean usually turns a sparse term-document matrix dense. Truncated SVD can work on the uncentered sparse matrix, reducing memory and producing a lower-dimensional linear representation used in Latent Semantic Analysis.
How should the number of Truncated SVD components be chosen?
Compare rank values on held-out reconstruction and the actual retrieval, clustering, or prediction metric while tracking memory and latency. Spectral decay is useful context, but no fixed explained-variance threshold guarantees semantic quality.
Why can randomized Truncated SVD change between runs?
Randomized solvers sample an approximate matrix range, so the seed, oversampling, power iterations, and spectrum affect the result. Fix and record those settings, then compare subspaces and downstream behavior rather than requiring identical signs for every component.
Is Truncated SVD the same as Latent Semantic Analysis?
Truncated SVD is the general matrix operation. LSA is a text-retrieval application that applies it to a weighted term-document matrix and interprets the low-rank factors as latent associations. The factors do not automatically form named or coherent topics.