What is MDS?
MDS (Multidimensional Scaling) is a family of methods that places objects in a low-dimensional coordinate space so that distances among the coordinates approximate supplied dissimilarities.
Quick Facts
| Full Name | Multidimensional Scaling |
|---|---|
| Created | Classical scaling developed in the 1930s; nonmetric MDS formalized by Roger Shepard and Joseph Kruskal in the 1960s |
| Specification | Official Specification |
How It Works
Distinguish classical, metric, and nonmetric objectives
Classical MDS double-centers squared dissimilarities to obtain a Gram-like matrix, then uses its leading positive eigenpairs as coordinates. If the input contains exact Euclidean distances from centered observations, this result matches PCA scores up to rotation and reflection. Negative eigenvalues reveal that the dissimilarities are not exactly realizable in the requested Euclidean geometry.
Metric MDS directly minimizes discrepancies between dissimilarity magnitudes and embedding distances. Nonmetric MDS instead fits a monotonic transformation so the rank order is respected; Kruskal's original formulation introduced a quantitative Stress criterion for that hypothesis.
Treat dissimilarity construction as part of the model
A distance matrix can come from numeric features, ratings, edit costs, domain comparisons, or graph-derived quantities. It must have an explicit scale, missing-value policy, symmetry rule, and diagonal convention. A method that accepts arbitrary dissimilarities does not make them statistically meaningful or Euclidean.
Inspect triangle-inequality violations when metric geometry matters, negative eigenvalues for classical MDS, duplicated objects, disconnected or censored pairs, and sensitivity to transformations. Do not interpret axes as named features unless an external analysis supports that meaning.
Evaluate Stress, initialization, and scalability together
Iterative stress minimization is non-convex and can produce different local solutions, so compare multiple initializations and record the best and distribution of fit values. Current scikit-learn MDS documentation supports metric and nonmetric fitting, precomputed dissimilarities, classical initialization, and normalized or raw Stress variants.
Pairwise matrices require quadratic storage, limiting full MDS at large sample counts. Compare fit across target dimensions, use held-out or resampled dissimilarities when possible, and inspect residuals by distance range rather than accepting one aggregate Stress value or an attractive map.
Key Characteristics
- Accepts a pairwise dissimilarity matrix as its central input
- Returns coordinates defined only up to rigid transformations and translation
- Includes classical eigendecomposition, metric Stress, and nonmetric rank-preserving variants
- Can reveal non-Euclidean input through negative classical eigenvalues or persistent residuals
- Usually requires quadratic pairwise storage and computation
- Needs explicit fit, stability, residual, and interpretability checks
Common Use Cases
- Visualizing objects available only through pairwise dissimilarities
- Recovering approximate coordinates from geographic or network distance tables
- Mapping human similarity judgments while preserving their ordinal structure
- Auditing whether a distance definition admits a useful low-dimensional representation
- Providing the final distance-embedding stage inside Isomap
Example
Loading code...Frequently Asked Questions
What is the difference between classical, metric, and nonmetric MDS?
Classical MDS eigendecomposes a double-centered squared-distance matrix. Metric MDS fits distance magnitudes by minimizing Stress. Nonmetric MDS fits a monotonic relationship and primarily preserves dissimilarity order. Their objectives and diagnostics are not interchangeable.
How is classical MDS related to PCA?
When dissimilarities are Euclidean distances computed from centered observations, classical MDS recovers the same principal coordinate subspace as PCA, up to rotation, reflection, and component signs. MDS can also begin from distances when original features are unavailable.
What does MDS Stress measure?
Stress measures disagreement between supplied dissimilarities, or fitted disparities in nonmetric MDS, and distances in the output configuration. Its normalization and weighting convention matter, so values from different formulations are not automatically comparable.
Why do MDS results change between runs?
Iterative metric and nonmetric MDS can converge to different local minima from different initial coordinates. Use multiple starts, fixed seeds for reproducibility, convergence checks, and solution alignment before comparing configurations.
Can MDS embed a new object without refitting?
Standard MDS is usually transductive because coordinates are fitted jointly from all pairwise dissimilarities. Out-of-sample extensions exist for particular classical or landmark formulations, but they require distances to a frozen reference set and separate validation.