Latent semantic analysis
Sparse text feature matrices
Scikit-Learn · Class
TruncatedSVD
TruncatedSVD reduces feature matrices to a chosen number of dimensions using truncated singular value decomposition without centering the data. It efficiently supports sparse count and TF-IDF matrices, where the transformation is commonly called latent semantic analysis (LSA).
How it works
TruncatedSVD computes a low-rank linear projection from the largest singular components without centering the input. Its sparse-matrix support makes it suitable for reducing count or TF-IDF feature matrices, where the result is commonly used as latent semantic analysis.
Syntax
class sklearn.decomposition.TruncatedSVD(n_components=2, *, algorithm='randomized', n_iter=5, n_oversamples=10, power_iteration_normalizer='auto', random_state=None, tol=0.0)Parameters
| Name | Type | Description |
|---|---|---|
n_components | int | Desired dimensionality of the output data; default 2. |
algorithm | {'arpack', 'randomized'} | SVD solver: 'arpack' or 'randomized'; default 'randomized'. |
n_iter | int | Number of iterations for the randomized SVD solver; default 5. |
n_oversamples | int | Number of additional random vectors used by the randomized solver; default 10. |
power_iteration_normalizer | {'auto', 'QR', 'LU', 'none'} | Normalization strategy for randomized power iterations; default 'auto'. |
random_state | int, RandomState instance or None | Controls randomness for reproducible randomized results; default None. |
tol | float | Tolerance for ARPACK convergence; default 0.0. |
Returns
fit_transform(X, y=None) returns the transformed data with shape (n_samples, n_components); transform(X) returns X projected into the fitted lower-dimensional space.
Key points
- Use TruncatedSVD to replace a high-dimensional feature matrix with a lower-dimensional representation; in text processing this is also called LSA.
- The absence of centering allows the transformer to process sparse matrices efficiently without first densifying them.
- When applied to term-count or TF-IDF matrices, the reduced representation is referred to as latent semantic analysis.
- Choose between the default randomized solver for speed and the ARPACK-based solver when that algorithm is appropriate.
- Set
n_componentsto determine how many dimensions the transformed data should contain. - Increase
n_iterwhen the randomized solver needs additional iterations to improve its approximation.
Common mistakes
- Unlike PCA, TruncatedSVD does not center features before decomposition; apply centering only when your data representation and memory constraints permit it.
- The solver choice matters: ARPACK uses SciPy's sparse eigensolver, whereas randomized SVD uses a different approximation method.
Example: Reduce sparse text features with TruncatedSVD
from sklearn.decomposition import TruncatedSVD
from sklearn.feature_extraction.text import TfidfVectorizer
texts = ["cats chase mice", "dogs chase balls", "cats and dogs"]
X = TfidfVectorizer().fit_transform(texts)
svd = TruncatedSVD(n_components=2, random_state=42)
X_reduced = svd.fit_transform(X)
print(X_reduced.shape)This creates a sparse TF-IDF matrix, fits a two-component truncated SVD, and transforms documents into the reduced space.
When to use it
Memory-efficient dimensionality reduction
Large sparse feature matrices
Lower-dimensional modeling
Feature compression before a downstream estimator
Interview questions
QHow does TruncatedSVD differ from PCA?
It performs linear dimensionality reduction by retaining a requested number of leading singular components. Unlike PCA, it operates without first centering the input, so it can efficiently accept sparse matrices.
Follow-up: Why is it useful for text? Term-count and TF-IDF matrices are sparse, and projecting them onto truncated singular components provides a lower-dimensional semantic representation.
QWhat algorithms can TruncatedSVD use?
The randomized solver is the default and uses a randomized SVD method. arpack calls SciPy's ARPACK-based truncated SVD implementation.
Follow-up: What parameter controls the randomized solver's iteration count? n_iter controls the number of iterations used by that solver.
QWhat does n_components control?
n_components specifies the desired dimensionality of the transformed output and defaults to 2. Calling fit_transform learns the decomposition and returns the input projected into that lower-dimensional space.
Follow-up: Which learned attributes help assess the retained structure? components_, explained_variance_, explained_variance_ratio_, and singular_values_ expose the fitted decomposition and variance information.
Related
PCANMFIncrementalPCADictionaryLearningFactorAnalysis