Out-of-core dimensionality reduction
A dataset is too large to load into memory for ordinary PCA.
Scikit-Learn · Class
IncrementalPCA
IncrementalPCA performs PCA-style SVD-based dimensionality reduction while processing data in batches. It is designed for large or sparse inputs when ordinary batch PCA would use too much memory.
How it works
IncrementalPCA applies SVD to centered, unscaled features while processing data in batches. It can update the decomposition with partial_fit, keeping memory tied mainly to batch size and feature count rather than the full sample count.
Syntax
class sklearn.decomposition.IncrementalPCA(n_components=None, *, whiten=False, copy=True, batch_size=None)Parameters
| Name | Type | Description |
|---|---|---|
n_components | int, default=None | Number of components to keep. If None, all components are kept. |
whiten | bool, default=False | When True, whiten the component outputs. Default is False. |
copy | bool, default=True | If False, overwrite input data during fitting to save memory. Default is True. |
batch_size | int, default=None | Number of samples per batch for fit. If None, inferred as 5 * n_features. |
Returns
An IncrementalPCA estimator instance; fitting methods return the fitted estimator, while transform and fit_transform return data represented in component space.
Key points
- The class uses SVD to project observations into a lower-dimensional component space.
- It is a memory-efficient alternative to batch PCA for sufficiently large datasets.
- Sparse input is supported, which can avoid densifying large sparse datasets.
- Its working-memory scale depends on batch size and feature count, not directly on total sample count.
- batch_size sets the sample count processed per fit batch.
Common mistakes
- Features are centered but not standardized; scale them separately when differing feature magnitudes would distort the decomposition.
- batch_size does not control partial_fit calls; it is used when fit performs the batching.
- With fit, an omitted batch_size is inferred rather than left unlimited.
Example: Reduce Iris features incrementally
from sklearn.decomposition import IncrementalPCA
from sklearn.datasets import load_iris
X, _ = load_iris(return_X_y=True)
ipca = IncrementalPCA(n_components=2, batch_size=10)
X_reduced = ipca.fit_transform(X)
print(X_reduced.shape)Fits a two-component incremental decomposition in small batches and transforms the Iris features into the reduced space.
When to use it
Incremental updates with partial_fit
Samples arrive sequentially from storage or a data service.
Memory-mapped fitting
A large dataset is stored as a NumPy memory map.
Sparse feature reduction
The input matrix is sparse and should remain memory-efficient.
Interview questions
QWhat problem does IncrementalPCA solve?
IncrementalPCA performs linear dimensionality reduction with SVD, retaining the most significant singular vectors. Unlike ordinary PCA's batch-only processing, it can consume data in minibatches and update the model incrementally.
Follow-up: How can it handle data that does not fit in memory? Call partial_fit on sequential chunks, or call fit on a NumPy memory-mapped file.
QHow does batch size affect resource usage?
Its memory usage is approximately proportional to batch_size times n_features, so it does not need all samples resident at once. Increasing batch_size generally increases memory consumption while influencing approximation behavior.
Follow-up: What is the default batch size for fit? When batch_size is None, it is inferred as 5 times the number of features.
QWhat do the principal constructor options control?
n_components selects how many components are retained. whiten controls whether transformed component outputs are rescaled to have unit variance, while copy determines whether input data may be overwritten during fitting.
Follow-up: Does IncrementalPCA scale each original feature before SVD? No; it centers the features but does not scale them.
Related
PCAKernelPCASparsePCAMiniBatchDictionaryLearning