Nonlinear dimensionality reduction
Projecting data whose meaningful structure is nonlinear.
Scikit-Learn · Class
KernelPCA
KernelPCA performs nonlinear dimensionality reduction using kernels. It can also learn an approximate inverse mapping back to the original feature space.
How it works
KernelPCA builds a kernel representation of the samples and extracts principal directions from that representation, enabling nonlinear dimensionality reduction. When inverse transformation is enabled, it learns a regularized mapping from the reduced representation back to the original feature space, so reconstruction is approximate.
Syntax
class sklearn.decomposition.KernelPCA(n_components=None, *, kernel='linear', gamma=None, degree=3, coef0=1, kernel_params=None, alpha=1.0, fit_inverse_transform=False, eigen_solver='auto', tol=0, max_iter=None, iterated_power='auto', remove_zero_eig=False, random_state=None, copy_X=True, n_jobs=None)Parameters
| Name | Type | Description |
|---|---|---|
n_components | int or None, default=None | Number of components; None keeps all nonzero components. |
kernel | {‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘cosine’, ‘precomputed’} or callable, default=‘linear’ | Kernel used for PCA; may be a supported kernel name or callable. |
gamma | float, default=None | Kernel coefficient for RBF, polynomial, and sigmoid kernels; None uses 1/n_features. |
degree | float, default=3 | Degree for polynomial kernels. |
coef0 | float, default=1 | Independent term in polynomial and sigmoid kernels. |
kernel_params | dict, default=None | Additional parameters for the kernel function. |
alpha | float, default=1.0 | Ridge regularization used when learning the inverse transform. |
fit_inverse_transform | bool, default=False | Whether to learn an inverse transformation. |
eigen_solver | {‘auto’, ‘dense’, ‘arpack’, ‘randomized’}, default=‘auto’ | Eigensolver selection. |
tol | float, default=0 | Convergence tolerance for the eigensolver. |
max_iter | int, default=None | Maximum eigensolver iterations; None uses the solver default. |
iterated_power | int or ‘auto’, default=‘auto’ | Number of power iterations for the randomized solver. |
Returns
A fitted KernelPCA transformer; transform returns samples represented in the retained kernel principal components, and inverse_transform returns approximate samples when inverse fitting is enabled.
Key points
- It extends dimensionality reduction beyond linear projections by applying kernel methods.
- The default component setting retains every nonzero component.
- The kernel parameter selects the similarity function used to construct the PCA representation.
- gamma controls the kernel coefficient for RBF, polynomial, and sigmoid kernels.
- degree sets the polynomial degree when the polynomial kernel is selected.
Common mistakes
- Leaving gamma unset makes it depend on the number of input features.
Example: Kernel PCA with an RBF kernel
from sklearn.datasets import load_digits
from sklearn.decomposition import KernelPCA
X, _ = load_digits(return_X_y=True)
transformer = KernelPCA(
n_components=7,
kernel="rbf",
gamma=10,
fit_inverse_transform=True,
alpha=0.1,
)
X_transformed = transformer.fit_transform(X)
X_reconstructed = transformer.inverse_transform(X_transformed)Fits an RBF-kernel projection to the digits data, reduces it to seven components, and computes an approximate reconstruction.
When to use it
Approximate reconstruction
Recovering an approximate representation in the original feature space after compression.
Denoising and compression
Removing noise or compressing structured data with a kernel-based representation.
Interview questions
QWhat problem does KernelPCA solve, and how does it differ from PCA?
KernelPCA is PCA performed in a kernel-induced representation, so it can discover nonlinear structure that ordinary PCA cannot. The class supports kernels such as linear, polynomial, RBF, sigmoid, cosine, precomputed, or a callable.
Follow-up: Why can inverse_transform be imperfect? It learns a mapping back to the original feature space rather than applying an analytical inverse, so the result is only an approximation.
QHow does n_components affect the fitted representation?
n_components limits the number of retained components. If it is None, all nonzero components are kept.
Follow-up: What happens if the requested component count is larger than the available nonzero components? The estimator cannot provide more nonzero components than the kernel decomposition contains.
QHow does KernelPCA compute its components and what controls that computation?
The estimator can use dense, ARPACK, or randomized truncated eigendecomposition depending on eigen_solver and the data. With auto, scikit-learn selects an approach based on input shape and the number of requested components.
Follow-up: When is inverse_transform useful? Enable fit_inverse_transform to learn a regularized back-mapping, then use inverse_transform for approximate reconstruction.
Related
PCAKernelRidgepairwise_kernelsfittransformfit_transforminverse_transform