Dimensionality reduction
Reduce a supervised model's input dimensionality before final training.
Scikit-Learn · Class
RFE
RFE selects features by repeatedly fitting an importance-producing estimator and removing the least important features. It returns rankings and a Boolean mask for the retained subset.
How it works
RFE repeatedly fits a supervised estimator, reads its feature-importance information, removes the least important current features, and continues until the requested subset size remains. The fitted selector exposes the chosen mask, ranking, and estimator refit on the reduced data.
Syntax
class sklearn.feature_selection.RFE(estimator, *, n_features_to_select=None, step=1, verbose=0, importance_getter='auto')Parameters
| Name | Type | Description |
|---|---|---|
estimator | Estimator instance | Supervised estimator with fit and feature-importance information, such as coef_ or feature_importances_. |
n_features_to_select | int or float, default=None | Number or fraction of features to retain; defaults to half when None. |
step | int or float, default=1 | Number of features or fraction of features removed at each iteration. |
verbose | int, default=0 | Controls verbosity of output. |
importance_getter | str or callable, default='auto' | Feature-importance attribute path or callable; default is automatic extraction. |
Returns
A fitted feature selector exposing support_, ranking_, n_features_, and the reduced fitted estimator through estimator_.
Key points
- RFE ranks features while recursively eliminating less important ones.
- Leaving
n_features_to_selectunset retains half of the input features. - An integer
n_features_to_selectspecifies the exact number of retained features. - A fractional
stepremoves that percentage of the current features, rounding down. - A callable importance getter can derive one importance value per feature from the fitted estimator.
- The documented usage includes retrieving a fixed number of the most informative features.
Example: Select five features with RFE
from sklearn.datasets import make_friedman1
from sklearn.feature_selection import RFE
from sklearn.linear_model import LinearRegression
X, y = make_friedman1(n_samples=100, n_features=10, random_state=0)
selector = RFE(LinearRegression(), n_features_to_select=5)
selector.fit(X, y)
print(selector.support_)
print(selector.ranking_)Fits RFE with linear regression and prints the selected-feature mask and feature rankings.
When to use it
Feature ranking
Inspect the relative usefulness of input variables through recursive rankings.
Model-driven selection
Use models exposing coefficients or feature importances to select a compact subset.
Interview questions
QHow does RFE select features?
RFE fits the supplied estimator on the current feature set, obtains feature importance, removes the least important features, and repeats until the target count is reached. The estimator must provide feature-importance information.
Follow-up: How can importance extraction be customized? Set importance_getter to an attribute path or callable; a callable receives the fitted estimator and returns feature importances.
QWhat do step and n_features_to_select control?
An integer step removes that many features per iteration. A fractional step between zero and one removes the corresponding percentage, rounded down.
Follow-up: What happens when n_features_to_select is omitted? The selector defaults to retaining half of the input features.
QWhich fitted attributes expose RFE's result?
After fitting, support_ is a Boolean mask for selected features, ranking_ gives each feature's rank, and estimator_ is the estimator fitted on the reduced feature set.
Follow-up: Which rank denotes a selected feature? Rank 1 identifies selected features; larger ranks indicate features eliminated earlier or later in the recursive process.
Related
RFECVSelectFromModelSequentialFeatureSelector