Keep the strongest individually discriminative features before training a classifier.
Classification preprocessing
Scikit-Learn · Class
SelectKBest
SelectKBest ranks features using a univariate scoring function and keeps the k highest-scoring ones. It is useful for reducing feature dimensionality before fitting an estimator.
- Since
- 0.18
How it works
SelectKBest evaluates every feature independently with the supplied scoring function, ranks the resulting scores, and keeps the k highest-scoring columns during transformation. The selector learns feature scores when fitted and can then reduce new data to the same selected columns.
Syntax
class sklearn.feature_selection.SelectKBest(score_func=<function f_classif>, *, k=10)Parameters
| Name | Type | Description |
|---|---|---|
score_func | callable, default=f_classif | Callable that accepts X and y and returns scores, or scores plus p-values; defaults to f_classif. |
k | int or "all", default=10 | Number of top features to retain, or "all" to bypass selection; defaults to 10. |
Returns
A fitted feature selector whose transform method retains the selected columns; learned attributes include scores_ and pvalues_.
Key points
- The class keeps the k features with the greatest scores produced by score_func.
- A scoring function can accept X and y and return both feature scores and p-values.
- A scorer may return only scores when p-values are not needed.
- The k parameter controls how many highest-ranked features remain.
- After fitting, scores_ stores one score for each original feature.
- pvalues_ contains p-values when the scorer supplies them; otherwise it is None.
- Unsupervised selection is supported when the scoring function needs only X.
Common mistakes
- Equal feature scores do not guarantee a reproducible selection order.
- The default f_classif scorer is intended for classification rather than regression.
- Use k="all" when a parameter search should include the no-selection option.
Example: Select the top two features
from sklearn.datasets import load_iris
from sklearn.feature_selection import SelectKBest, f_classif
X, y = load_iris(return_X_y=True)
X_selected = SelectKBest(score_func=f_classif, k=2).fit_transform(X, y)
print(X_selected.shape)This selects the two features with the highest ANOVA F-test scores from the Iris dataset.
When to use it
Reduce a wide dataset to a fixed number of top-ranked columns.
Dimensionality reduction
Include k="all" in a parameter search to compare selection with bypassing it.
Model selection
Select features with a scorer that computes values from X without requiring y.
Unsupervised filtering
Interview questions
QWhat does SelectKBest do?
SelectKBest is a univariate feature selector: it scores each feature independently and retains the k features with the largest scores. Its default scorer is f_classif, and k defaults to 10.
Follow-up: How would you avoid selecting features using information from the test set? Put SelectKBest inside a Pipeline so fitting the selector occurs only on training folds.
QWhat can score_func return?
score_func receives X and y and may return either scores alone or a pair containing scores and p-values. The learned scores_ attribute contains one score per input feature, while pvalues_ is None when the scorer returns only scores.
Follow-up: What is required for unsupervised use? Supply a scoring function that accepts y=None and computes scores using X.
QHow does SelectKBest integrate with preprocessing?
fit runs the scoring function and determines the selected features; fit_transform combines fitting with transformation. transform then projects data onto the selected feature columns, preserving the selected feature count.
Follow-up: How can the selector be disabled during a search? Include k="all", which bypasses feature selection.
Related
SelectPercentileGenericUnivariateSelectf_classifchi2f_regressionmutual_info_classif