Heterogeneous ensembles
Combine diverse classifiers whose errors differ.
Scikit-Learn · Class
StackingClassifier
StackingClassifier combines predictions from named base classifiers with a final classifier. It trains the final classifier on cross-validated base predictions to improve generalization.
- Since
- 0.22
How it works
The class fits named base classifiers on the complete training data, creates cross-validated predictions from them, and trains a final classifier on those predictions. Its prediction interface can optionally include the original features when passthrough is enabled.
Syntax
class sklearn.ensemble.StackingClassifier(*estimators, final_estimator=None, *, cv=None, stack_method='auto', n_jobs=None, passthrough=False, verbose=0)Parameters
| Name | Type | Description |
|---|---|---|
estimators | list of (str, estimator) | Named base classifiers to stack; each item is a (name, estimator) pair. |
final_estimator | estimator, default=None | Classifier that combines base-estimator outputs; defaults to LogisticRegression. |
cv | int, cross-validation generator, iterable, or 'prefit', default=None | Cross-validation splitter or strategy used to create predictions for the final classifier; may be an integer, splitter, iterable, or 'prefit'. |
stack_method | {'auto', 'predict_proba', 'decision_function', 'predict'}, default='auto' | Base-estimator method used to produce stacked features; 'auto' selects an available method. |
n_jobs | int, default=None | Number of jobs used when fitting base estimators; None means one job. |
passthrough | bool, default=False | Whether to append the original input features to the stacked outputs. |
verbose | int, default=0 | Controls verbosity during fitting. |
Returns
A fitted StackingClassifier exposing classification prediction methods such as predict and predict_proba.
Key points
- It is an ensemble classifier that combines several estimators through a final classifier.
- Base estimators are supplied as uniquely named pairs, allowing them to be addressed within the model.
- Leaving final_estimator unset selects LogisticRegression as the combining model.
- cv controls how cross-validation generates training predictions for the final estimator.
Example: Stacking classifiers on Iris
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, StackingClassifier
X, y = load_iris(return_X_y=True)
model = StackingClassifier(
estimators=[
("rf", RandomForestClassifier(n_estimators=10, random_state=0)),
("lr", LogisticRegression(max_iter=1000)),
],
final_estimator=LogisticRegression(max_iter=1000),
cv=5,
)
model.fit(X, y)
print(model.predict(X[:3]))This example trains two base classifiers and uses cross-validated outputs to train a logistic-regression final classifier.
When to use it
Meta-classification
Use a learned classifier to combine probability, decision, or class predictions.
Feature passthrough
Retain raw input information alongside estimator outputs.
Interview questions
QWhat does StackingClassifier do?
It combines outputs from several base classifiers and feeds those outputs into a final classifier. The base estimators are fitted on all training data, while the final estimator learns from cross-validated base predictions.
Follow-up: Why use cross-validated predictions? They are out-of-sample predictions for the training rows, which reduces the risk that the final estimator learns from overly optimistic in-sample outputs.
QHow are the base and final estimators specified?
The estimators parameter is a list of name-estimator pairs. The final_estimator combines their outputs, and if it is omitted, LogisticRegression is used by default.
Follow-up: How can a base estimator be excluded? Set its parameter to 'drop' through set_params.
QHow does stack_method control the stacked features?
stack_method selects each base estimator's output method, such as predict_proba, decision_function, or predict. With 'auto', the class tries methods in that order and uses the first available one.
Follow-up: What does passthrough change? When true, the original input features are appended to the base-estimator outputs before final-estimator training and prediction.
Related
StackingRegressorVotingClassifiercross_val_predictLogisticRegression