Boost weak classifiers
Classification tasks where a sequence of weak learners can progressively correct earlier mistakes.
Scikit-Learn · Class
AdaBoostClassifier
AdaBoostClassifier is a classification meta-estimator that sequentially fits classifiers while emphasizing difficult training examples. It combines the fitted classifiers, with n_estimators and learning_rate controlling the ensemble.
How it works
AdaBoostClassifier first trains a classifier on the original data, then adds further copies whose training emphasis shifts toward previously misclassified samples. Their weighted combination produces the final classification, with learning_rate controlling each estimator’s influence.
Syntax
class sklearn.ensemble.AdaBoostClassifier(estimator=None, *, n_estimators=50, learning_rate=1.0, random_state=None)Parameters
| Name | Type | Description |
|---|---|---|
estimator | estimator or None, default=None | Base estimator from which the boosted ensemble is built; the current signature defaults it to None. |
n_estimators | int, default=50 | Maximum number of estimators used for boosting; training may stop earlier after a perfect fit. |
learning_rate | float, default=1.0 | Weight applied to each classifier at each boosting iteration. |
random_state | int, RandomState instance or None, default=None | Controls the randomness of the estimator. |
Returns
A fitted AdaBoostClassifier instance after fit; prediction methods return class labels or class probabilities as documented for the estimator API.
Key points
- The class is a classification meta-estimator in sklearn.ensemble.
- Later learners concentrate more on examples earlier learners classified incorrectly.
- n_estimators specifies the upper limit on boosting stages.
- learning_rate scales each classifier’s influence during boosting.
- Larger learning_rate values give every fitted classifier more influence.
- n_estimators accepts integer values starting at one.
Common mistakes
- The supplied estimator must support sample weights and expose the expected class metadata.
- Training can finish before n_estimators when the ensemble achieves a perfect fit.
Example: Train and evaluate AdaBoostClassifier
from sklearn.datasets import make_classification
from sklearn.ensemble import AdaBoostClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score
X, y = make_classification(n_samples=300, n_features=4, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.25, random_state=0
)
model = AdaBoostClassifier(n_estimators=50, learning_rate=1.0, random_state=0)
model.fit(X_train, y_train)
print(accuracy_score(y_test, model.predict(X_test)))Creates a classification dataset, trains AdaBoost, and evaluates test-set accuracy.
When to use it
Focus on hard examples
Datasets where incorrectly classified training samples should receive greater emphasis in later stages.
Tune ensemble capacity
Experiments requiring control over ensemble size and per-stage estimator contribution.
Interview questions
QHow does AdaBoostClassifier work?
It is a meta-estimator that starts with a classifier on the original dataset and then fits additional copies while increasing emphasis on incorrectly classified instances. The resulting classifiers are combined for the final prediction.
Follow-up: What makes an estimator compatible? It must support sample weighting and provide valid classes_ and n_classes_ attributes.
QWhat does n_estimators control?
n_estimators sets the maximum number of boosting stages. The process may terminate earlier if the current ensemble fits perfectly.
Follow-up: What happens if the value is invalid? n_estimators must be at least 1.
QWhat is the role of learning_rate?
learning_rate scales the contribution of each classifier at every boosting iteration. Increasing it makes each classifier contribute more strongly.
Follow-up: How can model capacity be adjusted alongside it? Trade off learning_rate against the number of estimators during validation.
Related
AdaBoostRegressorsklearn.ensembleDecisionTreeClassifier