Fast boosted-tree classification
Large tabular classification datasets
Scikit-Learn · Class
HistGradientBoostingClassifier
Histogram-based gradient boosting classifier for fast tree ensembles on large datasets. It natively handles NaNs and offers controls for regularization, early stopping, categorical features, and constraints.
- Since
- 0.21
How it works
The classifier builds a gradient-boosted ensemble of decision trees using feature-value histograms to select split points. It can handle missing values directly, learns how missing observations should flow at each split, and supports classification controls such as regularization, early stopping, categorical features, and monotonic or interaction constraints.
Syntax
class sklearn.ensemble.HistGradientBoostingClassifier(loss='log_loss', *, learning_rate=0.1, max_iter=100, max_leaf_nodes=31, max_depth=None, min_samples_leaf=20, l2_regularization=0.0, max_features=1.0, max_bins=255, categorical_features='from_dtype', monotonic_cst=None, interaction_cst=None, warm_start=False, early_stopping='auto', scoring='loss', validation_fraction=0.1, n_iter_no_change=10, tol=1e-07, verbose=0, random_state=None, class_weight=None)Parameters
| Name | Type | Description |
|---|---|---|
loss | str | The loss function used for boosting; the documented default is log_loss. |
learning_rate | float | Shrinkage applied to each boosting iteration; default 0.1. |
max_iter | int | Maximum number of boosting iterations; default 100. |
max_leaf_nodes | int or None | Maximum number of leaves per tree; default 31. |
max_depth | int or None | Maximum tree depth; default None. |
min_samples_leaf | int | Minimum number of samples per leaf; default 20. |
l2_regularization | float | L2 regularization coefficient; default 0.0. |
max_features | float | Fraction of features considered at each iteration; default 1.0. |
max_bins | int | Maximum number of bins for continuous features; default 255. |
categorical_features | array-like, boolean, or 'from_dtype' | How categorical features are identified; default 'from_dtype'. |
monotonic_cst | array-like or None | Monotonicity constraints; default None. |
interaction_cst | array-like or None | Allowed feature-interaction groups; default None. |
Returns
A fitted HistGradientBoostingClassifier instance from fit; prediction methods return class labels or class probabilities.
Key points
- This class is a classification estimator built from histogram-based gradient-boosted trees.
- It is intended to provide a major speed improvement over GradientBoostingClassifier when the sample count is large.
- For every split, training learns the better branch for samples whose relevant feature is missing.
Common mistakes
- The documented speed advantage is aimed at large datasets; for smaller data, do not assume it will be faster.
- Missing-value support does not mean arbitrary preprocessing is unnecessary for every other data-quality issue; only NaNs are explicitly covered here.
Example: Train and predict with histogram gradient boosting
from sklearn.datasets import load_iris
from sklearn.ensemble import HistGradientBoostingClassifier
X, y = load_iris(return_X_y=True)
model = HistGradientBoostingClassifier(random_state=0)
model.fit(X, y)
print(model.predict_proba(X[:2]))Loads the Iris data, trains the classifier, and prints class probabilities for two samples.
When to use it
Classification without mandatory imputation
Datasets containing NaNs
Monotonic or interaction-constrained modeling
Applications requiring controlled feature relationships
Interview questions
QWhat is HistGradientBoostingClassifier designed for?
It is a histogram-based gradient-boosted classification tree estimator. The documentation highlights it as substantially faster than GradientBoostingClassifier on datasets with at least about 10,000 samples.
Follow-up: When would you choose the conventional estimator instead? Use the conventional implementation when the dataset is small or when its API or behavior better matches the project requirements.
QHow does it handle missing values?
It accepts NaNs without requiring imputation. During training, the tree grower learns whether missing-valued samples should take the left or right branch at each split according to potential gain.
Follow-up: What happens for a feature that had no missing values during training? The documentation states that missing values at prediction time are sent to the child containing more samples.
QWhich parameters commonly control model complexity and training?
Its constructor exposes boosting and tree controls such as learning_rate, max_iter, max_leaf_nodes, max_depth, min_samples_leaf, and l2_regularization. It also provides early-stopping, categorical-feature, monotonic-constraint, and interaction-constraint options.
Follow-up: Which parameter replaces the traditional estimator's tree-count terminology? max_iter controls the number of boosting iterations, rather than using n_estimators.
Related
GradientBoostingClassifierHistGradientBoostingRegressorGradientBoostingRegressor