General-purpose classification
Tabular supervised classification with nonlinear feature interactions.
Scikit-Learn · Class
ExtraTreesClassifier
ExtraTreesClassifier is an ensemble classifier that fits randomized decision trees on dataset sub-samples and averages them. It supports classification with randomized splits and can natively handle missing values for random splits.
How it works
ExtraTreesClassifier builds an ensemble of randomized decision trees on different dataset sub-samples. It averages the trees' outputs to improve predictive accuracy and limit over-fitting; random feature thresholds make the trees more diverse than ordinary decision trees.
Syntax
class sklearn.ensemble.ExtraTreesClassifier(n_estimators=100, *, criterion='gini', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='sqrt', max_leaf_nodes=None, min_impurity_decrease=0.0, bootstrap=False, oob_score=False, n_jobs=None, random_state=None, verbose=0, warm_start=False, class_weight=None, ccp_alpha=0.0, max_samples=None, monotonic_cst=None)Parameters
| Name | Type | Description |
|---|---|---|
n_estimators | int | Number of trees in the forest; default 100. |
criterion | {"gini", "entropy", "log_loss"} | Split-quality criterion: gini, entropy, or log_loss; default gini. |
max_depth | int or None | Maximum tree depth; None expands nodes until stopping conditions are met. |
min_samples_split | int or float | Minimum samples required to split an internal node; default 2. |
min_samples_leaf | int or float | Minimum samples required at a leaf; default 1. |
max_features | {"sqrt", "log2", int, float} or None | Fraction of features considered for each split, or a supported feature-count option; default sqrt. |
bootstrap | bool | Whether bootstrap samples are used; default False. |
n_jobs | int or None | Number of jobs used for fitting and prediction; None means 1. |
random_state | int, RandomState instance or None | Seed or random-state object controlling estimator randomness. |
class_weight | dict, balanced, balanced_subsample or None | Class weighting strategy, including balanced options; default None. |
max_samples | int, float or None | Maximum number of samples per tree when bootstrap=True; default None. |
warm_start | bool | Whether to reuse a previous solution when adding more trees; default False. |
Returns
A fitted ExtraTreesClassifier instance from fit; predict returns class labels and predict_proba returns class probabilities.
Key points
- The classifier is a meta-estimator that combines many randomized decision trees rather than relying on one tree.
- Averaging the trees' predictions is intended to improve accuracy and reduce over-fitting.
- n_estimators sets how many decision trees are fitted; its default is 100.
- criterion selects the split-quality measure, such as Gini impurity, entropy, or log loss.
- max_features controls the feature subset examined when finding each split; the default is sqrt.
- bootstrap determines whether trees are built from bootstrap samples; it defaults to False.
Example: Train and predict
from sklearn.datasets import make_classification
from sklearn.ensemble import ExtraTreesClassifier
X, y = make_classification(n_samples=100, n_features=4, random_state=0)
clf = ExtraTreesClassifier(n_estimators=100, random_state=0)
clf.fit(X, y)
print(clf.predict(X[:3]))Creates a synthetic classification dataset, trains a reproducible forest, and predicts three samples.
When to use it
Missing-value classification
Datasets containing NaN values when random-split handling is acceptable.
Feature selection
Reducing the feature set using tree-derived importance scores.
Weighted classes
Imbalanced classification when class weighting is appropriate.
Interview questions
QWhat is ExtraTreesClassifier?
It is an ensemble classifier that fits many randomized decision trees and averages their predictions. The randomization and averaging improve predictive accuracy while helping control over-fitting.
Follow-up: How does it differ from a random forest? Extra trees randomize split thresholds in addition to selecting randomized features, increasing tree diversity.
QWhat does n_estimators control?
n_estimators controls the number of trees in the forest and defaults to 100. Increasing it generally makes the ensemble more stable, at the cost of additional computation.
Follow-up: What controls parallelism? n_jobs sets the number of jobs used for fitting and prediction; None means one job, while -1 uses all processors.
QHow are splits randomized?
The default max_features is sqrt, so each split considers a random subset of features. The best split among the randomly generated candidate thresholds for those features is selected.
Follow-up: What happens when max_features is 1? The resulting tree is totally random because only one feature is considered at each split.
Related
ExtraTreesRegressorRandomForestClassifierExtraTreeClassifierRandomForestRegressor