Balance model weaknesses
Several different classifiers have comparable performance but make different errors.
Scikit-Learn · Class
VotingClassifier
VotingClassifier combines named classifiers by majority labels or aggregated probabilities. It supports hard and soft voting, optional classifier weights, and parallel fitting.
- Since
- 0.17
How it works
VotingClassifier clones and fits the supplied named classifiers, then combines their outputs. Hard voting selects the majority class label, while soft voting selects the class with the largest sum of predicted probabilities and is intended for well-calibrated classifiers.
Syntax
class sklearn.ensemble.VotingClassifier(*estimators*, ***, voting='hard', weights=None, n_jobs=None, flatten_transform=True, verbose=False)Parameters
| Name | Type | Description |
|---|---|---|
estimators | list of (str, estimator) tuples | Named estimator pairs to be cloned and fitted. |
voting | {'hard', 'soft'}, default='hard' | Voting rule: hard majority labels or soft probabilities; default is hard. |
weights | array-like of shape (n_classifiers,), default=None | Numeric classifier weights; default is None. |
n_jobs | int, default=None | Parallel jobs used during fit; default is None. |
flatten_transform | bool, default=True | Whether transform output is flattened; default is True. |
verbose | bool, default=False | Print elapsed fitting time when true; default is False. |
Returns
A fitted VotingClassifier with prediction, probability, scoring, and transformation methods documented on the API page.
Key points
- VotingClassifier provides an ensemble classifier for combining initially unfitted estimators.
- fit clones each supplied estimator rather than fitting the original estimator objects directly.
- With hard voting, each classifier contributes its predicted class label and the majority determines the result.
- With soft voting, the class having the greatest summed probability is selected.
- weights can give classifiers unequal influence in both voting modes.
- Leaving weights unset gives every classifier equal influence.
Common mistakes
- Soft voting relies on predicted probabilities, so use it when the component classifiers are well calibrated.
Example: Soft-voting classifier
from sklearn.datasets import load_iris
from sklearn.ensemble import VotingClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
X, y = load_iris(return_X_y=True)
clf = VotingClassifier(
estimators=[
("lr", make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))),
("svc", make_pipeline(StandardScaler(), SVC(probability=True))),
],
voting="soft",
)
clf.fit(X, y)
print(clf.predict(X[:3]))Trains two probability-producing classifiers and combines their predictions with soft voting.
When to use it
Probability-based ensemble
Class probabilities are meaningful and calibrated across component models.
Weighted voting
Some classifiers should contribute more strongly than others.
Interview questions
QHow do hard and soft voting differ?
It combines multiple unfitted classifiers using either majority labels or probabilities. With hard voting, the selected class is the mode of the component predictions; with soft voting, it is the class corresponding to the largest sum of predicted probabilities.
Follow-up: How would classifier importance be changed? Supply weights; they weight hard-vote label occurrences or soft-vote probabilities before aggregation.
QWhat does VotingClassifier do during fit?
The estimators argument is a list of name-estimator pairs. Calling fit clones those original estimators and stores the fitted clones in estimators_.
Follow-up: Why are names needed? They identify estimators for parameter access and model configuration in the estimator interface.
QHow do weights affect ensemble predictions?
weights accepts one numeric weight per classifier. If it is None, all classifiers receive uniform weight; otherwise weights influence label counts in hard voting or probability averaging in soft voting.
Follow-up: What does n_jobs control? It controls the number of parallel jobs used during fit; None means one job unless a joblib backend changes the context, and -1 uses all processors.
Related
VotingRegressorClassifierMixinPipelineGridSearchCV