Diagnose underfitting and overfitting
Compare training and validation performance as dataset size grows.
Scikit-Learn · Function
learning_curve
Computes cross-validated training and test scores for multiple training-set sizes. It helps assess how estimator performance changes as more training data is provided.
How it works
The function clones and fits the estimator repeatedly on progressively sized subsets of the training data. For every size, it evaluates both training and held-out test partitions across the cross-validation splits, then returns the scores; optional timing arrays are included when requested.
Syntax
sklearn.model_selection.learning_curve(estimator, X, y, *, groups=None, train_sizes=array([0.1, 0.33, 0.55, 0.78, 1.]), cv=None, scoring=None, exploit_incremental_learning=False, n_jobs=None, pre_dispatch='all', verbose=0, shuffle=False, random_state=None, error_score=nan, return_times=False, params=None)Parameters
| Name | Type | Description |
|---|---|---|
estimator | object | Estimator implementing fit; it is cloned for each validation. |
X | array-like or sparse matrix | Training features with shape (n_samples, n_features). |
y | array-like or None | Targets for classification or regression; may be None. |
cv | int, cross-validation generator, or iterable | Cross-validation splitter or integer; default None. |
train_sizes | array-like | Relative or absolute training-example counts; default np.linspace(0.1, 1.0, 5). |
scoring | str or callable | Metric name or callable; default None. |
exploit_incremental_learning | bool | Use incremental learning when supported; default False. |
n_jobs | int or None | Number of parallel jobs; default None. |
shuffle | bool | Whether to shuffle before selecting training prefixes; default False. |
random_state | int, RandomState, or None | Seed or random-state object used when shuffling; default None. |
error_score | numeric or 'raise' | Behavior on fitting or scoring errors; default nan. |
return_times | bool | Include fit and score times in the result; default False. |
Returns
Returns train_sizes, train_scores, and test_scores; with return_times=True, also returns fit_times and score_times.
Key points
- Use learning_curve to study how model performance changes as progressively larger training sets are used.
- Each requested size creates a smaller or larger subset for fitting within every cross-validation run.
- Scores are aggregated across folds separately for each training-size value.
- The scoring argument determines the metric applied to both the training and held-out test portions.
Common mistakes
- Values in train_sizes may be relative fractions or absolute example counts; the function uses them to select the training-set sizes.
- The chosen cross-validation strategy determines how the data is partitioned for each repeated fit and evaluation.
- The supplied estimator must implement fit and is cloned separately for each validation run.
Example: Evaluate learning curves
from sklearn.datasets import load_iris
from sklearn.model_selection import learning_curve
from sklearn.tree import DecisionTreeClassifier
X, y = load_iris(return_X_y=True)
model = DecisionTreeClassifier(random_state=0)
sizes, train_scores, test_scores = learning_curve(
model, X, y, cv=5, train_sizes=[0.2, 0.5, 1.0],
scoring="accuracy", n_jobs=-1
)
print(sizes)
print(test_scores.mean(axis=1))Computes cross-validated training and test accuracy for three training-set sizes.
When to use it
Assess value of more data
Estimate whether additional labeled examples may improve results.
Profile fit and scoring time
Measure computational cost at different training sizes.
Interview questions
QWhat does learning_curve return?
learning_curve evaluates an estimator at several training-set sizes. For each size, it performs cross-validation, computes training and test scores, and averages those scores across folds.
Follow-up: How can fit and scoring costs be measured? Set return_times=True; the result then includes fit-time and score-time arrays in addition to sizes and scores.
QWhat are the main controls for the experiment?
train_sizes specifies the evaluated training-set sizes, either as relative fractions or absolute counts. cv controls the splitting strategy, while scoring selects the metric used for training and test evaluation.
Follow-up: What happens when scoring is omitted? The estimator's default scoring method is used.
QWhat inputs does the function require?
The estimator must provide fit because it is cloned and trained repeatedly. X contains the features, and y contains the corresponding classification or regression targets, or can be None where supported.
Follow-up: Why is cloning useful? It gives each validation run an independent unfitted estimator, preventing state from leaking between runs.
Related
LearningCurveDisplaycross_validatevalidation_curvemodel_selection