Compare training and validation behavior
Diagnose whether changing one hyperparameter improves generalization.
Scikit-Learn · Function
Evaluate an estimator across values of one parameter and collect cross-validation training and test scores. The result helps reveal how that parameter affects model performance.
The function clones and evaluates an estimator for every value in param_range. Cross-validation produces one training-score and one test-score for each parameter value and fold, using the estimator’s default scorer unless scoring is supplied.
sklearn.model_selection.validation_curve(estimator, X, y, *, param_name, param_range, groups=None, cv=None, scoring=None, n_jobs=None, pre_dispatch='all', verbose=0, error_score=nan, params=None)| Name | Type | Description |
|---|---|---|
estimator | estimator object | Estimator implementing fit and predict; it is cloned for each validation. |
X | array-like | Training input samples. |
y | array-like | Target values. |
param_name | str | Name of the estimator parameter to vary. |
param_range | array-like of shape (n_values,) | Candidate values to evaluate for the selected parameter. |
groups | array-like, default=None | Groups used by the cross-validation splitter, when applicable. |
cv | int, cross-validation generator or iterable, default=None | Cross-validation splitter specification. |
scoring | str, callable or None, default=None | Scoring strategy; defaults to the estimator’s score method when omitted. |
n_jobs | int, default=None | Number of jobs run in parallel. |
pre_dispatch | int or str, default='all' | Number of jobs dispatched ahead of execution. |
verbose | int, default=0 | Controls the verbosity of parallel execution. |
error_score | ‘raise’ or numeric, default=nan | Value assigned when a fitting or scoring error occurs. |
A tuple containing train_scores and test_scores, each an array of shape (n_ticks, n_cv_folds).
import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import validation_curve
from sklearn.svm import SVC
X, y = load_iris(return_X_y=True)
train_scores, test_scores = validation_curve(
SVC(kernel="linear"),
X,
y,
param_name="C",
param_range=np.logspace(-3, 2, 5),
cv=5,
scoring="accuracy",
)
print(train_scores.shape, test_scores.shape)Evaluates a linear SVM across five C values and five cross-validation folds, returning training and test scores for each combination.
Diagnose whether changing one hyperparameter improves generalization.
Create a plot of model performance across regularization or complexity values.
Inspect fold-level variation for a single parameter sweep.
validation_curve evaluates an estimator over several values of one named parameter and returns training and test scores. It resembles a one-parameter grid search, but also reports training scores and is mainly useful for plotting.
Follow-up: How would you inspect variability across folds? Use the returned arrays’ fold dimension to compute a mean and spread for each parameter value.
The returned train_scores and test_scores have shape (n_ticks, n_cv_folds). Each row corresponds to one value from param_range, and each column corresponds to one cross-validation fold.
Follow-up: What should be plotted? Typically plot the mean across folds against param_range, with dispersion such as standard deviation shown separately.
cv controls the cross-validation splitting strategy and may be an integer, a cross-validation generator, or an iterable. If cv is omitted, the function uses its documented default splitting behavior.
Follow-up: How can evaluation use a custom metric? Pass a scorer through scoring, such as a scorer name or callable accepted by scikit-learn.
sklearn.model_selection.ValidationCurveDisplaysklearn.model_selection.cross_validatesklearn.model_selection.GridSearchCVsklearn.model_selection.learning_curve