Small-sample validation
Estimating model performance when the dataset is small and retaining nearly all observations for each training run matters.
Scikit-Learn · Function
LeaveOneOut
LeaveOneOut is a cross-validator that holds out one sample at a time while using all remaining samples for training. It creates as many splits as there are samples.
How it works
LeaveOneOut generates one train/test split for every sample. In each split, exactly one sample is the test set and all other samples are used for training, so the number of splits equals the number of samples.
Syntax
class sklearn.model_selection.LeaveOneOut()Returns
An iterator yielding pairs of train and test index arrays.
Key points
- Use it as an index generator for repeatedly separating training rows from held-out rows.
- Every fold tests exactly one observation, with all other observations available for training.
- The same partitioning can be expressed with KFold using one fold per sample or LeavePOut with one omitted sample.
Common mistakes
- It can be computationally expensive because it trains and evaluates once for every sample.
Example: Iterate leave-one-out splits
import numpy as np
from sklearn.model_selection import LeaveOneOut
X = np.array([[1, 2], [3, 4], [5, 6]])
y = np.array([0, 1, 1])
loo = LeaveOneOut()
for train_idx, test_idx in loo.split(X, y):
print("train:", train_idx, "test:", test_idx)Creates one split per sample and prints the training and singleton test indices.
When to use it
Observation sensitivity
Inspecting how predictions change when each individual observation is omitted.
Custom cross-validation loops
Generating explicit train/test index pairs for custom evaluation loops.
Interview questions
QHow does LeaveOneOut construct its splits?
LeaveOneOut is a cross-validator that creates one split for each sample. The held-out test set contains one sample, while every other sample forms the training set.
Follow-up: How many splits are produced for n samples? Exactly n splits, because each sample is held out once.
QWhat does LeaveOneOut return?
Its test sets are singletons, and the corresponding training sets contain all remaining samples. The splitter returns train and test indices rather than fitting an estimator itself.
Follow-up: What does split return? It yields pairs of index arrays identifying the training and test rows for each fold.
QHow does LeaveOneOut relate to other cross-validation splitters?
LeaveOneOut is equivalent to KFold with n_splits equal to the number of samples and to LeavePOut with p equal to 1. Its main drawback is the potentially high computation cost.
Follow-up: When might another strategy be preferable? For large datasets, use a cross-validation method with fewer folds to reduce repeated model fitting.
Related
KFoldLeavePOutget_n_splitssplit