Adapt training after validation stagnation
Validation loss stops decreasing during model.fit.
Keras · Class
ReduceLROnPlateau lowers an optimizer's learning rate when a monitored metric stops improving. Configure the monitored metric, improvement direction, patience, reduction factor, cooldown, and minimum learning rate.
ReduceLROnPlateau watches a selected training metric and lowers the optimizer learning rate after the metric fails to improve for the configured patience period. It can enforce a minimum learning rate and pause monitoring after each reduction.
keras.callbacks.ReduceLROnPlateau(monitor="val_loss", factor=0.1, patience=10, verbose=0, mode="auto", min_delta=0.0001, cooldown=0, min_lr=0.0, **kwargs)| Name | Type | Description |
|---|---|---|
monitor | string | Metric or quantity to monitor. |
factor | float | Multiplier applied to the current learning rate when reducing it; default 0.1. |
patience | integer | Number of unimproved epochs before reducing the learning rate; default 10. |
verbose | integer | Logging level: 0 is quiet and 1 emits update messages. |
mode | string | Direction of improvement: auto, min, or max. |
min_delta | float | Minimum change treated as a new optimum; default 0.0001. |
cooldown | integer | Epochs to wait after a reduction before normal monitoring resumes; default 0. |
min_lr | float | Lower bound for the learning rate; default 0.0. |
**kwargs | keyword arguments | Additional callback keyword arguments. |
A keras.callbacks.ReduceLROnPlateau callback instance.
import keras
model = keras.Sequential([
keras.layers.Input(shape=(784,)),
keras.layers.Dense(10, activation="softmax"),
])
model.compile(optimizer="adam", loss="sparse_categorical_crossentropy")
reduce_lr = keras.callbacks.ReduceLROnPlateau(
monitor="val_loss", factor=0.2, patience=5, min_lr=0.001
)
model.fit(
x_train, y_train,
validation_data=(x_val, y_val),
epochs=20,
callbacks=[reduce_lr],
)Creates the callback and passes it to model.fit so validation-loss plateaus trigger learning-rate reductions.
Validation loss stops decreasing during model.fit.
A metric should improve while preserving a nonzero learning-rate floor.
Training needs automatic direction handling for common metric names.
It monitors the configured quantity and waits for the specified number of epochs without improvement. It then multiplies the learning rate by factor, subject to min_lr.
Follow-up: How would you reduce a validation metric that should increase? Set mode="max", or use auto when Keras can infer the direction from the metric name.
factor controls the multiplicative reduction, so a factor of 0.2 changes lr to lr * 0.2. patience controls how many unimproved epochs occur before that reduction.
Follow-up: What prevents the rate from becoming too small? Set min_lr to the desired lower bound.
mode accepts auto, min, or max. min treats decreasing values as improvement, max treats increasing values as improvement, and auto infers the direction from the monitored quantity's name.
Follow-up: When might explicit mode be preferable? Use it when the metric name is ambiguous or its desired direction is not inferred correctly.
keras.callbacks.Callbackkeras.callbacks.EarlyStoppingkeras.callbacks.LearningRateSchedulerModel.fit