Keras · Class

optimizers.RMSprop

RMSprop is a Keras optimizer that scales gradients using a discounted moving average of their squared values. It optionally uses centered variance normalization, plain momentum, weight decay, clipping, EMA, loss scaling, and gradient accumulation.

How it works

RMSprop keeps a discounted moving average of squared gradients and scales each gradient by the square root of that average. With centered=True, it also tracks the moving average of gradients to estimate variance; momentum is plain rather than Nesterov momentum.

Syntax

keras.optimizers.RMSprop(learning_rate=0.001, rho=0.9, momentum=0.0, epsilon=1e-07, centered=False, weight_decay=None, clipnorm=None, clipvalue=None, global_clipnorm=None, use_ema=False, ema_momentum=0.99, ema_overwrite_frequency=None, loss_scale_factor=None, gradient_accumulation_steps=None, name=None, **kwargs)

Parameters

Name	Type	Description
`learning_rate`	`float, LearningRateSchedule, or callable`	Learning-rate value, schedule, or no-argument callable; defaults to 0.001.
`rho`	`float`	Discounting factor for old gradients; defaults to 0.9.
`momentum`	`float`	Plain momentum factor; defaults to 0.0.
`epsilon`	`float`	Small numerical-stability constant; defaults to 1e-07.
`centered`	`boolean`	Whether to normalize using estimated variance rather than the uncentered second moment.
`weight_decay`	`float or None`	Optional weight-decay coefficient.
`clipnorm`	`float or None`	Optional per-gradient norm clipping threshold.
`clipvalue`	`float or None`	Optional elementwise gradient clipping threshold.
`global_clipnorm`	`float or None`	Optional global gradient norm clipping threshold.
`use_ema`	`boolean`	Whether to apply exponential moving averages.
`ema_momentum`	`float`	Momentum for the exponential moving average; defaults to 0.99.
`ema_overwrite_frequency`	`int or None`	Optional frequency for overwriting model variables with EMA values.

Returns

An RMSprop optimizer instance.

Key points

The optimizer maintains a discounted moving average of squared gradients.
Each gradient is scaled by dividing it by the square root of that moving average.
learning_rate controls the step-size scale and defaults to 0.001.
rho controls how strongly the moving average retains information from older gradients.
centered=True normalizes gradients using estimated variance; centered=False uses the uncentered second moment.
Centered mode keeps an additional gradient moving average to estimate variance.
use_ema=True enables exponential moving averages for optimizer variables.

Common mistakes

The momentum option does not provide Nesterov momentum; RMSprop's implementation uses plain momentum.
Enabling centered mode adds another moving average, so it changes normalization from the uncentered second moment to an estimated variance.

Example: Use RMSprop with a Keras model

import keras

optimizer = keras.optimizers.RMSprop(
    learning_rate=0.001,
    rho=0.9,
    centered=True,
)

model = keras.Sequential([
    keras.layers.Input(shape=(4,)),
    keras.layers.Dense(1),
])
model.compile(optimizer=optimizer, loss="mse")

Creates a centered RMSprop optimizer and uses it to compile a small Keras model.

When to use it

General model optimization

Training Keras models with adaptive parameter-wise step sizes.

Centered RMSprop

Using variance-based normalization for gradient updates.

EMA-based training

Combining optimizer updates with moving-average model weights.

Weight decay and clipping

Applying built-in regularization and gradient-safety controls.

Interview questions

QHow does RMSprop adapt its updates?

RMSprop maintains a discounted moving average of squared gradients and divides gradients by the root of that average. This adaptively scales updates across parameters.

Follow-up: What does centered mode change? It also tracks the moving average of gradients and uses it to estimate gradient variance.

QWhat are the default learning rate and rho, and what does rho mean?

The default learning rate is 0.001, and rho defaults to 0.9. rho is the discounting factor for old gradients.

Follow-up: Can learning_rate be scheduled? Yes; the documentation allows a LearningRateSchedule or a no-argument callable that returns the value to use.

QHow can RMSprop be supplied when compiling a model?

RMSprop can be passed as an optimizer instance to model.compile(), or selected by the string identifier "rmsprop". A string uses the optimizer's default parameters.

Follow-up: When would an instance be preferable? Use an instance when configuring non-default options such as learning_rate, centered, clipping, or EMA.

keras.optimizers.SGD
keras.optimizers.Adam
keras.optimizers.AdamW
keras.optimizers.Optimizer

Source: RMSprop - Keras. Checked 2026-10-15.