Keras · Class

optimizers.SGD

keras.optimizers.SGD performs gradient descent with optional momentum and Nesterov acceleration. It supports schedules, weight decay, gradient clipping, EMA, loss scaling, and gradient accumulation.

How it works

SGD updates each trainable parameter using its gradient and learning rate. Momentum adds a velocity term that accelerates movement in consistent directions and reduces oscillation; enabling Nesterov changes the update to use a look-ahead form. The optimizer can also apply EMA, decay, clipping, accumulation, and scheduled learning rates.

Syntax

keras.optimizers.SGD(learning_rate=0.01, momentum=0.0, nesterov=False, weight_decay=None, clipnorm=None, clipvalue=None, global_clipnorm=None, use_ema=False, ema_momentum=0.99, ema_overwrite_frequency=None, loss_scale_factor=None, gradient_accumulation_steps=None, name=None, **kwargs)

Parameters

Name	Type	Description
`learning_rate`	`float \| keras.optimizers.schedules.LearningRateSchedule \| callable`	Float, LearningRateSchedule, or zero-argument callable determining the step size; defaults to 0.01.
`momentum`	`float`	Nonnegative momentum coefficient; defaults to 0.0.
`nesterov`	`boolean`	Whether to use Nesterov momentum; defaults to false.
`weight_decay`	`float \| None`	Optional weight-decay coefficient.
`clipnorm`	`float \| None`	Optional per-gradient norm clipping threshold.
`clipvalue`	`float \| None`	Optional per-gradient value clipping threshold.
`global_clipnorm`	`float \| None`	Optional global gradient norm clipping threshold.
`use_ema`	`boolean`	Whether to apply exponential moving averages; defaults to false.
`ema_momentum`	`float`	EMA momentum; defaults to 0.99.
`ema_overwrite_frequency`	`int \| None`	Number of update steps between EMA weight overwrites.
`loss_scale_factor`	`float \| None`	Optional loss-scale factor.
`gradient_accumulation_steps`	`int \| None`	Optional number of steps over which gradients are accumulated.

Returns

An instance of keras.optimizers.SGD, an optimizer that applies gradient-descent updates to trainable variables.

Key points

SGD is Keras's gradient-descent optimizer, optionally enhanced with momentum.
learning_rate sets the scale of each gradient-based parameter update and defaults to 0.01.
momentum=0 gives standard SGD; larger nonnegative values add a velocity that can speed progress and damp oscillations.
nesterov selects the Nesterov form of the momentum update and defaults to false.
use_ema enables exponential moving averages of model weights during training.
A learning-rate schedule can vary the optimizer's step size as training proceeds.

Common mistakes

momentum must be nonnegative; setting it to zero selects ordinary, non-momentum gradient descent.

Example: SGD with momentum

import keras

model = keras.Sequential([
    keras.layers.Dense(10, activation="softmax", input_shape=(4,))
])
model.compile(
    optimizer=keras.optimizers.SGD(
        learning_rate=0.01, momentum=0.9, nesterov=True
    ),
    loss="sparse_categorical_crossentropy",
)

Creates an SGD optimizer with momentum and Nesterov acceleration, then uses it to compile a small classifier.

When to use it

Baseline optimization

Train a model with a simple, transparent gradient update.

Momentum-based training

Reduce oscillation and accelerate movement in consistent gradient directions.

Scheduled learning rate

Use a changing step size over the course of training.

Exponential moving average

Maintain smoothed model weights during training.

Interview questions

QHow does Keras SGD update parameters?

With momentum set to zero, SGD subtracts learning_rate multiplied by the gradient from each parameter. With positive momentum, it maintains a velocity, updates that velocity from the previous velocity and current gradient, and adds the velocity to the parameter.

Follow-up: How does Nesterov momentum differ? It changes the parameter update to include the momentum-scaled velocity together with the current gradient term, providing a look-ahead variant of the momentum rule.

QWhat forms can learning_rate take?

learning_rate controls the step size and defaults to 0.01. It may be a floating-point value, a LearningRateSchedule, or a callable returning the current value.

Follow-up: How could the rate change during training? Pass a schedule such as an ExponentialDecay instance as learning_rate.

QWhat do momentum and nesterov control?

The nesterov flag enables Nesterov momentum and defaults to false. It is meaningful as an alternative form of the momentum update, while momentum itself defaults to zero.

Follow-up: What happens with nesterov enabled but zero momentum? The momentum contribution is zero, so the update effectively reduces to the ordinary SGD update.

keras.optimizers.Optimizer
keras.optimizers.schedules.LearningRateSchedule
keras.optimizers.schedules.ExponentialDecay
Optimizer.apply_gradients

Source: SGD - Keras. Checked 2026-10-15.