TensorFlow · Class

optimizers.Nadam

Nadam is a Keras optimizer that combines Adam-style adaptive moment estimates with Nesterov momentum. It also supports clipping, weight decay, EMA, loss scaling, and gradient accumulation.

How it works

Nadam updates parameters with Adam's adaptive first- and second-moment estimates while adding Nesterov momentum. Configure its learning rate and moment-decay factors, with optional weight decay, gradient clipping, EMA, loss scaling, and gradient accumulation.

Syntax

tf.keras.optimizers.Nadam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, weight_decay=None, clipnorm=None, clipvalue=None, global_clipnorm=None, use_ema=False, ema_momentum=0.99, ema_overwrite_frequency=None, loss_scale_factor=None, gradient_accumulation_steps=None, name="nadam", **kwargs)

Parameters

Name	Type	Description
`learning_rate`	`float, learning-rate schedule, or callable`	Learning rate; defaults to 0.001.
`beta_1`	`float, constant float tensor, or zero-argument callable`	Exponential decay rate for first-moment estimates; defaults to 0.9.
`beta_2`	`float, constant float tensor, or zero-argument callable`	Exponential decay rate for second-moment estimates; defaults to 0.999.
`epsilon`	`float`	Small value used to avoid division by zero; defaults to 1e-7.
`weight_decay`	`float or None`	Weight-decay coefficient; defaults to None.
`clipnorm`	`float or None`	Maximum norm for each gradient; defaults to None.
`clipvalue`	`float or None`	Maximum absolute value for each gradient; defaults to None.
`global_clipnorm`	`float or None`	Maximum global norm for gradients; defaults to None.
`use_ema`	`bool`	Whether to apply exponential moving averages to model weights; defaults to False.
`ema_momentum`	`float`	EMA momentum; defaults to 0.99 and is used only with EMA enabled.
`ema_overwrite_frequency`	`int or None`	How often to overwrite weights with EMA weights; defaults to None.
`loss_scale_factor`	`float or None`	Loss-scale factor for mixed-precision training; defaults to None.

Returns

A Nadam optimizer instance.

Key points

tf.keras.optimizers.Nadam is the Keras optimizer class for the Nadam algorithm.
Nadam extends Adam-style adaptive optimization with Nesterov momentum.
beta_1 controls decay of the first-moment estimate and defaults to 0.9.
beta_2 controls decay of the second-moment estimate and defaults to 0.999.
Setting use_ema=True enables exponential moving averaging of model weights.
gradient_accumulation_steps averages gradients across multiple steps before applying updates.

Common mistakes

The ema_momentum setting has an effect only when exponential moving averages are enabled.
Gradient accumulation delays variable updates, so optimizer and model weights are not changed on every step when this option is set.

Example: Train a model with Nadam

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(4,))
])
model.compile(optimizer=tf.keras.optimizers.Nadam(learning_rate=1e-3), loss="mse")
model.fit(tf.random.normal((32, 4)), tf.random.normal((32, 1)), epochs=1, verbose=0)

Creates a Nadam optimizer with a custom learning rate and uses it to train a small Keras model.

When to use it

General neural-network optimization

Training Keras models with adaptive first- and second-moment optimization plus Nesterov momentum.

Gradient clipping

Keeping gradient magnitudes within defined bounds during unstable training.

Gradient accumulation

Approximating larger effective batches when memory limits prevent larger batches.

EMA-based weight averaging

Smoothing model weights through exponential moving averages.

Interview questions

QWhat is Nadam, and what are its main defaults?

Nadam is an Adam optimizer variant that uses Nesterov momentum. Its defaults include a learning rate of 0.001, beta_1 of 0.9, beta_2 of 0.999, and epsilon of 1e-7.

Follow-up: How can Nadam reduce gradient instability? Configure clipnorm, clipvalue, or global_clipnorm to constrain gradients before updates.

QHow does Nadam support exponential moving averages?

Set use_ema=True to maintain an exponential moving average of model weights. The optimizer can periodically overwrite the model weights with the moving average, and ema_momentum controls the averaging rate.

Follow-up: When is ema_momentum ignored? It is only relevant when use_ema is true.

QWhat does gradient accumulation do in Nadam?

When gradient_accumulation_steps is an integer, gradients are averaged across that many steps before model and optimizer variables are updated. This is useful when an effective larger batch is desired without storing the full batch.

Follow-up: What trade-off follows? Parameter updates occur less frequently, so optimization behavior differs from updating after every batch.

tf.keras.optimizers.Optimizer
tf.keras.optimizers.Adam
tf.keras.optimizers.Adamax

Source: tf.keras.optimizers.Nadam | TensorFlow v2.16.1. Checked 2026-10-15.