TensorFlow · Class

optimizers.Adam

Adam is a Keras optimizer that adaptively estimates first- and second-order gradient moments to update model variables. Its API includes controls for learning rates, AMSGrad, decay, clipping, moving averages, loss scaling, and gradient accumulation.

How it works

Adam maintains adaptive first- and second-moment estimates of gradients to update model variables. Its constructor also supports learning-rate scheduling, AMSGrad, weight decay, gradient clipping, exponential moving averages, loss scaling, and gradient accumulation.

Syntax

tf.keras.optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=False, weight_decay=None, clipnorm=None, clipvalue=None, global_clipnorm=None, use_ema=False, ema_momentum=0.99, ema_overwrite_frequency=None, loss_scale_factor=None, gradient_accumulation_steps=None, name='adam', **kwargs)

Parameters

Name	Type	Description
`learning_rate`	`float, schedule, callable, or tensor-like`	Learning rate, default 0.001; may be a float, schedule, callable, or other supported learning-rate value.
`beta_1`	`float, tensor, or no-argument callable`	Exponential decay rate for first-moment estimates; default 0.9.
`beta_2`	`float, tensor, or no-argument callable`	Exponential decay rate for second-moment estimates; default 0.999.
`epsilon`	`float`	Small numerical-stability value; default 1e-7.
`amsgrad`	`bool`	Whether to use AMSGrad; default false.
`weight_decay`	`float or None`	Weight-decay coefficient.
`clipnorm`	`float or None`	Per-variable gradient-norm clipping threshold.
`clipvalue`	`float or None`	Per-variable gradient-value clipping threshold.
`global_clipnorm`	`float or None`	Global gradient-norm clipping threshold.
`use_ema`	`bool`	Whether to maintain exponential moving averages of model variables.
`ema_momentum`	`float`	Momentum used for exponential moving averages; default 0.99.
`ema_overwrite_frequency`	`int or None`	Frequency for overwriting variables with their moving averages.

Returns

An Adam optimizer instance.

Key points

tf.keras.optimizers.Adam is the Keras optimizer class for the Adam algorithm.
Adam adapts parameter updates using estimates of gradient means and gradient-square moments.
beta_1 controls exponential decay of the first-moment estimate.
beta_2 controls exponential decay of the second-moment estimate.
Set amsgrad to enable the AMSGrad variant.
The default learning rate is 0.001, unless a different value or schedule is supplied.

Example: Train a Keras model with Adam

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(1,))
])
model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
    loss="mean_squared_error",
)
model.fit([[0.0], [1.0], [2.0]], [0.0, 1.0, 2.0], epochs=2, verbose=0)

Creates a one-layer Keras model, configures Adam with an explicit learning rate, and trains it on a tiny regression dataset.

When to use it

Optimize a compiled model with adaptive updates

Standard supervised Keras training

Apply norm or value clipping

Training with unstable or unusually large gradients

Accumulate gradients across multiple steps

Memory-constrained training

Maintain exponential moving averages

Training that benefits from parameter averaging

Interview questions

QWhat is Adam, and what are its main default hyperparameters?

Adam is a stochastic-gradient optimizer that adapts updates using estimates of the first and second moments of gradients. Its documented defaults are a learning rate of 0.001, beta_1 of 0.9, beta_2 of 0.999, and epsilon of 1e-7.

Follow-up: How can Adam's update behavior be modified without changing optimizers? Set parameters such as beta_1, beta_2, epsilon, amsgrad, or a learning-rate schedule when constructing it.

QHow do you select the AMSGrad variant?

Set amsgrad=True to use the AMSGrad variant. The Adam constructor documents this option as applying the AMSGrad form described in the referenced convergence paper.

Follow-up: What other stabilization control is available? The optimizer accepts epsilon, which is used as a small numerical-stability term in the update computation.

QWhich advanced training controls does this class expose?

Adam can integrate gradient clipping through clipnorm, clipvalue, or global_clipnorm. It also supports weight_decay, exponential moving averages, loss scaling, and gradient accumulation through constructor arguments.

Follow-up: When would accumulation be useful? It can simulate larger effective batches when updates should be delayed across multiple gradient calculations.

tf.keras.optimizers.Optimizer
tf.keras.optimizers.AdamW
tf.keras.optimizers.RMSprop
tf.keras.optimizers.SGD
tf.keras.optimizers.get
tf.keras.optimizers.serialize

Source: tf.keras.optimizers.Adam | TensorFlow v2.16.1. Checked 2026-10-15.