TensorFlow · Class

optimizers.AdamW

AdamW is a Keras optimizer that combines Adam’s adaptive moment estimation with weight decay. It supports configurable learning rates, moment coefficients, clipping, EMA, loss scaling, and gradient accumulation.

How it works

AdamW combines Adam-style adaptive first- and second-moment estimates with decoupled weight decay. Configure its learning rate and regularization behavior when constructing the optimizer, then pass it to model compilation or gradient-based training.

Syntax

tf.keras.optimizers.AdamW(learning_rate=0.001, weight_decay=0.004, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=False, clipnorm=None, clipvalue=None, global_clipnorm=None, use_ema=False, ema_momentum=0.99, ema_overwrite_frequency=None, loss_scale_factor=None, gradient_accumulation_steps=None, name='adamw', **kwargs)

Parameters

Name	Type	Description
`learning_rate`	`float, LearningRateSchedule, or callable`	Learning-rate value, schedule, or callable; default 0.001.
`weight_decay`	`float`	Weight-decay value; default 0.004.
`beta_1`	`float, constant float tensor, or callable`	Exponential decay rate for first-moment estimates; default 0.9.
`beta_2`	`float, constant float tensor, or callable`	Exponential decay rate for second-moment estimates; default 0.999.
`epsilon`	`float`	Small numerical-stability constant; default 1e-7.
`amsgrad`	`bool`	Whether to use the AMSGrad variant; default False.
`clipnorm, clipvalue, global_clipnorm, use_ema, ema_momentum, ema_overwrite_frequency, loss_scale_factor, gradient_accumu`	`various`	Gradient clipping and optimizer behavior options documented by the constructor.

Returns

An AdamW optimizer instance.

Key points

AdamW is TensorFlow’s Keras optimizer implementation of the AdamW algorithm.
Its adaptive update tracks exponentially smoothed first- and second-order gradient moments.
The algorithm adds weight decay to the Adam-style optimization process.
learning_rate accepts a scalar, a learning-rate schedule, or a no-argument callable.
beta_1 controls decay of the first-moment estimates.
beta_2 controls decay of the second-moment estimates.

Common mistakes

If no learning rate is supplied, the optimizer uses 0.001.
Setting amsgrad enables the AMSGrad variant rather than the default AdamW update.

Example: Compile a model with AdamW

import tensorflow as tf

model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(4,))
])
model.compile(
    optimizer=tf.keras.optimizers.AdamW(
        learning_rate=1e-3,
        weight_decay=4e-3
    ),
    loss="mse"
)
model.fit(tf.random.normal((8, 4)), tf.random.normal((8, 1)), epochs=1)

Creates an AdamW optimizer with explicit learning-rate and weight-decay settings, then uses it to compile and train a small Keras model.

When to use it

Model compilation

Training Keras models with adaptive optimization and explicit regularization.

Scheduled optimization

Changing the learning rate as training progresses.

Clipping and accumulation

Controlling update magnitudes or optimizer state behavior.

Interview questions

QWhat distinguishes AdamW from Adam?

AdamW is an Adam-based stochastic-gradient optimizer that estimates first- and second-order moments while separately decaying weights. This separates weight decay from the adaptive gradient update.

Follow-up: How can the learning rate vary during training? Pass a LearningRateSchedule or a no-argument callable instead of a fixed scalar.

QWhat are the important AdamW constructor options?

The constructor accepts a learning rate, weight decay, moment coefficients, epsilon, AMSGrad selection, clipping controls, EMA options, loss scaling, gradient accumulation, and a name. The documented defaults include learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-7, and amsgrad=False.

Follow-up: Which parameter controls second-moment smoothing? beta_2 controls the exponential decay rate for second-moment estimates.

QHow would you use AdamW in a Keras training workflow?

Use AdamW as the optimizer supplied to model.compile, then train with fit, or use it in a custom gradient loop. Its learning rate may be a scalar, schedule, or callable, allowing the update scale to change over time.

Follow-up: What additional optimizer behavior can be enabled? EMA, gradient clipping, loss scaling, and gradient accumulation are available through the constructor options.

tf.keras.optimizers.Adam
tf.keras.optimizers.Optimizer
tf.keras.optimizers.schedules.LearningRateSchedule

Source: tf.keras.optimizers.AdamW. Checked 2026-10-15.