Keras · Class

optimizers.AdamW

AdamW is Keras's adaptive optimizer with decoupled weight decay. It extends Adam-style first- and second-moment estimation with configurable clipping, EMA, loss scaling, and gradient accumulation.

How it works

AdamW updates parameters using adaptive first- and second-moment estimates, like Adam, while applying a separate weight-decay method. Its configuration also supports gradient clipping, exponential moving averages, loss scaling, and gradient accumulation.

Syntax

keras.optimizers.AdamW(learning_rate=0.001, weight_decay=0.004, beta_1=0.9, beta_2=0.999, epsilon=1e-07, amsgrad=False, clipnorm=None, clipvalue=None, global_clipnorm=None, use_ema=False, ema_momentum=0.99, ema_overwrite_frequency=None, loss_scale_factor=None, gradient_accumulation_steps=None, name=None, **kwargs)

Parameters

Name	Type	Description
`learning_rate`	`float, LearningRateSchedule, or callable`	Learning rate, supplied as a float, LearningRateSchedule, or no-argument callable; defaults to 0.001.
`weight_decay`	`float`	Weight-decay value used by the optimizer; defaults to 0.004.
`beta_1`	`float, constant float tensor, or callable`	Exponential decay rate for first-moment estimates; defaults to 0.9.
`beta_2`	`float, constant float tensor, or callable`	Exponential decay rate for second-moment estimates; defaults to 0.999.
`epsilon`	`float`	Small value used for numerical stability; defaults to 1e-07.
`amsgrad`	`boolean`	Whether to use AMSGrad; defaults to False.
`clipnorm`	`float`	Per-gradient norm clipping threshold.
`clipvalue`	`float`	Per-gradient value clipping threshold.
`global_clipnorm`	`float`	Global gradient norm clipping threshold.
`use_ema`	`boolean`	Whether to apply exponential moving averages to weights.
`ema_momentum`	`float`	Momentum for the weight exponential moving average; defaults to 0.99.
`ema_overwrite_frequency`	`integer`	Frequency for overwriting model weights with their moving average.

Returns

An AdamW optimizer instance.

Key points

AdamW is Keras's optimizer implementation for the AdamW algorithm.
Its adaptive update tracks first- and second-order moment estimates.
Weight decay is an additional part of the optimization method.
beta_1 sets the decay rate used for first-moment estimates and defaults to 0.9.
beta_2 sets the decay rate used for second-moment estimates and defaults to 0.999.
The amsgrad flag enables the AMSGrad variant and defaults to false.

Common mistakes

If no learning rate is supplied, AdamW uses 0.001.
Setting amsgrad=True selects the AMSGrad variant rather than the default AdamW update.

Example: Compile a model with AdamW

import keras

model = keras.Sequential([
    keras.layers.Dense(10, activation="softmax", input_shape=(4,))
])
optimizer = keras.optimizers.AdamW(
    learning_rate=0.001,
    weight_decay=0.004,
)
model.compile(
    optimizer=optimizer,
    loss="sparse_categorical_crossentropy",
)

Creates AdamW with its documented learning-rate and weight-decay settings, then uses it when compiling a Keras model.

When to use it

Adaptive training with weight decay

Training neural networks with adaptive optimization and explicit regularization.

Gradient clipping

Constraining unstable or unusually large gradients during optimization.

Exponential moving averages

Maintaining a smoothed version of model weights during training.

Loss scaling

Using reduced-precision training while controlling gradient magnitude.

Interview questions

QWhat distinguishes AdamW from Adam?

AdamW is an optimizer based on adaptive estimates of first- and second-order moments. It additionally decays weights using the decoupled weight-decay approach described in the cited paper.

Follow-up: How would you make the learning rate dynamic? Pass a LearningRateSchedule or a no-argument callable as learning_rate.

QWhat do beta_1 and beta_2 control?

beta_1 controls exponential decay for first-moment estimates, while beta_2 controls exponential decay for second-moment estimates. Their documented defaults are 0.9 and 0.999.

Follow-up: Which parameter changes the optimizer's second-moment smoothing? beta_2.

QWhat gradient-clipping controls does AdamW expose?

AdamW supports clipping by norm, clipping by value, or clipping the global norm through clipnorm, clipvalue, and global_clipnorm. These options limit gradients before the update is applied.

Follow-up: How could you clip the complete gradient set rather than each gradient independently? Use global_clipnorm.

keras.optimizers.Adam
keras.optimizers.SGD
keras.optimizers.Optimizer
keras.optimizers.schedules.LearningRateSchedule

Source: AdamW. Checked 2026-10-15.