TensorFlow · Class

layers.BatchNormalization

BatchNormalization normalizes layer inputs toward zero mean and unit standard deviation. It uses current-batch statistics during training and moving statistics during inference.

How it works

The layer standardizes selected input channels while retaining trainable scale and offset parameters. During training it uses the current batch's statistics and updates moving averages; during inference it uses those accumulated moving statistics instead.

Syntax

tf.keras.layers.BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones', moving_mean_initializer='zeros', moving_variance_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None, synchronized=False, **kwargs)

Parameters

Name	Type	Description
`axis`	`int`	Dimension or dimensions to normalize; defaults to the last dimension.
`momentum`	`float`	Weight given to old moving statistics when updating them.
`epsilon`	`float`	Small value added to variance for numerical stability.
`center`	`bool`	Whether to add the learned offset beta.
`scale`	`bool`	Whether to multiply by the learned scale gamma.
`beta_initializer`	`Initializer`	Initializer for beta.
`gamma_initializer`	`Initializer`	Initializer for gamma.
`moving_mean_initializer`	`Initializer`	Initializer for the moving mean.
`moving_variance_initializer`	`Initializer`	Initializer for the moving variance.
`beta_regularizer`	`Regularizer`	Regularizer for beta.
`gamma_regularizer`	`Regularizer`	Regularizer for gamma.
`beta_constraint`	`Constraint`	Constraint for beta.

Returns

A layer that normalizes its inputs; calling it returns the normalized tensor.

Key points

BatchNormalization is a Keras layer for normalizing activations.
Its transformation targets outputs with mean near zero and standard deviation near one.
Normalization is applied independently for each selected channel or feature position.
Training statistics come from the batch currently being processed.
Inference relies on moving statistics accumulated across training batches.
The learned scale starts at one and can be omitted with scale=False.
The learned offset starts at zero and can be omitted with center=False.

Common mistakes

Always make the training or inference mode explicit when manually calling the layer, because the normalization statistics differ.
A call without an explicit training argument uses inference behavior, so it does not normalize from the current batch.

Example: Training and inference behavior

import tensorflow as tf

layer = tf.keras.layers.BatchNormalization()
x = tf.constant([[1., 2.], [3., 4.]])
training_output = layer(x, training=True)
inference_output = layer(x, training=False)
print(training_output)
print(inference_output)

This runnable example contrasts batch-statistics normalization during training with moving-statistics normalization during inference.

When to use it

Stabilize hidden-layer activations

A convolutional or dense network needs better-conditioned intermediate activations.

Per-channel feature normalization

A model should normalize features separately along a chosen channel dimension.

Separate training and inference behavior

Training and serving need distinct statistic handling.

Interview questions

QHow does BatchNormalization behave in training versus inference?

BatchNormalization keeps outputs near zero mean and unit standard deviation. In training mode it computes statistics from the current batch, whereas inference uses moving averages accumulated from training.

Follow-up: Why can inference differ from training? Inference substitutes stored moving mean and variance for the current batch's statistics, producing deterministic behavior independent of the evaluation batch.

QWhat does the axis argument control?

The default axis is -1, meaning the last input dimension is normalized as the channel or feature axis. The constructor also exposes axis so another dimension can be selected.

Follow-up: What should be checked when using channels-first tensors? Set axis to the dimension containing channels rather than relying on the last-dimension default.

QWhat are gamma, beta, epsilon, center, and scale used for?

The normalized value is scaled by trainable gamma and shifted by trainable beta. scale=False removes gamma, while center=False removes beta; epsilon protects the variance denominator.

Follow-up: When might scale be disabled? If a following linear operation already provides scaling, disabling scale can avoid a redundant learned factor.

tf.keras.layers.LayerNormalization
tf.keras.layers.Normalization
tf.keras.layers.Layer

Source: tf.keras.layers.BatchNormalization | TensorFlow v2.16.1. Checked 2026-10-15.