Keras · Class

layers.LayerNormalization

LayerNormalization normalizes each example independently across selected axes, rather than using batch-wide statistics. Optional trainable gamma scaling and beta centering refine the normalized output.

How it works

For every example, the layer computes statistics over the axes selected by axis, then divides by the stabilized standard deviation and applies optional trainable scaling and offsetting. With the default final-axis setting, each feature vector is normalized independently of the other examples in the batch.

Syntax

keras.layers.LayerNormalization(axis=-1, epsilon=0.001, center=True, scale=True, beta_initializer="zeros", gamma_initializer="ones", beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None, **kwargs)

Parameters

Name	Type	Description
`axis`	`Integer or List/Tuple; default -1`	Axes over which normalization is computed; typically the feature axes. The default -1 means the final input dimension.
`epsilon`	`Float; default 0.001`	Small value used for numerical stability in normalization.
`center`	`Boolean; default True`	Whether to add the trainable beta offset; default true.
`scale`	`Boolean; default True`	Whether to multiply by the trainable gamma factor; default true.
`beta_initializer`	`Initializer; default "zeros"`	Initializer for beta; default zeros.
`gamma_initializer`	`Initializer; default "ones"`	Initializer for gamma; default ones.
`beta_regularizer`	`Regularizer; default None`	Optional regularizer for beta.
`gamma_regularizer`	`Regularizer; default None`	Optional regularizer for gamma.
`beta_constraint`	`Constraint; default None`	Optional constraint for beta.
`gamma_constraint`	`Constraint; default None`	Optional constraint for gamma.
`**kwargs`	`Keyword arguments`	Additional layer keyword arguments.

Returns

A tensor with the same shape as the input, after layer normalization and any enabled affine transformation.

Key points

The layer normalizes each example separately, so one example's statistics do not depend on other batch members.
Normalization aims to move the mean activation of each example toward zero.
The same transformation aims for an activation standard deviation near one.
The default axis value of -1 refers to the final input dimension.
Trainable gamma and beta are shaped across the dimensions selected for normalization.
center defaults to true, and scale also defaults to true according to their documented behavior.

Common mistakes

axis selects the dimensions whose statistics are computed; choosing axes other than the feature dimensions changes what is normalized.
Setting center=False disables the trainable offset, so beta is not used.
Setting scale=False disables the trainable multiplicative factor, so gamma is not used.

Example: Normalize feature vectors

import keras
import numpy as np

x = np.array([[1., 2., 3.], [4., 5., 6.]])
layer = keras.layers.LayerNormalization()
y = layer(x)
print(y)

Normalizes each example across its final axis and applies the default trainable scale and offset.

When to use it

Stabilize hidden representations between sublayers.

Transformer and attention blocks

Normalize the final feature axis independently for every example.

Feature-vector models

Use per-example statistics without relying on batch-wide estimates.

Variable batch sizes or small batches

Interview questions

QHow does LayerNormalization differ from BatchNormalization?

LayerNormalization computes normalization independently for each example rather than aggregating statistics across the batch. It targets a mean near zero and a standard deviation near one over the configured axes.

Follow-up: How does axis affect the result? It determines the dimensions over which moments are computed; -1 means the input's last dimension.

QWhat do center and scale control?

center controls whether a trainable beta offset is added, while scale controls whether a trainable gamma factor multiplies the normalized result. Both options default to true.

Follow-up: What happens when either option is disabled? The corresponding parameter is ignored, so beta is unused when center is false and gamma is unused when scale is false.

QWhy are epsilon, beta_initializer, and gamma_initializer relevant?

The default epsilon is 0.001 and is used to stabilize the normalization calculation. beta starts from zeros and gamma from ones, making the initial affine transformation an identity.

Follow-up: Why are those initializers useful? They let normalization begin without an additional offset or rescaling before training changes the parameters.

keras.layers.BatchNormalization
keras.layers.Normalization
keras.layers.RMSNormalization

Source: Layer normalization layer - Keras. Checked 2026-10-15.