Activation regularization
Reduce co-adaptation and overfitting in a neural network during training.
Keras · Class
Dropout
Dropout is a regularization layer that randomly zeroes input units during training to help reduce overfitting. Surviving units are rescaled, while inference passes inputs through unchanged.
How it works
Dropout randomly replaces input elements with zero during training according to rate, then scales the surviving elements by the reciprocal of the keep probability. It is active only when the call receives training=True; model.fit supplies that mode automatically, while inference leaves values unchanged.
Syntax
keras.layers.Dropout(rate, noise_shape=None, seed=None, **kwargs)Parameters
| Name | Type | Description |
|---|---|---|
rate | float | Fraction of input units to drop; a floating-point value from 0 to 1. |
noise_shape | 1D integer tensor or tuple | One-dimensional integer mask shape used for the binary dropout mask. |
seed | Python int | Python integer random seed. |
inputs | tensor | Input tensor of any rank. |
training | Python bool | Selects training behavior with dropout or inference behavior without it. |
Returns
The transformed input tensor, with dropout applied in training mode and unchanged values in inference mode.
Key points
- During training, each input unit is independently eligible to be replaced with zero according to rate.
- Non-dropped values are amplified by the inverse keep probability so the overall input scale remains consistent.
- Standard model.fit training enables the layer without requiring an explicit training flag at every call.
- A temporal mask can be shared across all timesteps by using a noise_shape whose timestep dimension is one.
Common mistakes
- Setting trainable=False does not disable dropout; use training=False for inference behavior.
- Calling the layer without training=True outside model.fit can omit dropout, so pass the flag explicitly when manually controlling training.
Example: Training versus inference
import keras
import numpy as np
layer = keras.layers.Dropout(0.2, seed=7)
x = np.ones((2, 4), dtype="float32")
y = layer(x, training=True)
z = layer(x, training=False)
print(y)
print(z) # unchanged input during inferenceThis runnable example applies dropout explicitly in training mode and shows the pass-through behavior in inference mode.
When to use it
Temporal mask sharing
Apply a consistent feature mask over sequence timesteps.
Explicit mode control
Manually control stochastic training and deterministic evaluation in custom calls.
Interview questions
QWhat does the rate parameter control?
The rate is the fraction of input units dropped on each training step. Surviving values are multiplied by 1 / (1 - rate), preserving the aggregate scale of the input.
Follow-up: Why is the scaling applied during training rather than inference? It keeps inference as a direct pass-through while matching the expected activation scale during optimization.
QWhen is Dropout active?
Dropout operates only when the call's training argument is True. model.fit sets this mode automatically; inference calls do not drop values.
Follow-up: What should a custom training loop do? Pass training=True for optimization calls and training=False for evaluation or prediction calls.
QHow does noise_shape change dropout behavior?
noise_shape defines the shape of the binary mask multiplied by the input. For batch, timestep, and feature inputs, (batch_size, 1, features) shares one mask across timesteps.
Follow-up: Why use a shared temporal mask? It applies consistent feature selection across a sequence instead of independently masking every timestep.
Related
keras.layers.SpatialDropout1Dkeras.layers.SpatialDropout2Dkeras.layers.SpatialDropout3D