Training regularization
Dense or convolutional networks that overfit their training data.
Keras · Class
layers.Dropout
Dropout randomly removes input units during training to reduce overfitting, then rescales the units that remain. It does nothing during inference.
How it works
At training time, the layer independently zeros input values according to the chosen rate and rescales the values that remain by the reciprocal of the keep probability. It is inactive during inference, and Keras supplies the training flag automatically when the layer is used through model.fit.
Syntax
keras.layers.Dropout(rate, noise_shape=None, seed=None, **kwargs)Parameters
| Name | Type | Description |
|---|---|---|
rate | float | Fraction of input units to drop; a floating-point value from 0 through 1. |
noise_shape | 1D integer tensor or tuple | Shape of the binary mask multiplied with the input; a one-dimensional integer tensor. |
seed | int | Optional Python integer random seed. |
inputs | tensor | Input tensor of any rank. |
training | bool | Whether the layer operates in training mode and applies dropout. |
Returns
The documentation does not specify a separate return section; the layer produces a tensor with dropout applied during training or the unchanged input during inference.
Key points
- Dropout is a regularization layer that masks input values during training.
- rate specifies the fraction of input units removed on each training step and must be between 0 and 1.
- Nonzero outputs are rescaled by the inverse keep probability so the expected sum remains unchanged.
- Inference leaves the input values intact because dropout is only active in training mode.
- noise_shape lets you define the mask dimensions instead of using a separate mask shape for every input element.
- seed optionally supplies an integer random seed for the layer's stochastic masking.
Common mistakes
- Dropout is inactive unless the call runs in training mode; explicitly pass training=True when invoking the layer outside the usual training flow.
- Setting trainable=False does not disable dropout, because this layer has no variables or weights to freeze.
Example: Shared temporal dropout mask
import keras
from keras import layers
x = keras.Input(shape=(10, 8))
y = layers.Dropout(0.2, noise_shape=(None, 1, 8))(x)
model = keras.Model(x, y)
outputs = model(x, training=True)This model uses one dropout mask across all timesteps during training.
When to use it
Shared temporal masking
Sequence inputs where selected features should be masked consistently at every timestep.
Seeded dropout
Reproducible experiments involving stochastic regularization.
Interview questions
QWhat does Dropout do, and what does rate mean?
Dropout randomly sets a fraction of input units to zero during training, where rate is the fraction removed. The surviving values are multiplied by 1/(1-rate), preserving the total expected activation.
Follow-up: What happens during inference? The layer does not drop values when training is false, so the forward pass uses the inputs unchanged.
QHow does Keras distinguish training from inference for Dropout?
The training argument determines whether dropout is applied. model.fit sets training appropriately, while direct calls can pass training=True explicitly.
Follow-up: Why is trainable=False not an alternative? Dropout has no trainable variables, so that flag does not change its behavior.
QHow would you share a dropout mask across sequence timesteps?
noise_shape controls the shape of the binary mask multiplied with the input. For inputs shaped (batch_size, timesteps, features), (batch_size, 1, features) reuses one mask across timesteps.
Follow-up: When is this useful? It can impose consistent feature masking over a sequence rather than independently masking every timestep.
Related
keras.layers.GaussianDropoutkeras.layers.SpatialDropout1Dkeras.layers.SpatialDropout2Dkeras.layers.SpatialDropout3D