Reduce overfitting by randomly masking activations during training.
Regularization
PyTorch · Function
F.dropout randomly zeroes input elements during training with probability p. Use training=False to disable masking, such as during evaluation.
F.dropout independently masks input elements during training according to probability p, using Bernoulli samples. When training is false, it leaves dropout inactive; inplace=True performs the operation on the input tensor itself.
torch.nn.functional.dropout(input, p=0.5, training=True, inplace=False)| Name | Type | Description |
|---|---|---|
input | Tensor | Input tensor to which dropout is applied. |
p | float | Probability of an element being zeroed; default 0.5. |
training | bool | Whether to apply dropout; default True. |
inplace | bool | Whether to perform the operation in place; default False. |
Tensor
import torch
import torch.nn.functional as F
x = torch.ones(2, 4)
y = F.dropout(x, p=0.5, training=True)
print(y.shape) # torch.Size([2, 4])Applies dropout to a tensor while training is enabled.
Regularization
Custom forward passes
Evaluation control
F.dropout randomly sets input elements to zero with probability p during training, using Bernoulli sampling. The function returns a Tensor.
Follow-up: How would you disable it for evaluation? Pass training=False, commonly by forwarding a module’s training state.
p is the probability that an input element is zeroed, and its default is 0.5. training controls whether dropout is applied and defaults to True.
Follow-up: What behavior should a model use at inference time? It should call the function with training=False so elements are not randomly masked.
inplace controls whether the operation writes into the input tensor. It defaults to False; setting it to True performs dropout in place.
Follow-up: When is the default preferable? Use the out-of-place form when the original tensor is needed elsewhere or mutation could complicate gradient computation.
torch.nn.Dropouttorch.nn.functional.alpha_dropouttorch.nn.functional.dropout1dtorch.nn.functional.dropout2dtorch.nn.functional.dropout3d