Convert logits into per-class probabilities
Multiclass model outputs
PyTorch · Function
torch.softmax
torch.softmax converts tensor values into normalized, nonnegative scores along a chosen dimension. Each slice along that dimension sums to 1, and an optional dtype controls the computation type.
How it works
torch.softmax applies the softmax transformation independently to slices along the selected dimension. It exponentiates each value relative to the sum of exponentials in that slice, producing an output with the same shape whose values lie between 0 and 1; dtype can optionally be used for the computation.
Syntax
torch.softmax(input, dim, *, dtype=None)Parameters
| Name | Type | Description |
|---|---|---|
input | Tensor | Input tensor to transform. |
dim | int | Dimension along which softmax is computed. |
dtype | torch.dtype, optional | Optional desired dtype of the returned tensor; the input is cast before computation when specified. |
Returns
Tensor
Key points
- The function accepts an input tensor, a normalization dimension, and an optional dtype, then returns a tensor.
- torch.softmax is an alias for torch.nn.functional.softmax.
- Normalization happens independently for every slice selected by dim.
- Each normalized slice contains values from 0 through 1 and totals 1.
- dtype lets callers request the output tensor's data type.
Common mistakes
- Do not pass softmax output directly to NLLLoss; use torch.log_softmax instead.
- For numerically risky input types, specify dtype to cast before softmax is evaluated.
Example: Convert logits to probabilities
import torch
logits = torch.tensor([[1.0, 2.0, 3.0]])
probabilities = torch.softmax(logits, dim=1)
print(probabilities)
print(probabilities.sum(dim=1))Computes class probabilities across dimension 1; each row sums to 1.
When to use it
Normalize scores along a feature or sequence dimension
Attention or weighting computations
Choose a computation dtype to reduce overflow risk
Mixed-precision or narrow-type inputs
Interview questions
QWhat does the dim argument control?
The dim argument selects the dimension whose slices are normalized. Within each slice, the output values are in [0, 1] and sum to 1.
Follow-up: What should replace softmax before NLLLoss? Use torch.log_softmax, which the documentation notes is faster and has better numerical properties.
QWhat are the shape and value guarantees of the result?
The function returns a Tensor with the same shape as the input. Its values are rescaled into the range [0, 1].
Follow-up: How can you verify normalization? Sum the result along the same dimension passed to softmax; each slice should sum to 1.
QWhy might dtype be supplied?
dtype optionally specifies the returned tensor's data type, and the input is cast to that type before the operation. This option can help prevent data-type overflow.
Follow-up: What happens when dtype is omitted? The documentation gives its default as None, so no explicit dtype conversion is requested.
Related
torch.nn.functional.softmaxtorch.log_softmaxtorch.nn.Softmaxtorch.nn.functional.log_softmax