PyTorch · Class

nn.Softmax

nn.Softmax normalizes tensor slices along a selected dimension into values from 0 to 1 whose sum is 1. It preserves the input shape.

How it works

Softmax exponentiates values and divides each one by the sum of exponentials in its selected dimension. The chosen dimension determines which slices become normalized distributions, while the output keeps the input shape and contains values from 0 through 1.

Syntax

class torch.nn.Softmax(dim=None)

Parameters

Name	Type	Description
`dim`	`int`	Integer dimension along which Softmax is computed; each slice along this dimension sums to 1.

Returns

A tensor with the same dimensions and shape as the input, with values in the range [0, 1].

Key points

The class accepts an n-dimensional input tensor and performs Softmax normalization.
Each normalized slice produces nonnegative values no greater than 1 whose total is 1.
For an element xi, the normalized value is exp(xi) divided by the sum of exp(xj) over the relevant slice.
The dim parameter selects the axis over which the exponential normalization is performed.
Softmax preserves the input tensor's shape rather than collapsing the normalized dimension.
For sparse inputs, unspecified entries participate as negative infinity.

Common mistakes

Do not place this module directly before NLLLoss; use LogSoftmax instead, because the documentation notes that it is faster and has better numerical properties.

Example: Normalize rows with Softmax

import torch
from torch import nn

m = nn.Softmax(dim=1)
x = torch.randn(2, 3)
y = m(x)
print(y.shape)
print(y.sum(dim=1))

Creates a two-row tensor and normalizes each row across dimension 1; each row of the result sums to 1.

When to use it

Class-score normalization

Convert model scores along a class axis into normalized class probabilities.

Feature-wise distributions

Normalize feature values independently across a chosen tensor axis.

Weight normalization

Produce attention-style weights when values along the selected axis should sum to one.

Interview questions

QWhat does nn.Softmax do?

nn.Softmax applies the Softmax function to an n-dimensional tensor. For each slice along dim, it rescales values to the range [0, 1] and makes their sum equal to 1.

Follow-up: What determines which values compete with one another? The dim argument: each slice along that dimension is normalized independently.

QWhat is the purpose of dim?

dim is the integer dimension along which normalization occurs. Every slice along that dimension sums to 1, so choosing the wrong dimension changes the resulting distributions.

Follow-up: How can the intended dimension be checked? Compare the tensor shape with the axis representing the classes or features that should compete.

QWhat are the input and output shapes?

The module returns a tensor with the same dimensions and shape as its input. Its values lie in the range [0, 1].

Follow-up: Does Softmax reduce the tensor? No; it transforms values along one axis without changing the tensor's shape.

torch.nn.LogSoftmax
torch.nn.functional.softmax
torch.nn.Softmax2d

Source: Softmax — PyTorch 2.13 documentation. Checked 2026-10-16.