Global reduction
Aggregate every value in a tensor into one scalar-like tensor.
PyTorch · Function
torch.sum
Returns the sum of all elements in a tensor, or reduces selected dimension(s). Use keepdim to retain reduced axes and dtype to control the accumulation and result type.
How it works
torch.sum reduces tensor values by addition. Without dim it produces one tensor containing the total; with dim it reduces the specified dimension or dimensions, optionally preserving reduced axes with keepdim, and dtype can control the computation and result type.
Syntax
torch.sum(input, *, dtype=None) → Tensor
torch.sum(input, dim, keepdim=False, *, dtype=None) → TensorParameters
| Name | Type | Description |
|---|---|---|
input | Tensor | Tensor to reduce. |
dim | int or tuple/list of ints, optional | Dimension or dimensions over which to sum. A list reduces all listed dimensions. |
keepdim | bool, optional | Whether reduced dimensions remain as size-one axes. |
dtype | torch.dtype, optional | Desired data type for the returned tensor; the input is cast before reduction. |
Returns
A Tensor containing the total sum or the sums along the selected dimension(s).
Key points
- Calling torch.sum(input) reduces the entire tensor to one result.
- Supplying dim performs a reduction along that axis rather than across the whole tensor.
- A list passed to dim lets one call reduce several axes.
- keepdim=True preserves the input rank and changes reduced axes to size one.
- The dtype keyword controls the output data type and the type used during the operation.
Common mistakes
- For integer tensors, choose dtype explicitly when the accumulation type must avoid overflow.
- Do not assume the result always has the input dtype; PyTorch may promote it automatically.
- When keepdim is false, reduced dimensions are removed from the output.
Example: Sum along a dimension
import torch
x = torch.arange(24, dtype=torch.float32).reshape(2, 3, 4)
result = torch.sum(x, dim=1, keepdim=True)
print(result.shape) # torch.Size([2, 1, 4])This sums the second dimension while retaining it as a size-one dimension.
When to use it
Axis reduction
Compute per-feature, per-row, or per-batch totals along selected axes.
Shape-preserving reduction
Keep tensor rank compatible with later broadcasting operations.
Controlled accumulation type
Use a wider or specific type when summing integer data.
Interview questions
QWhat is the difference between torch.sum(x) and torch.sum(x, dim=1)?
With no dim, torch.sum returns the sum of every element in the input tensor. With dim, it reduces only the selected dimension or dimensions.
Follow-up: How can multiple axes be reduced? Pass dim as a list of dimensions, such as [1, 2].
QWhat does keepdim change?
keepdim=True leaves each reduced axis in the result with length one. With the default keepdim=False, those axes are removed.
Follow-up: Why retain singleton dimensions? They preserve rank and make broadcasting with the original shape easier.
QWhen should dtype be supplied?
dtype specifies the desired type of the returned tensor, and the input is cast to that type before summation. This can help prevent overflow and enforce a required result type.
Follow-up: What happens when dtype is omitted? The result type can be automatically promoted, for example from int32 to int64.
Related
torch.Tensor.sumtorch.squeezetorch.cumsum