Linear algebra
Matrix multiplication and batched matrix multiplication
PyTorch · Function
torch.einsum
Computes tensor contractions using Einstein-summation notation. It aligns dimensions by labels, multiplies corresponding elements, and sums labels omitted from the output.
How it works
torch.einsum interprets letters in an equation as dimension labels for each operand. It multiplies operand elements whose labels align, broadcasts ellipsis-covered dimensions when used, and sums dimensions omitted from the output; an explicit arrow selects the output labels.
Syntax
torch.einsum(*equation*, **operands*) → TensorParameters
| Name | Type | Description |
|---|---|---|
equation | str | Equation string describing the subscripts for each operand and, optionally after ->, the output subscripts. |
operands | Tensor | Input tensors participating in the Einstein summation. |
Returns
Tensor containing the computed Einstein summation.
Key points
- It expresses tensor contractions by multiplying aligned elements and summing selected dimensions.
- Each operand's labels appear in dimension order inside its comma-separated equation segment.
- Dimensions not named in the output are reduced after elementwise products are formed.
- The notation
ij,jk->ikperforms ordinary matrix multiplication. - Use ellipsis when leading or otherwise covered dimensions should broadcast across operands.
- The documented sublist format offers an alternative to the string equation and supports Python's
Ellipsis.
Common mistakes
- Repeated labels align dimensions, so their sizes must satisfy broadcasting rules.
- In implicit-output form, labels appearing once determine the output and are alphabetically ordered.
Example: Matrix multiplication
import torch
A = torch.tensor([[1., 2., 3.], [4., 5., 6.]])
B = torch.tensor([[7., 8.], [9., 10.], [11., 12.]])
C = torch.einsum("ij,jk->ik", A, B)
print(C)Multiplies a 2×3 matrix by a 3×2 matrix, producing a 2×2 result.
When to use it
Multidimensional reductions
Tensor contractions with explicit reduction dimensions
Batch-aware tensor algebra
Broadcasted operations over leading dimensions
Interview questions
QHow does torch.einsum compute its result?
The equation assigns labels to operand dimensions. Einsum multiplies elements after aligning equal labels, then reduces labels absent from the output; for example, ij,jk->ik expresses matrix multiplication.
Follow-up: How would batch matrix multiplication be expressed? Use ellipsis for batch dimensions, such as ...ij,...jk->...ik, provided the ellipsis-covered shapes broadcast.
QWhat is the difference between implicit and explicit output notation?
An explicit -> names output labels and controls their order. If omitted, labels occurring once are retained and sorted alphabetically, while repeated labels are reduced.
Follow-up: What does an ellipsis contribute in the output? It represents broadcast dimensions and must be included in the output if those dimensions should remain rather than be summed out.
QWhat inputs does torch.einsum accept?
The equation is a string whose comma-separated label groups correspond to operands, and operands are tensors supplied after it. Labels are letters for dimensions; ellipsis can stand for one or more broadcast dimensions.
Follow-up: What alternative equation form is documented? A sublist format can pass integer label lists and Python's Ellipsis object instead of a string equation.
Related
torch.backends.opt_einsumtorch.backends.opt_einsum.enabledtorch.backends.opt_einsum.strategy