Join feature dimensions
Combine feature blocks that share batch and other dimensions while extending a feature axis.
PyTorch · Function
torch.cat
Concatenates a sequence of tensors along a specified existing dimension. Inputs must match in every other dimension, with a documented exception for a 1-D empty tensor.
How it works
torch.cat takes a non-empty sequence of tensors and joins them along the selected dimension. Every non-empty input must match in all other dimensions; the output has the same rank and combines the sizes along the concatenation dimension.
Syntax
torch.cat(*tensors*, *dim=0*, ***, *out=None*) → TensorParameters
| Name | Type | Description |
|---|---|---|
tensors | sequence of Tensors | Non-empty tensors must have the same shape except in the concatenation dimension. |
dim | int, optional | Dimension along which the tensors are concatenated; defaults to 0. |
out | Tensor, optional | Optional tensor used as the output. |
Returns
Tensor
Key points
- The function joins multiple tensors along one chosen axis and returns the combined tensor.
- Splitting operations can conceptually undo a compatible concatenation.
- dim identifies the existing axis along which tensor sizes are combined.
- An optional out tensor can receive the operation's result.
- The first argument is a sequence of tensors rather than a single tensor.
Common mistakes
- Inputs with mismatched sizes outside the selected dimension are invalid.
- A one-dimensional empty tensor is the documented exception to the shape requirement.
Example: Concatenate rows
import torch
x = torch.tensor([[1, 2], [3, 4]])
y = torch.tensor([[5, 6]])
result = torch.cat((x, y), dim=0)
print(result)
Concatenates two tensors along their first dimension, producing a 3 × 2 tensor.
When to use it
Merge batches or rows
Append batches or rows when tensors have compatible non-concatenated shapes.
Undo compatible splits
Reassemble tensors after partitioning them with a split operation.
Interview questions
QWhat does torch.cat do?
It joins a sequence of tensors along an existing dimension and returns one tensor. The input tensors must agree in shape except along the dimension being joined.
Follow-up: How does it differ from torch.stack()? torch.cat() preserves the rank and joins an existing dimension, whereas stacking introduces a new dimension.
QWhat is the default dim and what does dim control?
The default dimension is 0. Passing dim selects the axis whose sizes are added, while all other dimensions must match.
Follow-up: What happens if dim is out of range? The operation cannot select a valid concatenation axis and raises an error.
QWhat are the inputs and return value?
The tensors argument is a non-empty sequence of tensors, and out optionally specifies the output tensor. The operation returns the concatenated Tensor.
Follow-up: Which related operation can reverse a concatenation conceptually? torch.split() or torch.chunk() can divide the result into pieces.
Related
torch.splittorch.chunktorch.stacktorch.concatenate