Separate packed features
Divide a feature or embedding dimension into separate projections.
PyTorch · Function
torch.chunk
Splits a tensor along a chosen dimension into a requested number of chunks. Outputs are views and may be fewer than requested when the division is not possible.
How it works
torch.chunk divides a tensor along a selected dimension and returns tuple elements that are views of the original tensor. It aims for the requested number of pieces, distributing elements as evenly as possible, but can return fewer pieces when the requested division cannot be performed.
Syntax
torch.chunk(input: Tensor, chunks: int, dim: int = 0) → Tuple[Tensor, ...]Parameters
| Name | Type | Description |
|---|---|---|
input | Tensor | Tensor to split. |
chunks | int | Number of chunks to return. |
dim | int = 0 | Dimension along which to split the tensor; defaults to 0. |
Returns
Tuple[Tensor, ...]
Key points
- Use torch.chunk to partition one tensor into a requested number of pieces.
- The returned tensors are views rather than independent copies.
- Even divisibility produces equally sized outputs.
- For uneven sizes, the last output is the smaller one.
- Choose torch.tensor_split when the output count must be exact.
Common mistakes
- Do not assume the returned tuple always has exactly chunks elements; use torch.tensor_split when an exact count is required.
- The results share storage with the input, so in-place changes to a chunk can affect the original tensor.
Example: Split a one-dimensional tensor
import torch
x = torch.arange(13)
parts = torch.chunk(x, chunks=6)
print(len(parts))
print([part.tolist() for part in parts])This creates six requested chunks along dimension 0 and prints their count and contents; because 13 cannot be divided evenly into six parts, the final chunk is smaller.
When to use it
Branch tensor processing
Partition a tensor for independent processing across branches.
Batch or sequence partitioning
Split a batch or sequence dimension into manageable pieces.
Interview questions
QWhat does torch.chunk do, and what does it return?
torch.chunk(input, chunks, dim=0) splits input along dim and returns a tuple of tensors. The returned pieces are views, and dim defaults to 0.
Follow-up: How would you guarantee exactly the requested number of outputs? Use torch.tensor_split(), which the documentation identifies as always returning exactly that number.
QHow are chunk sizes determined?
When the selected dimension divides evenly by chunks, all outputs have the same size. Otherwise, the pieces are generally balanced, with the final piece smaller.
Follow-up: What happens if the requested division is not possible? The function may return fewer chunks than requested, so code should not blindly unpack the expected count.
QWhat are the function's parameters?
The chunks argument specifies the requested number of chunks, while dim selects the dimension to split and defaults to zero. The input argument is the tensor being divided.
Follow-up: How can the operation be applied to the last dimension? Pass dim=-1, provided that dimension exists for the input tensor.
Related
torch.tensor_split()torch.Tensor.chunk