Read values without retaining the training graph
Inference or logging
PyTorch · Module
tensor.detach
Returns a tensor disconnected from the current autograd graph. The result shares storage with the source, does not require gradients, and has no forward-mode AD gradients.
How it works
Calling detach returns a tensor that shares storage with the original but is disconnected from its backward computation graph. The returned tensor does not require gradients and does not retain forward-mode automatic-differentiation gradients, so later operations on it are not connected to the original history.
Syntax
Tensor.detach()Returns
A new Tensor detached from the current graph; it shares storage with the original tensor and does not require gradients.
Key points
- The method produces a new tensor object whose autograd history is disconnected from the current graph.
- The detached result is permanently excluded from gradient requirements.
- Forward-mode automatic-differentiation gradients are absent from the returned tensor.
- The source and detached tensors observe each other’s in-place data changes because their storage is shared.
Common mistakes
- Detachment does not copy data: mutations through either tensor are visible through the other and can cause correctness-check failures.
- Detach removes forward-mode AD gradients as well as ordinary reverse-mode gradient tracking.
Example: Detach a tensor from autograd
import torch
x = torch.tensor([1.0, 2.0], requires_grad=True)
y = (x * 2).sum()
z = y.detach()
print(z.requires_grad) # False
z[0] = 99.0
print(y.item())
Creates a tensor with autograd history, detaches a scalar result, and verifies that the detached tensor does not require gradients. The final mutation illustrates shared storage when the detached value is non-scalar.
When to use it
Use a tensor in later computations without backpropagating through its history
Gradient-isolated branches
Discard autograd history without immediately copying the underlying data
Memory-conscious tensor handling
Interview questions
QWhat does Tensor.detach() do?
detach() returns a new tensor disconnected from the current computation graph. It does not require gradients, and it shares storage with the source tensor.
Follow-up: What happens if the detached tensor is modified in place? The source tensor observes the same storage mutation, which can also trigger correctness-check errors.
QDoes detach() create a copy?
The returned tensor is not connected to the graph used to compute the original tensor, so subsequent operations on it cannot propagate gradients back through that history. It also has no forward-mode AD gradients.
Follow-up: How would you avoid shared-storage side effects? Use a separate copy after detaching, such as detach().clone(), when independent storage is required.
QWhat storage relationship exists between the original and detached tensors?
No. The result shares storage with the original tensor, even though it has independent autograd history. In-place changes made through either tensor are therefore visible through both.
Follow-up: Why might this matter in a correctness check? An in-place update can invalidate values saved for gradient computation and produce an error.
Related
torch.Tensor.detach_torch.Tensor.clonetorch.Tensor.requires_grad_torch.no_grad