PyTorch · Module

tensor.backward

Computes gradients of a tensor through its autograd graph and accumulates them in leaf tensors. Non-scalar outputs require a matching gradient tensor.

How it works

Calling backward differentiates the tensor through its autograd graph using the chain rule. Gradients are accumulated in leaf tensors’ .grad fields; a non-scalar output requires a same-shaped gradient tensor, while create_graph can retain derivative tracking for higher-order derivatives.

Syntax

Tensor.backward(*gradient=None*, *retain_graph=None*, *create_graph=False*, *inputs=None*)

Parameters

Name	Type	Description
`gradient`	`Tensor, optional`	Gradient of the differentiated function with respect to self; omit for a scalar tensor.
`retain_graph`	`bool, optional`	Whether to preserve the graph used for gradient computation. If omitted, it is inferred from create_graph.
`create_graph`	`bool, optional`	Whether to construct a graph for the derivative, enabling higher-order derivative products.
`inputs`	`Sequence[Tensor] or dict[str, Tensor], optional`	Sequence of inputs whose .grad fields receive accumulated gradients; other tensors are ignored.

Returns

None

Key points

The method computes derivatives of the current tensor with respect to graph leaves.
Autograd propagates derivatives through the recorded computation graph using the chain rule.
For non-scalar outputs, gradient must match the differentiated tensor’s type and shape.
create_graph=True records derivative operations so higher-order derivative products can be computed.

Common mistakes

Repeated backward calls add to existing leaf gradients, so clear or reset .grad before a new independent optimization step.
A multi-element tensor cannot call backward without supplying a matching gradient argument.
The computation graph is normally released after backward, so another traversal requires retaining it.

Example: Differentiate a scalar tensor

import torch

x = torch.tensor(2.0, requires_grad=True)
y = x**3
y.backward()
print(x.grad)  # tensor(12.)

A scalar output can start backpropagation without an explicit gradient; the derivative is accumulated in x.grad.

When to use it

Backpropagate a scalar loss

Training a neural network

Provide output sensitivities

Vector-valued objectives

Build higher-order derivative graphs

Meta-learning or curvature calculations

Interview questions

QWhat does Tensor.backward do?

It applies the chain rule to the autograd graph and accumulates derivatives in graph leaves. For scalar outputs, no gradient argument is needed.

Follow-up: What changes for a vector output? Supply a gradient tensor matching the output’s type and shape; this represents the gradient with respect to the output.

QWhen is the gradient argument required?

The gradient argument supplies the output-side derivative used to propagate gradients. It may be omitted when the differentiated tensor is scalar.

Follow-up: Why must its shape match? Each output element needs a corresponding weighting for the vector-Jacobian product.

QHow do create_graph and retain_graph differ?

create_graph=True constructs a graph for the derivative calculation, enabling higher-order derivative products. retain_graph controls whether the original graph remains available after differentiation.

Follow-up: What is the default retain_graph behavior? When omitted, the value is inferred from create_graph.

torch.autograd.grad
torch.Tensor.requires_grad
torch.Tensor.grad

Source: torch.Tensor.backward. Checked 2026-10-16.