PyTorch · Module

autograd

torch.autograd provides PyTorch’s automatic differentiation machinery for scalar-valued functions. Mark tensors with requires_grad=True, execute the computation, and use backward or related APIs to obtain gradients.

How it works

Autograd records operations involving tensors that require gradients while executing the forward computation. Calling backward on a scalar result traverses that recorded computation in reverse and accumulates derivatives into the grad fields of eligible leaf tensors.

Syntax

torch.autograd

Returns

The module provides automatic-differentiation classes and functions; individual APIs define their own return values.

Key points

The module supplies the core classes and functions for differentiating scalar-valued computations.
Enable tracking selectively with requires_grad=True on tensors whose derivatives you need.
Supported tensor categories include floating-point and complex types rather than every dtype.

Common mistakes

Autograd does not compute gradients for integer or boolean tensors; use supported floating-point or complex dtypes.

Example: Compute a scalar gradient

import torch

x = torch.tensor(3.0, requires_grad=True)
y = x**2 + 2 * x
 y.backward()
print(x.grad)  # tensor(8.)

Creates a scalar tensor that tracks operations, then computes and reads its derivative.

When to use it

Neural-network optimization

Train models by differentiating a scalar loss with respect to learnable tensors.

Differentiable numerical programs

Compute derivatives for custom numerical objectives involving supported tensor types.

Selective gradient computation

Control which computations participate in gradient tracking by selecting tensors and grad modes.

Interview questions

QWhat is torch.autograd and how is it enabled?

torch.autograd is PyTorch’s automatic-differentiation package. It computes derivatives of scalar-valued functions, requiring tensors whose gradients are needed to be created or marked with requires_grad=True.

Follow-up: How does it obtain derivatives? It records operations during the forward pass and applies the chain rule by traversing the resulting graph backward.

QHow do you trigger backpropagation?

Calling backward() computes derivatives of the output with respect to tracked inputs. For a scalar output, no gradient argument is needed; for a non-scalar output, a matching gradient tensor is required.

Follow-up: What happens to the resulting derivatives? They are accumulated in the grad attributes of leaf tensors that require gradients.

QWhich tensor dtypes support autograd?

Autograd supports floating-point and complex tensor types, including half, float, double, bfloat16, cfloat, and cdouble. Integer tensors are therefore not valid differentiated inputs.

Follow-up: How can gradient tracking be disabled temporarily? Use a no-grad context or decorator when computations should not be recorded.

torch.autograd.Function
torch.autograd.backward
torch.autograd.grad
torch.no_grad
torch.inference_mode
torch.Tensor.requires_grad
torch.Tensor.backward

Source: Automatic differentiation package - torch.autograd. Checked 2026-10-15.