PyTorch · Class

nn.ReLU

nn.ReLU applies the rectified linear unit element-wise, replacing negative values with zero and retaining nonnegative values. It supports inputs of any dimensionality and returns the same shape.

How it works

ReLU processes every element independently using (x) = max(0, x). It preserves positive values, replaces negative values with zero, and returns a tensor with the same shape as its input. The optional inplace setting allows the operation to modify the input storage.

Syntax

torch.nn.ReLU(inplace=False)

Parameters

Name	Type	Description
`inplace`	`bool`	Whether the operation may be performed in-place; defaults to False.

Returns

A tensor with the same shape as the input.

Key points

Each tensor element is transformed independently by ReLU.
The mathematical rule is max(0, x): negative values become zero, while nonnegative values pass through.
Inputs may have any number of dimensions.
The output preserves the input shape.
Calling the module performs its forward computation on the supplied input.

Common mistakes

Setting inplace=True may modify the input tensor directly, so use it only when that mutation is compatible with the surrounding computation.

Example: Apply ReLU to a tensor

import torch
from torch import nn

relu = nn.ReLU()
x = torch.tensor([-2.0, 0.0, 3.0])
y = relu(x)
print(y)  # tensor([0., 0., 3.])

Creates a ReLU module and applies it element-wise to a tensor; negative values become zero while nonnegative values remain unchanged.

When to use it

Neural-network activations

Add nonlinear transformations between linear or convolutional layers.

Composable model blocks

Build activation sequences with torch.nn.Sequential.

CReLU-style features

Construct CReLU by applying ReLU to x and -x before concatenation.

Interview questions

QWhat computation does nn.ReLU perform?

ReLU applies the rectified linear function element-wise: max(0, x). It maps negative inputs to zero and leaves nonnegative inputs unchanged.

Follow-up: What shape does ReLU return? It returns the same shape as its input, regardless of the number of input dimensions.

QWhat does the inplace parameter control?

The inplace argument controls whether the operation can be performed in-place. Its default is False, so the module does not request in-place processing unless explicitly configured.

Follow-up: Why might inplace=True require care? Because modifying the input can conflict with later uses of that tensor or with autograd's saved values.

QWhat input and output shapes does ReLU support?

ReLU accepts inputs with any number of dimensions. Its output has the same shape as the input, because the transformation is applied independently to each element.

Follow-up: How could ReLU be used in a CReLU construction? Apply it to both x and -x, then concatenate the two resulting tensors.

torch.nn.ReLU6
torch.nn.LeakyReLU
torch.nn.PReLU
torch.nn.RReLU

Source: ReLU — PyTorch 2.11 documentation. Checked 2026-10-16.