TensorFlow · Class

layers.ReLU

A configurable Keras layer implementing rectified linear activation. Adjust its slope, threshold, and optional upper bound to obtain standard, leaky, thresholded, or capped ReLU behavior.

How it works

ReLU applies a piecewise rectified transformation to each input element. Values below the threshold use a configurable negative slope, values between the threshold and maximum pass through, and values at or above the maximum are capped when max_value is set.

Syntax

tf.keras.layers.ReLU(max_value=None, negative_slope=0.0, threshold=0.0, **kwargs)

Parameters

Name	Type	Description
`max_value`	`float or None`	Maximum activation value; None means unlimited.
`negative_slope`	`float`	Coefficient for values below threshold.
`threshold`	`float`	Threshold for the activation.
`**kwargs`	`keyword arguments`	Base layer keyword arguments, such as name and dtype.

Returns

A layer that produces the transformed tensor when called.

Key points

ReLU is a Keras layer that performs a rectified linear activation element by element.
With default settings, the transformation is the standard maximum of the input and zero.
The constructor exposes a negative slope, threshold, optional maximum, and standard layer keyword arguments.
negative_slope determines the coefficient applied to inputs below the threshold.
max_value controls the activation ceiling when a finite value is supplied.
threshold selects the boundary below which the lower-slope behavior is used.

Common mistakes

max_value must be nonnegative; leaving it as None removes the upper cap.
The threshold is constrained to nonnegative values, so it cannot be used directly for a negative cutoff.

Example: Parameterized ReLU layer

import tensorflow as tf

relu = tf.keras.layers.ReLU(
    max_value=10.0,
    negative_slope=0.5,
    threshold=0.0,
)
result = relu(tf.constant([-10.0, -5.0, 0.0, 5.0, 10.0]))
print(result.numpy())

Creates a capped, leaky ReLU layer and applies it to a tensor.

When to use it

Standard ReLU activation

Use the defaults in hidden layers when a simple nonlinearity is needed.

Leaky activation

Set negative_slope above zero to retain a nonzero response for inputs below the threshold.

Capped activation

Provide max_value to limit unusually large positive activations.

Thresholded activation

Raise threshold to create thresholded activation behavior.

Interview questions

QWhat does tf.keras.layers.ReLU do by default?

The default layer computes standard ReLU: negative inputs become zero and nonnegative inputs pass through. Its default configuration is max_value=None, negative_slope=0.0, and threshold=0.0.

Follow-up: How would you produce a leaky and capped variant? Set negative_slope to a positive value and max_value to the desired upper bound.

QWhat is the purpose of negative_slope and threshold?

negative_slope controls the linear slope below threshold. With the default threshold of zero, negative inputs are multiplied by negative_slope rather than necessarily being discarded.

Follow-up: What happens if threshold is positive? Inputs below that threshold use the below-threshold branch, while inputs at or above it follow the pass-through or capped branches.

QHow does max_value change ReLU?

max_value specifies the largest activation returned; None means there is no upper limit. The layer therefore supports ordinary, leaky, thresholded, and clipped ReLU behavior through its constructor arguments.

Follow-up: What shape and dtype does the call preserve? The output follows the input tensor's elementwise shape and dtype through the layer operation.

tf.keras.activations.relu
tf.nn.relu
tf.keras.layers.LeakyReLU
tf.keras.layers.Layer

Source: tf.keras.layers.ReLU. Checked 2026-10-15.