Add a nonlinear activation after an affine transformation.
Hidden layers in a neural network
TensorFlow · Function
tf.nn.relu
Computes the rectified-linear activation, replacing negative tensor elements with zero and leaving nonnegative elements unchanged. The result has the same dtype as the input.
How it works
The function applies the rectified-linear transformation element by element: negative values become zero, while nonnegative values remain unchanged. It returns a tensor with the same data type as the input and optionally accepts an operation name.
Syntax
tf.nn.relu(
features: Annotated[Any, TV_Relu_T], name=None
) -> Annotated[Any, TV_Relu_T]Parameters
| Name | Type | Description |
|---|---|---|
features | Tensor | A Tensor containing the values to which the rectified-linear operation is applied; supported types are listed by the documentation. |
name | string | Optional name for the operation. |
Returns
A Tensor with the same type as features.
Key points
- ReLU maps each input element to the larger of that element and zero.
- The main input is a Tensor containing the values to transform.
- You can optionally assign a graph-operation name through
name. - The result is a Tensor that retains the input dtype.
- For the documented example input of -2, 0, and 3, the output is 0, 0, and 3 with float32 dtype.
Common mistakes
- The input dtype is restricted to TensorFlow's documented supported types; unsupported dtypes are not accepted.
- The output does not convert the input to a new dtype; it preserves the input tensor's type.
Example: Apply ReLU to a tensor
import tensorflow as tf
features = tf.constant([-2.0, 0.0, 3.0])
activated = tf.nn.relu(features)
print(activated.numpy()) # [0. 0. 3.]Creates a floating-point tensor and replaces its negative value with zero while preserving zero and the positive value.
When to use it
Clamp negative tensor values to zero.
Tensor preprocessing
Measure or visualize positive-response patterns in intermediate tensors.
Activation inspection
Interview questions
QWhat does tf.nn.relu compute?
tf.nn.relu computes max(features, 0) element by element. Consequently, negative entries produce zero, while zero and positive entries pass through unchanged.
Follow-up: How does it differ from tf.nn.relu6? relu6 additionally caps the result at 6, whereas relu has no upper bound.
QWhat are the parameters of tf.nn.relu?
The features argument must be a Tensor with one of the dtypes listed in the documentation, including common floating-point and integer types. The optional name argument supplies a name for the operation.
Follow-up: What should be checked when a call fails due to dtype? Compare the tensor's dtype with the documented supported-type list and cast it if appropriate.
QWhat does tf.nn.relu return?
The function returns a Tensor whose type matches features. It applies the transformation independently to tensor elements, so the output keeps the input shape while changing values according to the rectifier.
Follow-up: Why can this be used as a neural-network activation? It introduces a nonlinear mapping that preserves positive responses and suppresses negative ones.
Related
tf.nn.relu6tf.nn.leaky_relutf.nn