Multi-class prediction probabilities
A model produces one score per class for each example.
TensorFlow · Function
tf.nn.softmax
Computes softmax activations by converting logits into normalized values along a selected axis. The result has the same shape and type as the input, with each normalized slice summing to one.
How it works
The function exponentiates values along the selected axis and normalizes each value by the sum of those exponentials. It returns a tensor with the same shape and data type as logits, so each independently normalized slice represents a probability distribution.
Syntax
tf.nn.softmax(logits, axis=None, name=None)Parameters
| Name | Type | Description |
|---|---|---|
logits | Tensor | A non-empty Tensor. Supported types are half, float32, and float64. |
axis | int | The dimension along which softmax is computed; defaults to -1, the last dimension. |
name | string | Optional operation name. |
Returns
A Tensor with the same type and shape as logits.
Key points
- Transforms logits into normalized softmax activations.
- A common use is representing predictions over mutually exclusive classes.
- Values along the normalized axis form a distribution whose total is one.
- Conceptually, it computes exp(logits) divided by the axis-wise sum of exp(logits).
- Leaving axis unset applies softmax over the final tensor dimension.
- The output preserves both the input tensor's shape and numeric type.
Common mistakes
- When using softmax cross-entropy, pass raw, unnormalized logits to the loss function instead of applying this function first.
- The operation raises InvalidArgumentError when logits is empty or the chosen axis is outside the tensor's dimensions.
Example: Convert logits to class probabilities
import tensorflow as tf
logits = tf.constant([[-1.0, 0.0, 1.0]])
probabilities = tf.nn.softmax(logits, axis=-1)
print(probabilities)
print(tf.reduce_sum(probabilities, axis=-1))Converts one row of class logits into probabilities and verifies that the class-axis values sum to one.
When to use it
Per-row normalization
A tensor contains a batch dimension followed by a class dimension.
Interpreting logits
A model's final scores need to be inspected as a distribution.
Interview questions
QWhat does tf.nn.softmax compute?
It computes normalized exponentials: each value is exponentiated and divided by the sum of exponentials along the chosen axis. The result has the same shape and type as logits, and the values in each normalized slice sum to one.
Follow-up: How would you normalize class scores in a tensor shaped [batch, classes]?
QWhat is the default axis, and why does it matter?
The default axis is -1, which means the last dimension. For a [batch, classes] tensor, that normally normalizes each row across its class values.
Follow-up: What changes if axis=0 for the same tensor?
QWhat input constraints and errors does the function document?
logits must be a non-empty Tensor with type half, float32, or float64. The selected axis must be valid; otherwise the operation can raise InvalidArgumentError.
Follow-up: Why are logits generally preferred over probabilities as input to a cross-entropy loss?
Related
tf.math.softmaxtf.nn.log_softmaxtf.nn.softmax_cross_entropy_with_logitstf.nn.sparse_softmax_cross_entropy_with_logits