Reduce outlier influence
Regression with occasional large residuals
TensorFlow · Class
losses.Huber
tf.keras.losses.Huber computes a robust regression loss between true and predicted values. It uses a quadratic penalty for small errors and a linear penalty for errors beyond delta.
How it works
Huber loss uses a squared-error penalty when the absolute prediction error is at most delta, then switches to a linear penalty for larger errors. The loss is averaged across the final dimension, and the configured reduction determines how batch values are aggregated.
Syntax
tf.keras.losses.Huber(
delta=1.0,
reduction='sum_over_batch_size',
name='huber_loss'
)Parameters
| Name | Type | Description |
|---|---|---|
delta | float | A float marking where the Huber function changes from quadratic to linear; defaults to 1.0. |
reduction | string or None | Reduction mode: "sum", "sum_over_batch_size", or None; defaults to "sum_over_batch_size". |
name | string or None | Optional instance name. |
Returns
A loss value computed from y_true and y_pred; the configured reduction controls how loss values are aggregated.
Key points
- Huber compares true targets with predicted targets and returns a loss suitable for regression.
- delta defines the boundary between the squared-error-like region and the robust linear region.
- After applying the piecewise formula, values are averaged along the final axis.
- Huber is a Loss subclass, so it can be instantiated and supplied where Keras expects a loss object.
- name provides an optional identifier for the created loss object.
Common mistakes
- Choose reduction deliberately: the default averages loss values over the batch, while "sum" accumulates them and None leaves per-example values unreduced.
Example: Evaluate Huber loss
import tensorflow as tf
loss = tf.keras.losses.Huber(delta=1.0)
y_true = tf.constant([[0.0, 1.0], [0.0, 0.0]])
y_pred = tf.constant([[0.6, 0.4], [0.4, 0.6]])
print(loss(y_true, y_pred).numpy())Creates a Huber loss object and evaluates it on two target/prediction batches.
When to use it
Configure a loss object
Keras model training
Control aggregation
Custom training loops
Interview questions
QHow does Huber loss differ from mean squared error?
Huber loss is quadratic for errors whose absolute value does not exceed delta and linear beyond that threshold. This makes large errors grow less aggressively than with squared error.
Follow-up: What does delta control? It sets the transition point: increasing it keeps more errors in the quadratic region, while decreasing it makes the linear region begin earlier.
QWhat is the purpose of delta?
The delta argument is a floating-point threshold where the penalty changes from quadratic to linear. Its default is 1.0.
Follow-up: How would you make the loss less sensitive to moderate outliers? Use a smaller delta so those errors enter the linear regime sooner.
QWhat do reduction and name configure?
The class accepts reduction values of "sum", "sum_over_batch_size", or None, with "sum_over_batch_size" as the default. The name argument optionally sets the loss instance's name.
Follow-up: When might None be useful? It can preserve unreduced loss values for custom weighting or aggregation before the training step combines them.
Related
tf.keras.losses.hubertf.keras.losses.Losstf.keras.Model.compile