Outlier-resistant regression
Regression with occasional large target errors.
PyTorch · Class
nn.SmoothL1Loss
SmoothL1Loss computes a robust regression loss that is quadratic for small errors and L1 for larger errors. Its reduction defaults to the mean, and beta controls the transition threshold.
How it works
SmoothL1Loss compares input and target element by element. It uses a quadratic penalty when the absolute error is below beta and an L1 penalty otherwise, then applies the selected reduction to produce the output.
Syntax
class torch.nn.SmoothL1Loss(size_average=None, reduce=None, reduction='mean', beta=1.0)Parameters
| Name | Type | Description |
|---|---|---|
size_average | bool, optional | Optional legacy argument controlling averaging behavior; use reduction instead. |
reduce | bool, optional | Optional legacy argument controlling reduction behavior; use reduction instead. |
reduction | str, optional | How to aggregate the element-wise losses: 'none', 'mean', or 'sum'. Default: 'mean'. |
beta | float, optional | Threshold at which the loss switches between its quadratic and L1 regions. Default: 1.0; must be non-negative. |
Returns
A Tensor containing the unreduced loss or the reduced result, depending on reduction.
Key points
- The criterion uses a quadratic penalty for errors smaller than beta and an L1 penalty for errors at or above that threshold.
- Compared with MSELoss, the loss is less affected by unusually large errors.
- The default reduction produces the average across loss elements.
- Use reduction='sum' to aggregate all element losses by addition.
- Calling the module with input and target performs its forward computation.
Common mistakes
- beta cannot be negative.
- With reduction='none', the result keeps one loss value per element instead of becoming a scalar.
Example: Compute Smooth L1 loss
import torch
from torch import nn
loss_fn = nn.SmoothL1Loss(beta=0.5, reduction="mean")
input = torch.tensor([[1.0, 2.0]], requires_grad=True)
target = torch.tensor([[1.2, 4.0]])
loss = loss_fn(input, target)
loss.backward()
print(loss.item())Computes a scalar Smooth L1 loss and propagates gradients through the input.
When to use it
Object-detection regression
Bounding-box or coordinate prediction.
Controlled gradient behavior
Training models where very large residuals can destabilize updates.
Interview questions
QHow does SmoothL1Loss differ from MSELoss?
It uses a squared term for element-wise errors whose absolute value is below beta, and an L1 term for larger errors. This makes it less sensitive to outliers than MSELoss.
Follow-up: How would you preserve every element's loss? Set reduction='none' so no reduction is applied.
QWhat does the reduction argument control?
reduction controls aggregation: 'none' returns unreduced losses, 'mean' averages them, and 'sum' adds them. The default is 'mean'.
Follow-up: What output does reduction='sum' produce? A single value equal to the sum of all output loss elements.
QWhat is beta used for?
beta is the threshold where the loss changes from its quadratic region to its L1 region, and its default is 1.0. It must be non-negative.
Follow-up: What happens when beta is changed? It moves the boundary between the two regions, changing which errors receive the quadratic treatment.
Related
torch.nn.functional.smooth_l1_losstorch.nn.HuberLosstorch.nn.MSELoss