Regression loss
Training regression models where prediction errors should be penalized quadratically.
PyTorch · Class
nn.MSELoss
torch.nn.MSELoss measures squared differences between corresponding input and target elements. By default, it returns their mean; it can also return unreduced values or their sum.
How it works
MSELoss compares corresponding elements of an input tensor and target tensor by squaring their differences. Its reduction setting then leaves those values unchanged, averages them, or adds them together before returning the loss.
Syntax
class torch.nn.modules.loss.MSELoss(size_average=None, reduce=None, reduction='mean')Parameters
| Name | Type | Description |
|---|---|---|
size_average | bool, optional | Deprecated; controls averaging behavior when reduction is not explicitly used. |
reduce | bool, optional | Deprecated; controls whether reduction occurs and takes precedence over size_average when false. |
reduction | str, optional | Reduction applied to the output: 'none', 'mean', or 'sum'. Default: 'mean'. |
Returns
A loss tensor: unreduced elementwise values for reduction='none', or a scalar for reduction='mean' or 'sum'.
Key points
- The loss squares the difference between corresponding input and target elements.
- Input may have any number of dimensions.
- Target must match the input shape.
- Use reduction='none' to retain one loss value per element.
- The mean mode divides the total squared error by the number of output elements.
- The sum mode returns the total of all elementwise squared errors.
- The default reduction is mean, so the returned loss is averaged.
Common mistakes
- The size_average and reduce arguments are deprecated; prefer the reduction argument.
- The target tensor must have the same shape as the input tensor.
- With multidimensional samples, the default mean is over every loss element, not necessarily one value per sample.
Example: Compute mean squared error
import torch
from torch import nn
loss_fn = nn.MSELoss()
input = torch.tensor([[1.0, 2.0]], requires_grad=True)
target = torch.tensor([[0.0, 4.0]])
loss = loss_fn(input, target)
loss.backward()
print(loss.item())Computes the default mean squared error and backpropagates through the input.
When to use it
Elementwise diagnostics
Inspecting individual errors across pixels, tokens, or other tensor positions.
Summed objectives
Scaling the objective by the number of output elements instead of averaging.
Interview questions
QWhat does torch.nn.MSELoss compute, and what is its default behavior?
MSELoss computes squared differences element by element between input and target. With the default reduction, it divides their sum by the number of output elements.
Follow-up: How would you preserve each squared error? Construct MSELoss with reduction='none', which applies no output reduction.
QWhat reduction modes are available?
The reduction option accepts 'none', 'mean', or 'sum'. 'none' returns unreduced values, 'mean' averages them, and 'sum' adds them.
Follow-up: How does 'sum' differ from 'mean' during optimization? 'sum' makes the loss scale with the number of elements, while 'mean' normalizes by that count.
QWhat shapes and outputs does MSELoss support?
Input and target can have any number of dimensions, but their shapes must match. The module returns a scalar for 'mean' or 'sum', and an elementwise tensor for 'none'.
Follow-up: What happens when reduction='none'? Each input-target position produces its own squared-error value.
Related
torch.nn.L1Losstorch.nn.SmoothL1Losstorch.nn.modules.loss._Loss