Penalize incorrect binary probabilities
Binary classification with probability outputs
PyTorch · Class
nn.BCELoss
BCELoss measures binary cross-entropy between target values and input probabilities. It supports optional weighting and configurable none, mean, or sum reduction.
How it works
BCELoss compares target values with input probabilities using binary cross-entropy. It can apply optional element weights and either return every element's loss, sum the losses, or average them; inputs must already represent probabilities.
Syntax
class torch.nn.modules.loss.BCELoss(weight=None, size_average=None, reduce=None, reduction='mean')Parameters
| Name | Type | Description |
|---|---|---|
weight | Tensor, optional | Optional per-batch-element weighting tensor. |
size_average | bool, optional | Deprecated compatibility argument controlling averaging. |
reduce | bool, optional | Deprecated compatibility argument controlling reduction. |
reduction | str, optional | Specifies how elementwise losses are reduced: 'none', 'mean', or 'sum'. Default: 'mean'. |
Returns
A scalar loss by default, or unreduced/per-element losses when reduction='none'.
Key points
- BCELoss evaluates binary cross-entropy when the model output is already expressed as probabilities.
- Use reduction='none' to retain one loss value for each input-target element.
- The default reduction is mean; the alternative sum aggregates all unreduced losses.
- A documented application is measuring reconstruction error, such as in an autoencoder.
Common mistakes
- The implementation clamps logarithm outputs to avoid infinite loss and nonlinear backward behavior at extreme probabilities.
Example: Compute binary cross-entropy
import torch
from torch import nn
criterion = nn.BCELoss()
probabilities = torch.tensor([0.9, 0.2])
targets = torch.tensor([1.0, 0.0])
loss = criterion(probabilities, targets)
print(loss.item())Computes binary cross-entropy from probability inputs and matching binary targets.
When to use it
Measure reconstruction error
Autoencoder reconstruction
Inspect unreduced element losses
Custom loss analysis
Interview questions
QWhat does BCELoss measure, and what is its default reduction?
BCELoss computes binary cross-entropy between input probabilities and targets. With the default reduction, it returns the mean of the elementwise losses.
Follow-up: How would you handle logits instead of probabilities? Use BCEWithLogitsLoss, which combines sigmoid and binary cross-entropy in a more numerically stable operation.
QHow do the reduction modes differ?
The unreduced loss is calculated per element. Setting reduction to 'sum' adds those values, while 'mean' averages them.
Follow-up: Why might 'none' be useful? It preserves per-element losses so they can be inspected or reduced with custom weighting.
QWhat does the weight argument represent?
The optional weight argument assigns a weight to each batch element's loss. The documentation describes it as a tensor with one weight per batch element.
Follow-up: What shape should the weight have for a batch? It should provide one weight corresponding to each batch element, consistent with the documented batch-size interpretation.
Related
torch.nn.BCEWithLogitsLosstorch.nn.modules.loss._Losstorch.nn.functional.binary_cross_entropy