Train a model that predicts one binary outcome from each logit.
Binary classification
PyTorch · Class
nn.BCEWithLogitsLoss
BCEWithLogitsLoss combines sigmoid and binary cross-entropy in one numerically stable module. It accepts logits and targets, supports optional weighting, and can return unreduced, averaged, or summed losses.
How it works
BCEWithLogitsLoss evaluates binary cross-entropy directly from logits by combining sigmoid and BCE internally. It uses a numerically stable formulation and can optionally weight elements, adjust positive examples with pos_weight, and reduce the resulting losses by none, mean, or sum.
Syntax
class torch.nn.BCEWithLogitsLoss(weight=None, size_average=None, reduce=None, reduction='mean', pos_weight=None)Parameters
| Name | Type | Description |
|---|---|---|
weight | Tensor, optional | Optional manual rescaling weight for each batch element. |
size_average | bool, optional | Deprecated compatibility argument controlling averaging behavior; use reduction instead. |
reduce | bool, optional | Deprecated compatibility argument controlling reduction behavior; use reduction instead. |
reduction | str, optional | Reduction mode: 'none', 'mean', or 'sum'; default is 'mean'. |
pos_weight | Tensor, optional | Optional weight for positive examples. |
Returns
A scalar loss when reduction is 'mean' or 'sum'; with 'none', an unreduced tensor containing elementwise losses.
Key points
- Use it as a single criterion for binary cross-entropy on logits; it incorporates the sigmoid operation internally.
- Its fused computation is preferable to manually composing sigmoid and BCELoss when numerical robustness matters.
- With reduction set to none, the module produces one loss value per input-target element before any aggregation.
- The default mean reduction divides the summed element losses by the number of output elements.
- Sum reduction returns the total of all elementwise losses.
- The supported reduction modes are none, mean, and sum.
Common mistakes
- Pass raw logits to this loss rather than applying a separate sigmoid layer before it.
- Choose reduction deliberately: the default mean averages the output, while none preserves elementwise values and sum adds them.
- The legacy size_average and reduce arguments are retained for compatibility; prefer the reduction argument in new code.
Example: Binary classification loss
import torch
from torch import nn
loss_fn = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([2.0]))
logits = torch.tensor([[0.0], [2.0]])
targets = torch.tensor([[1.0], [0.0]])
loss = loss_fn(logits, targets)
print(loss.item())Creates the loss with a positive-class weighting factor, evaluates raw model logits against floating-point binary targets, and prints the scalar mean loss.
When to use it
Increase the influence of positive examples with pos_weight.
Imbalanced binary labels
Compute elementwise binary losses for multi-label outputs.
Dense independent labels
Interview questions
QWhy use BCEWithLogitsLoss instead of Sigmoid followed by BCELoss?
It combines sigmoid and binary cross-entropy in one module. The combined implementation uses the log-sum-exp trick, making it more numerically stable than a separate sigmoid followed by BCELoss.
Follow-up: Why should a model supply logits rather than probabilities? Because the module performs the sigmoid internally as part of its stable computation, so pre-sigmoid logits preserve that numerical advantage.
QWhat does the reduction argument do?
The reduction argument controls output aggregation. none returns an unreduced loss for every element, mean averages the output, and sum adds all output values.
Follow-up: When is none useful? It lets code inspect or apply custom per-element aggregation after the loss has been computed.
QHow can class imbalance be addressed with this loss?
pos_weight changes the contribution of positive targets and is useful when positive and negative examples are imbalanced. The constructor also accepts weight for manual rescaling and supports the reduction choice.
Follow-up: How should weighting affect evaluation? Keep the weighting policy consistent with the training objective, but report unweighted metrics separately when measuring real-world class performance.
Related
torch.nn.BCELosstorch.sigmoid