Compute a classification objective from logits and labels.
Multiclass model training
PyTorch · Class
nn.CrossEntropyLoss
CrossEntropyLoss computes classification cross entropy from unnormalized logits and either class-index or class-probability targets. It supports class weighting, ignored indices, configurable reduction, and label smoothing.
How it works
CrossEntropyLoss compares unnormalized class logits with targets and combines log-softmax with negative log likelihood. It accepts class indices or probability distributions, supports optional class weighting, ignored indices, configurable reduction, and label smoothing.
Syntax
torch.nn.CrossEntropyLoss(weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='mean', label_smoothing=0.0)Parameters
| Name | Type | Description |
|---|---|---|
weight | Tensor | None | Optional 1D tensor assigning a weight to each class. |
size_average | bool | None | Legacy averaging option; superseded by reduction. |
ignore_index | int | Class-index target excluded from the loss. |
reduce | bool | None | Legacy reduction option; superseded by reduction. |
reduction | str | How to aggregate losses: 'none', 'mean', or 'sum'. |
label_smoothing | float | Amount of label smoothing; 0.0 means none. |
Returns
A scalar by default, or per-element losses when reduction='none'.
Key points
- Use the module as the training criterion for multiclass classification.
- Provide one weight per class when different classes should contribute unequally, such as in imbalanced data.
- Use reduction='none' when later code needs a separate loss value for each element.
- The default mean reduction aggregates element losses into a scalar weighted average.
- Use reduction='sum' when the desired aggregate is the total loss rather than an average.
- label_smoothing controls smoothing of the target distribution; zero disables smoothing.
Common mistakes
- Pass raw logits rather than probabilities or softmax outputs; the criterion applies the required normalization internally.
- Probability targets must contain values in the unit interval; invalid distributions can produce misleading loss values.
Example: Class-index classification loss
import torch
from torch import nn
loss_fn = nn.CrossEntropyLoss(ignore_index=-100, label_smoothing=0.1)
logits = torch.tensor([[2.0, 0.5, -1.0], [0.2, 1.4, 0.1]])
target = torch.tensor([0, 1])
loss = loss_fn(logits, target)
print(loss.item())Creates a cross-entropy criterion with ignored-label handling and label smoothing, then evaluates it on two samples.
When to use it
Weight classes differently with the weight tensor.
Imbalanced datasets
Calculate losses over spatial or other extra dimensions supported by the input shape.
Dense prediction
Train with target probability distributions instead of only hard class indices.
Soft-label training
Interview questions
QWhat does CrossEntropyLoss compute, and what should the model output?
It computes cross entropy from input logits and targets for classification with C classes. The input is expected to contain unnormalized logits, so a separate softmax is unnecessary.
Follow-up: How can it handle soft labels? Use a target tensor with the same shape as the input containing class probabilities, with values between 0 and 1.
QWhat target formats does the criterion support?
The target can contain class indices or class probabilities. Index targets use one class label per loss element; probability targets have the same shape as the logits.
Follow-up: What validation is needed for probability targets? Ensure their values are in [0,1] and that they represent valid class distributions.
QHow do reduction and ignore_index affect the result?
reduction='none' returns individual losses, 'sum' adds them, and 'mean' computes the documented averaged reduction. ignore_index excludes a specified class-index target from the loss.
Follow-up: When is ignore_index unsuitable? It applies to class-index targets, not to arbitrary entries of a probability target distribution.
Related
torch.nn.functional.cross_entropytorch.nn.NLLLosstorch.nn.LogSoftmax