PyTorch · Class

nn.TransformerEncoder

TransformerEncoder is a stack of TransformerEncoderLayer modules that processes a source sequence in order. It supports optional normalization, attention masks, causal masking, and nested-tensor conversion for highly padded inputs.

How it works

The class applies one TransformerEncoderLayer after another to a source sequence, optionally applying a final normalization module. During the forward pass, it can use an attention mask, padding mask, or a causal interpretation of the supplied mask; nested tensors can be enabled to improve performance when padding is substantial.

Syntax

torch.nn.modules.transformer.TransformerEncoder(encoder_layer, num_layers, norm=None, enable_nested_tensor=True, mask_check=True)

Parameters

Name	Type	Description
`encoder_layer`	`TransformerEncoderLayer`	An instance of TransformerEncoderLayer used as the stack's layer template.
`num_layers`	`int`	Number of sub-encoder layers in the stack.
`norm`	`Module \| None`	Optional layer-normalization component.
`enable_nested_tensor`	`bool`	Whether to automatically convert input to nested tensors and convert output back.
`mask_check`	`bool`	Whether to check masks during operation.
`src`	`Tensor`	Sequence supplied to the encoder.
`mask`	`Tensor \| None`	Optional mask for the source sequence.
`src_key_padding_mask`	`Tensor \| None`	Optional per-batch mask for source keys.
`is_causal`	`bool \| None`	Optional indication that the mask is causal.

Returns

The encoded output produced after the source passes through the encoder layers; the page does not provide a more specific return-type description.

Key points

It composes a configurable number of encoder blocks into one module.
num_layers determines how many encoder-layer instances participate in the stack.
norm optionally provides a normalization module for the encoder.
Nested-tensor conversion is intended to improve performance for inputs with substantial padding.
Forward processing is sequential across the configured encoder layers.

Common mistakes

The supplied encoder layer is replicated across the stack, so explicitly initialize the resulting layers when independent initialization is required.
Use is_causal only when the supplied mask should be interpreted as causal; it changes how mask handling is performed.

Example: Apply an encoder stack

import torch
from torch import nn

layer = nn.TransformerEncoderLayer(d_model=8, nhead=2, batch_first=True)
encoder = nn.TransformerEncoder(layer, num_layers=2)
src = torch.rand(3, 5, 8)
out = encoder(src)
print(out.shape)

Creates a two-layer encoder and processes three batches of five 8-dimensional vectors.

When to use it

Sequence encoding

Build a vanilla Transformer encoder for sequence representations.

Padding-aware batching

Process batches whose sequences contain substantial padding.

Causal attention

Apply autoregressive attention restrictions through a causal mask.

Interview questions

QWhat does TransformerEncoder do, and what are its core constructor arguments?

TransformerEncoder is a stack of N encoder layers. The constructor takes one TransformerEncoderLayer and the number of layers, while forward passes the source through those layers in sequence.

Follow-up: How can padding affect performance? Enable nested tensors, which automatically convert padded input and convert the result back; this is intended to help when padding is high.

QWhat is the purpose of norm and enable_nested_tensor?

The optional norm argument supplies a layer-normalization component applied by the encoder stack. If it is omitted, no explicit final normalization module is supplied through that argument.

Follow-up: What does enable_nested_tensor control? When true, input is automatically converted to a nested tensor and converted back on output.

QWhich inputs can control attention masking?

forward accepts src plus optional mask, src_key_padding_mask, and is_causal arguments. The first two provide sequence and per-batch key padding masks, while is_causal can apply a causal mask as mask.

Follow-up: When should mask_check matter? It is a constructor option for checking masks, so retain the default unless a deliberate compatibility or performance choice justifies changing it.

torch.nn.TransformerEncoderLayer
torch.nn.Transformer
torch.nn.TransformerDecoder

Source: TransformerEncoder — PyTorch 2.11 documentation. Checked 2026-10-16.