Transform each timestep's features into recurrent hidden representations.
Sequence modeling
PyTorch · Class
nn.RNN
nn.RNN is a multi-layer Elman recurrent module for sequence data. It supports tanh or ReLU activation, optional inter-layer dropout, and bidirectional processing.
How it works
nn.RNN applies a stacked Elman recurrence to sequence features, using tanh or ReLU at each layer. It returns the output sequence and the final hidden state, with tensor layout controlled by batch_first and direction count controlled by bidirectional.
Syntax
torch.nn.RNN(input_size, hidden_size, num_layers=1, nonlinearity='tanh', bias=True, batch_first=False, dropout=0.0, bidirectional=False, device=None, dtype=None)Parameters
| Name | Type | Description |
|---|---|---|
input_size | int | Number of expected input features. |
hidden_size | int | Number of features in each hidden state. |
num_layers | int | Number of recurrent layers; default 1. |
nonlinearity | str | Activation: 'tanh' or 'relu'; default 'tanh'. |
bias | bool | Whether to use bias weights; default True. |
batch_first | bool | Use (batch, sequence, feature) input/output layout; default False. |
dropout | float | Inter-layer dropout probability; default 0. |
bidirectional | bool | Whether to use a bidirectional RNN; default False. |
device | torch.device | Optional device for parameters. |
dtype | torch.dtype | Optional dtype for parameters. |
Returns
A tuple (output, h_n): output contains the output features from the final recurrent layer for each timestep, and h_n contains the final hidden state for each layer and direction.
Key points
- num_layers stacks recurrent layers; each layer receives the preceding layer's output.
- nonlinearity accepts only 'tanh' or 'relu', with tanh as the default.
- bidirectional enables forward and reverse recurrent processing.
- dropout defaults to zero and configures the probability used for inter-layer dropout when enabled.
Common mistakes
- dropout is inserted only between recurrent layers, so it has no effect when num_layers is 1.
- batch_first changes input and output layout but does not change the layout of the hidden-state tensor.
Example: Run a batched RNN
import torch
from torch import nn
rnn = nn.RNN(input_size=4, hidden_size=3, batch_first=True)
x = torch.randn(2, 5, 4)
y, h_n = rnn(x)
print(y.shape, h_n.shape) # torch.Size([2, 5, 3]) torch.Size([1, 2, 3])Creates a single-layer tanh RNN for batches of five-step sequences with four features, then returns every timestep output and the final hidden state.
When to use it
Build deeper recurrent encoders with multiple RNN layers.
Stacked recurrence
Combine past and future context with bidirectional processing.
Two-direction context
Interview questions
QWhat does nn.RNN compute?
The module consumes a sequence and computes an Elman recurrence at each timestep. Its nonlinearity can be tanh or ReLU, and multiple layers pass each layer's outputs to the next.
Follow-up: How does bidirectionality affect outputs? It runs forward and reverse recurrent directions, so the feature dimension of each output is multiplied by two.
QHow do tensor shapes depend on batch_first?
With batch_first=False, inputs and outputs use (sequence, batch, feature); with batch_first=True, they use (batch, sequence, feature). The hidden state remains separate from this layout choice.
Follow-up: What shape should the initial hidden state use for a bidirectional two-layer model? Its first dimension is num_layers times two, followed by batch and hidden size.
QWhich configuration options change the recurrence?
The nonlinearity argument accepts tanh or relu, defaulting to tanh. bias controls whether the input-to-hidden and hidden-to-hidden bias parameters are used.
Follow-up: When is dropout applied? For nonzero dropout, it is applied to outputs between recurrent layers, not after the final layer.
Related
torch.nn.RNNCelltorch.nn.LSTMtorch.nn.GRUtorch.nn.modules.rnn.RNNBasetorch.nn.utils.rnn.pack_padded_sequence