Reduce a variable-length feature sequence to one representation
Sequence classification
Keras · Class
layers.LSTM
Keras LSTM processes 3D sequences with recurrent state updates. Configure it to return the final output, every timestep, and optionally the final hidden and cell states.
How it works
LSTM consumes a batch of sequences shaped as (batch, timesteps, features) and updates recurrent states across timesteps. It can emit either the final output or the complete sequence, optionally expose its final states, process time in reverse, and preserve state between batches.
Syntax
keras.layers.LSTM(units, activation="tanh", recurrent_activation="sigmoid", use_bias=True, kernel_initializer="glorot_uniform", recurrent_initializer="orthogonal", bias_initializer="zeros", unit_forget_bias=True, kernel_regularizer=None, recurrent_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, recurrent_constraint=None, bias_constraint=None, dropout=0.0, recurrent_dropout=0.0, seed=None, return_sequences=False, return_state=False, go_backwards=False, stateful=False, unroll=False, use_cudnn="auto", **kwargs)Parameters
| Name | Type | Description |
|---|---|---|
units | int | Positive integer specifying the output dimensionality. |
activation | string or callable or None | Activation function for the input transformation; defaults to tanh and may be None. |
recurrent_activation | string or callable | Activation function for the recurrent transformation; defaults to sigmoid. |
use_bias | bool | Whether to add a bias vector. |
kernel_initializer | initializer | Initializer for the input kernel. |
recurrent_initializer | initializer | Initializer for the recurrent kernel. |
bias_initializer | initializer | Initializer for the bias vector. |
unit_forget_bias | bool | Whether to add one to the forget-gate bias initialization. |
kernel_regularizer | regularizer or None | Regularizer applied to the input kernel. |
recurrent_regularizer | regularizer or None | Regularizer applied to the recurrent kernel. |
bias_regularizer | regularizer or None | Regularizer applied to the bias. |
activity_regularizer | regularizer or None | Regularizer applied to layer activity. |
Returns
For a 3D input, returns a 3D sequence when return_sequences=True, otherwise a 2D final output. When return_state=True, also returns the final hidden and cell states.
Key points
- LSTM is a recurrent layer based on the Long Short-Term Memory architecture introduced by Hochreiter in 1997.
- return_sequences controls whether the layer emits only the last timestep's output or the output at every timestep.
- The layer accepts sequences represented by a three-dimensional tensor: batch, timesteps, and feature.
- The default main activation is tanh, while recurrent activation defaults to sigmoid.
- The recurrent transformation uses sigmoid by default.
Common mistakes
- The fastest cuDNN path is conditional: use tanh and sigmoid activations, keep recurrent dropout at zero, avoid unrolling, retain bias, and provide strictly right-padded masked inputs.
- Masked sequences must be strictly right-padded when relying on the cuDNN implementation.
- Keras selects between cuDNN and backend-native implementations according to available hardware and the configured constraints.
Example: Stack LSTM layers
import keras
from keras import layers
inputs = keras.Input(shape=(None, 8))
x = layers.LSTM(16, return_sequences=True)(inputs)
outputs = layers.LSTM(4)(x)
model = keras.Model(inputs, outputs)
model.summary()This model applies one LSTM at every timestep, then a second LSTM that emits only its final output.
When to use it
Return one representation per timestep with return_sequences=True
Sequence labeling
Expose final recurrent states for transferring context
Encoder-decoder models
Carry context between batches with stateful=True
Streaming or chunked sequences
Interview questions
QWhat input shape does LSTM expect, and what does return_sequences change?
The input is a 3D tensor with dimensions batch, timesteps, and feature. With return_sequences disabled, the layer returns the output from the final timestep; enabling it returns an output for every timestep.
Follow-up: How would you preserve recurrent context across separate batches? Set stateful=True and keep the batch organization consistent; reset the layer states when starting a new independent sequence.
QWhat does return_state provide?
return_state adds the final hidden state and final cell state to the layer result. These states can be used when connecting an encoder to a decoder or when manually managing recurrent context.
Follow-up: What changes when return_state and return_sequences are both enabled? The result contains the full output sequence followed by the final hidden and cell states.
QWhen can Keras use the cuDNN implementation?
The layer can use a cuDNN implementation or a backend-native implementation. The cuDNN path requires tanh activation, sigmoid recurrent activation, zero recurrent dropout, no unrolling, bias enabled, and compatible right-padded masking.
Follow-up: Why might an otherwise valid LSTM use the native path? Any violation of the cuDNN constraints, or unavailable compatible runtime hardware, can cause another implementation to be selected.
Related
LSTMCellRNNGRUBidirectionalMasking