Represent words or categorical IDs as dense vectors.
Integer-token sequences
Keras · Class
keras.layers.Embedding maps nonnegative integer indexes to dense vectors through a trainable embedding matrix. Configure the vocabulary capacity with input_dim and the vector width with output_dim.
Embedding stores a trainable matrix whose rows represent vocabulary entries. When called with integer indices, it looks up the corresponding rows and returns one vector per input index, adding an output-dimension axis to the input shape.
keras.layers.Embedding(input_dim, output_dim, embeddings_initializer="uniform", embeddings_regularizer=None, embeddings_constraint=None, mask_zero=False, weights=None, lora_rank=None, lora_alpha=None, quantization_config=None, **kwargs)| Name | Type | Description |
|---|---|---|
input_dim | Integer | Vocabulary size; valid indexes run up to input_dim - 1. |
output_dim | Integer | Width of each dense embedding vector. |
embeddings_initializer | Initializer | Initializer used for the embeddings matrix. |
embeddings_regularizer | Regularizer | Regularizer applied to the embeddings matrix. |
embeddings_constraint | Constraint | Constraint applied to the embeddings matrix. |
mask_zero | Boolean | Whether input value 0 is treated as padding and masked. |
weights | Array-like | Initial embedding values supplied directly. |
lora_rank | Integer or None | Rank of the low-rank adaptation applied to the embedding matrix. |
lora_alpha | Float or None | Scaling factor associated with low-rank adaptation. |
quantization_config | Configuration or None | Quantization configuration for the layer. |
**kwargs | Keyword arguments | Additional keyword arguments passed to the base layer. |
A 3D tensor with shape (batch_size, input_length, output_dim) for the documented 2D input.
input_dim to one more than the largest index that can occur.output_dim controls how many numeric features each lookup returns.embeddings_initializer to choose how the embedding matrix is initialized.import keras
import numpy as np
layer = keras.layers.Embedding(input_dim=1000, output_dim=64)
inputs = np.array([[4, 20], [7, 3]], dtype="int32")
outputs = layer(inputs)
print(outputs.shape) # (2, 2, 64)Creates a 64-dimensional vector for each integer index in a batch of length-two sequences.
Integer-token sequences
Sequence models
Large lookup tables
keras.layers.Embedding do?The layer maps integer indexes to dense vectors using a trainable embedding matrix. input_dim sets the vocabulary size, while output_dim sets each vector's width.
Follow-up: What output shape results from a 2D input? A tensor shaped (batch_size, input_length, output_dim).
input_dim and output_dim differ?input_dim is the maximum integer index plus one, so valid indexes range from zero through input_dim - 1. output_dim is the dimensionality of each returned embedding vector.
Follow-up: Why must input_dim include one beyond the largest index? The rows are zero-based, so the largest index needs its own row.
mask_zero?Setting mask_zero=True treats index zero as padding and propagates a mask for compatible downstream layers. Consequently, zero cannot represent a normal vocabulary entry, and downstream layers must support masking.
Follow-up: What input shape does the page specify? The documented input is 2D: (batch_size, input_length).
keras.layers.ReversibleEmbeddingkeras.initializerskeras.regularizers