Skip to content
masterclass Buddy
Start learning

TensorFlow · Class

layers.MultiHeadAttention

MultiHeadAttention is a Keras layer that lets each query position use information from a key and value sequence. It supports self-attention when all three inputs are the same.

Last updated

Syntax

tf.keras.layers.MultiHeadAttention(num_heads, key_dim, value_dim=None, dropout=0.0, use_bias=True, output_shape=None, attention_axes=None, sliding_window=None, flash_attention=None, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, use_gate=False, seed=None, **kwargs)

In-depth API docs are only for subscribers: parameters, examples, common mistakes and interview practice.

Subscribe
  • tf.keras.layers.Layer
  • tf.keras.layers.Attention

Source: tf.keras.layers.MultiHeadAttention | TensorFlow v2.21.0. Checked 2026-11-03.