Merge a skip connection with a transformed branch
Residual architectures
TensorFlow · Class
tf.keras.layers.Add performs elementwise addition over a list of same-shaped tensors and returns one tensor with that shape. It is commonly used to merge branches in a Keras model.
Add is a stateless Keras layer that combines a list of tensors by summing corresponding elements. Every input must have the same shape, and the returned tensor keeps that shape; in a functional model, call the layer with the tensors as a list.
tf.keras.layers.Add(**kwargs)| Name | Type | Description |
|---|---|---|
**kwargs | keyword arguments | Standard Keras layer keyword arguments. |
A single tensor containing the elementwise sum, with the same shape as the inputs.
import numpy as np
import tensorflow as tf
x1 = np.array([[1., 2.]])
x2 = np.array([[3., 4.]])
y = tf.keras.layers.Add()([x1, x2])
print(y.numpy()) # [[4. 6.]]Creates two same-shaped tensors and adds them element by element.
Residual architectures
Functional Keras graphs
Tensor arithmetic
Add performs elementwise addition. It receives a list of tensors with identical shapes and emits one tensor whose shape is unchanged.
Follow-up: How would you merge branches in the Keras Functional API? Instantiate Add and call it with the branch outputs as a list.
The layer constructor accepts **kwargs, which are standard layer keyword arguments. The tensors themselves are supplied when the layer is called.
Follow-up: Why is the tensor list not passed to the constructor? Because the layer object is configured first and then invoked on inputs as part of graph construction.
Add is useful for residual or skip connections when the branches produce matching shapes. The output is a single tensor containing the corresponding-element sums.
Follow-up: What must change if branch shapes differ? Project or otherwise transform a branch before Add so the inputs have the same shape.
tf.keras.layers.addtf.keras.layers.Layertf.keras.layers.Densetf.keras.Model