Smooth nonlinear hidden activations
Use ELU as an intermediate activation in a feed-forward or convolutional model.
Keras · Class
layers.ELU
layers.ELU applies the Exponential Linear Unit activation as a Keras layer. It leaves nonnegative values unchanged and uses an alpha-scaled exponential branch for negative values.
How it works
ELU transforms each input element with a linear rule for nonnegative values and an exponential rule for negative values. The alpha parameter controls the scale of the negative branch, while the remaining keyword arguments configure the underlying Keras layer.
Syntax
keras.layers.ELU(alpha=1.0, **kwargs)Parameters
| Name | Type | Description |
|---|---|---|
alpha | float | Slope or scale applied to the negative part of the ELU function; defaults to 1.0. |
**kwargs | keyword arguments | Keyword arguments accepted by the base Keras layer, such as name and dtype. |
Returns
A Keras layer that applies the Exponential Linear Unit function to its input.
Key points
- ELU is a Keras layer that applies the Exponential Linear Unit transformation to its input.
- For negative inputs, the output follows an exponential curve scaled by alpha.
- For nonnegative inputs, ELU returns the input unchanged.
- The alpha parameter controls the negative branch's slope and scale.
- If omitted, alpha is set to 1.0.
- Additional keyword arguments are forwarded to the base layer, including configuration such as name and dtype.
Example: Using ELU in a model
import keras
from keras import layers
model = keras.Sequential([
layers.Input(shape=(8,)),
layers.ELU(alpha=1.0),
layers.Dense(1),
])
outputs = model(keras.random.normal((2, 8)))
print(outputs.shape)Creates a model that applies ELU elementwise before a dense output layer.
When to use it
Tune negative-side behavior
Adjust the negative branch while retaining the identity behavior for nonnegative inputs.
Named and typed model layers
Configure the layer with standard Keras metadata or dtype settings.
Interview questions
QHow is the ELU function defined?
ELU applies the identity function when x is at least zero and uses alpha multiplied by exp(x) minus one when x is negative. This gives the activation a smooth negative branch rather than clamping all negative inputs to zero.
Follow-up: What does alpha change? It changes the slope and negative-side saturation level of the ELU branch without changing the positive branch.
QWhat arguments does layers.ELU accept?
The constructor takes alpha, which defaults to 1.0, and arbitrary base-layer keyword arguments. Base-layer keywords can include options such as name and dtype.
Follow-up: How would you attach a layer name? Pass name as a keyword argument, for example layers.ELU(name='hidden_elu').
QHow would you use ELU in a Keras model?
ELU is a layer object, so it can be inserted into a Keras model and called on tensors. It performs the exponential linear transformation on the layer output.
Follow-up: What alternative API represents the same operation directly? The related keras.activations.elu function applies ELU to an input tensor without creating an activation layer.
Related
keras.activations.elukeras.layers.Activationkeras.layers.ReLU