Adaptive activation slope
A model needs an activation whose behavior for negative inputs is learned during training.
Keras · Class
layers.PReLU
PReLU is a trainable activation that keeps nonnegative values unchanged and applies a learned alpha slope to negative values. Its alpha parameters can be initialized, regularized, constrained, and shared across selected axes.
How it works
PReLU applies a learned slope to negative inputs while leaving nonnegative inputs unchanged. Its learnable alpha values can be initialized, regularized, constrained, and optionally shared across selected axes.
Syntax
keras.layers.PReLU(alpha_initializer="Zeros", alpha_regularizer=None, alpha_constraint=None, shared_axes=None, **kwargs)Parameters
| Name | Type | Description |
|---|---|---|
alpha_initializer | str or initializer | Initializer function for the trainable alpha weights. |
alpha_regularizer | regularizer or None | Regularizer applied to the trainable alpha weights. |
alpha_constraint | constraint or None | Constraint applied to the trainable alpha weights. |
shared_axes | list or None | Axes along which alpha parameters are shared. |
**kwargs | keyword arguments | Additional base-layer keyword arguments, including name and dtype. |
Returns
A tensor with the same shape as the input, transformed according to the PReLU function.
Key points
- PReLU is an activation layer with a trainable negative-region slope.
- Negative inputs are multiplied by the learned alpha value.
- Nonnegative inputs pass through unchanged.
- alpha_initializer selects the initialization method for PReLU's trainable coefficients.
- alpha_regularizer can add a regularization penalty to the learned coefficients.
- alpha_constraint can enforce restrictions on the learned coefficients.
- Additional base-layer options such as name and dtype can be supplied through kwargs.
Common mistakes
- The negative-side coefficient is learned, so PReLU adds trainable parameters rather than behaving like a fixed activation.
- Use shared_axes deliberately: it changes which dimensions reuse the same learned alpha values.
Example: Channel-wise PReLU after convolution
import keras
from keras import layers
inputs = keras.Input(shape=(32, 32, 3))
x = layers.Conv2D(16, 3, padding="same")(inputs)
x = layers.PReLU(shared_axes=[1, 2])(x)
outputs = layers.GlobalAveragePooling2D()(x)
model = keras.Model(inputs, outputs)
model.summary()Creates a convolutional model whose PReLU layer learns one negative-side slope per channel by sharing parameters across height and width.
When to use it
Channel-wise convolutional activation
Convolutional feature maps should use one learned negative slope per channel instead of one per spatial location.
Configured trainable coefficients
The learned alpha values need controlled initialization or optimization behavior.
Interview questions
QWhat function does PReLU implement?
PReLU computes f(x) = alpha * x for x < 0 and f(x) = x for x >= 0. Alpha is learned and has the same shape as the input unless parameter sharing is configured.
Follow-up: How would you reduce parameters for convolutional feature maps? Set shared_axes=[1, 2] for batch-height-width-channel data so each channel shares its slope across spatial positions.
QWhat do the alpha-related arguments control?
alpha_initializer controls how the learnable weights start, while alpha_regularizer and alpha_constraint apply regularization and constraints to those weights. The defaults are a zero initializer and no regularizer or constraint.
Follow-up: Why might a constraint be useful? It can restrict the learned negative-region slopes to a desired range, using the layer's alpha_constraint argument.
QHow does shared_axes affect the layer?
shared_axes specifies axes along which PReLU reuses learnable parameters. For 2D convolutional feature maps, sharing axes 1 and 2 gives each filter one parameter across height and width.
Follow-up: What is the trade-off? Sharing reduces the number of learned coefficients but prevents different spatial positions from learning independent negative slopes.
Related
LayerConv2DReLULeakyReLUELU