TensorFlow · Class

Dense

A fully connected neural-network layer that transforms the last input dimension into units. It can add a bias and apply an optional element-wise activation.

How it works

Dense computes an affine transformation from the last input dimension to the requested number of output units, optionally adding a bias and applying an element-wise activation. For inputs with rank greater than two, it preserves the preceding dimensions and transforms only the final axis.

Syntax

tf.keras.layers.Dense(units, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None, lora_rank=None, **kwargs)

Parameters

Name	Type	Description
`units`	`int`	Positive integer specifying the dimensionality of the output space.
`activation`	`str or callable`	Optional element-wise activation function; omitted means linear activation.
`use_bias`	`bool`	Whether to include a bias vector.
`kernel_initializer`	`Initializer`	Initializer for the kernel weight matrix.
`bias_initializer`	`Initializer`	Initializer for the bias vector.
`kernel_regularizer`	`Regularizer`	Regularizer applied to the kernel weight matrix.
`bias_regularizer`	`Regularizer`	Regularizer applied to the bias vector.
`activity_regularizer`	`Regularizer`	Regularizer applied to the layer output.
`kernel_constraint`	`Constraint`	Constraint applied to the kernel weight matrix.
`bias_constraint`	`Constraint`	Constraint applied to the bias vector.
`lora_rank`	`int`	Optional integer rank enabling Low-Rank Adaptation.

Returns

A tensor whose final dimension has size units; preceding input dimensions are preserved.

Key points

units controls the width of the layer's output and must be a positive integer.
The layer performs a learned matrix multiplication, bias addition when enabled, and optional activation.
The activation is applied element by element after the affine transformation.
kernel_initializer determines how the layer initializes its learned weight matrix.
activity_regularizer can add regularization based on the layer output.
kernel_constraint restricts or transforms permissible kernel values during optimization.

Common mistakes

Setting lora_rank changes the forward pass to use LoRA; the base kernel becomes non-trainable and the update is represented by lower-rank trainable matrices.
Leaving activation unset produces a linear output rather than applying a nonlinear activation.
The bias is omitted from the computation when use_bias is false.

Example: Apply a dense layer

import tensorflow as tf

layer = tf.keras.layers.Dense(3, activation="relu")
x = tf.constant([[1.0, 2.0, 3.0, 4.0]])
y = layer(x)
print(y.shape)  # (1, 3)

Creates a three-unit dense layer, infers four input features on first use, and produces a tensor with three output features.

When to use it

Feature projection

Build a standard fully connected block between feature representations.

Classification head

Produce class scores or probabilities from a learned representation.

LoRA fine-tuning

Adapt a pretrained dense layer with fewer trainable parameters.

Interview questions

QWhat computation does Dense perform?

Dense maps the final input dimension to units using a learned kernel, optionally adds a bias, and then applies the selected activation. Conceptually, its computation is activation(dot(input, kernel) + bias).

Follow-up: How does it handle higher-rank input? It applies the same transformation independently across all dimensions preceding the last one, while changing only the final dimension to units.

QWhat are the roles of units, kernel, and activation?

units specifies the dimensionality of the output space and must be a positive integer. The kernel is learned by the layer, while activation controls the optional element-wise nonlinear transformation.

Follow-up: How can the parameter count be estimated? With input width d and units u, it is d×u plus u when use_bias is enabled.

QWhat does lora_rank do?

When lora_rank is provided, Dense uses Low-Rank Adaptation: its original kernel is made non-trainable and a low-rank trainable delta replaces direct kernel updates. Without that argument, the layer follows its ordinary dense computation.

Follow-up: Why use LoRA? It can reduce the number of trainable adaptation parameters while retaining the original kernel as fixed base weights.

tf.keras.layers
tf.keras.layers.EinsumDense
tf.keras.Layer

Source: tf.keras.layers.Dense | TensorFlow v2.16.1. Checked 2026-10-15.