Reduce the effective feature set by shrinking some coefficients to zero.
Feature selection
Scikit-Learn · Class
Lasso
Lasso is a linear regression estimator with an L1 penalty. It encourages sparse coefficients by allowing some feature weights to become exactly zero.
How it works
Lasso fits a linear regression model while adding an L1 penalty to the coefficient vector. The penalty encourages sparse solutions, so some coefficients can become exactly zero; scikit-learn fits the model with coordinate descent.
Syntax
class sklearn.linear_model.Lasso(alpha=1.0, *, fit_intercept=True, precompute=False, copy_X=True, max_iter=1000, tol=0.0001, warm_start=False, positive=False, random_state=None, selection='cyclic')Parameters
| Name | Type | Description |
|---|---|---|
alpha | float | Constant multiplying the L1 term; must be a non-negative float. Default: 1.0. |
fit_intercept | bool | Whether to calculate the intercept. Default: True. |
precompute | bool or array-like | Whether to use a precomputed Gram matrix. Default: False. |
copy_X | bool | Whether to copy X. Default: True. |
max_iter | int | Maximum number of iterations. Default: 1000. |
tol | float | Optimization tolerance. Default: 0.0001. |
warm_start | bool | Reuse the solution from the previous call to fit. Default: False. |
positive | bool | Restrict coefficients to be non-negative. Default: False. |
random_state | int or RandomState instance or None | Random-state seed used when selection is random. Default: None. |
selection | {'cyclic', 'random'} | Coordinate selection strategy: 'cyclic' or 'random'. Default: 'cyclic'. |
Returns
A fitted Lasso estimator from fit; predict returns predicted target values.
Key points
- Lasso is a linear regression estimator with an L1 regularization term.
- Its objective combines normalized squared error with alpha times the L1 norm of the coefficients.
- The fitted model can produce sparse coefficients, including coefficients exactly equal to zero.
- Increasing the regularization strength through alpha changes how sparse the coefficient vector tends to be.
- Coefficient fitting is performed using coordinate descent.
- Lasso corresponds to ElasticNet with l1_ratio set to 1.0, meaning it has no L2 penalty.
Common mistakes
- An alpha value of zero removes the L1 regularization; use LinearRegression instead for ordinary least squares.
- For performance-sensitive fitting, pass X as a Fortran-contiguous NumPy array to avoid unnecessary memory duplication.
Example: Fit and predict with Lasso
from sklearn.linear_model import Lasso
X = [[0, 0], [1, 1], [2, 2]]
y = [0, 1, 2]
model = Lasso(alpha=0.1)
model.fit(X, y)
print(model.predict([[1, 1]]))Creates a Lasso regressor, fits it to two-feature data, and predicts a target for one new sample.
When to use it
Build interpretable linear models with fewer active feature weights.
Sparse linear prediction
Control coefficient magnitude and sparsity when ordinary least squares overfits.
Regularized regression
Interview questions
QWhat objective does Lasso optimize?
Lasso minimizes squared-error loss plus an L1 penalty, written as [1m(1 / (2 * n_samples)) * ||y - Xw||^2_2 + alpha * ||w||_1[0m. The alpha parameter controls the penalty strength and therefore the sparsity of the learned coefficients.
Follow-up: What happens when alpha is zero? It becomes ordinary least squares; the documentation recommends LinearRegression for that case.
QWhy use Lasso instead of unregularized linear regression?
Lasso is a linear model that can set coefficients exactly to zero, producing a sparse model. This can reduce the number of features on which predictions depend.
Follow-up: Which parameter controls this behavior? alpha controls the degree of sparsity: larger regularization generally favors fewer nonzero coefficients.
QWhich optimization algorithm does Lasso use?
The estimator uses coordinate descent to fit the coefficients. Its constructor supports cyclic or random feature selection through selection, with cyclic as the default.
Follow-up: When can random selection help? With a sufficiently small tolerance, random coordinate selection can sometimes converge faster than cyclic selection.
Related
ElasticNetLassoCVLassoLarsLassoLarsCVLassoLarsICLinearRegressionlasso_path