Regularized regression
Predicting a continuous target with a linear relationship while controlling coefficient magnitude.
Scikit-Learn · Class
BayesianRidge
BayesianRidge is a probabilistic linear regression estimator with Bayesian regularization. It learns noise and weight precisions from the data while fitting the model.
How it works
BayesianRidge fits a probabilistic linear regression model while estimating the noise precision and weight precision from the data. It uses iterative updates that maximize the log marginal likelihood, producing regularized coefficients and exposing predictive uncertainty through the fitted noise estimate.
Syntax
class sklearn.linear_model.BayesianRidge(*, max_iter=300, tol=0.001, alpha_1=1e-06, alpha_2=1e-06, lambda_1=1e-06, lambda_2=1e-06, alpha_init=None, lambda_init=None, compute_score=False, fit_intercept=True, copy_X=True, verbose=False)Parameters
| Name | Type | Description |
|---|---|---|
max_iter | int, default=300 | Maximum number of iterations for the fitting procedure. |
tol | float, default=1e-3 | Convergence threshold for the iterative procedure. |
alpha_1 | float, default=1e-6 | Shape hyperparameter of the Gamma prior on alpha. |
alpha_2 | float, default=1e-6 | Rate hyperparameter of the Gamma prior on alpha. |
lambda_1 | float, default=1e-6 | Shape hyperparameter of the Gamma prior on lambda. |
lambda_2 | float, default=1e-6 | Rate hyperparameter of the Gamma prior on lambda. |
alpha_init | float or None, default=None | Initial value for alpha; if None, it is initialized from the data. |
lambda_init | float or None, default=None | Initial value for lambda; if None, it is initialized from the data. |
compute_score | bool, default=False | Whether to compute and store the marginal log-likelihood score. |
fit_intercept | bool, default=True | Whether to calculate the intercept. |
copy_X | bool, default=True | Whether to copy the input feature matrix. |
verbose | bool, default=False | Whether to output progress information. |
Returns
An instance of BayesianRidge, a fitted linear regression estimator with learned coefficients, intercept, alpha_, lambda_, sigma_, and optionally scores_.
Key points
- BayesianRidge is a linear regression estimator that applies Bayesian regularization.
- Use fit(X, y) to learn the regression model from feature data and targets.
- The default iteration limit is 300; increase it when iterative fitting needs more steps.
- tol controls the convergence threshold for the iterative optimization.
- Set compute_score=True to retain the log marginal likelihood values during fitting.
- By default, the estimator learns an intercept; disable this when features are already centered appropriately.
Example: Fit and predict with BayesianRidge
from sklearn.linear_model import BayesianRidge
X = [[0.0], [1.0], [2.0], [3.0]]
y = [0.1, 1.1, 1.9, 3.2]
model = BayesianRidge(compute_score=True)
model.fit(X, y)
print(model.predict([[2.5]]))
print(model.score(X, y))Fits Bayesian ridge regression, predicts a new target, and evaluates the fitted model on the training data.
When to use it
Probabilistic predictions
Regression tasks where estimating observation noise and coefficient uncertainty is useful.
Data-driven regularization
Selecting regularization strength from the observed data instead of fixing it manually.
Interview questions
QWhat does BayesianRidge estimate during fitting?
BayesianRidge is a probabilistic linear regression estimator with Bayesian regularization. During fitting, it estimates the coefficient vector and the precision parameters alpha and lambda by maximizing the log marginal likelihood.
Follow-up: How does lambda differ from alpha? Lambda is the precision of the coefficient prior, while alpha is the precision of the observation noise.
QWhat Bayesian assumptions underlie the estimator?
The coefficient prior is a spherical Gaussian with precision lambda. Gamma priors are used for alpha and lambda, making them conjugate priors for the Gaussian precision.
Follow-up: Why does this provide regularization? The Gaussian prior favors smaller coefficients, shrinking estimates toward zero while allowing the data to determine the regularization strength.
QWhat do max_iter, tol, and compute_score control?
The estimator stops after at most max_iter iterations or when the change falls below tol. Setting compute_score=True additionally stores the marginal log-likelihood values evaluated during the iterations.
Follow-up: When might compute_score be useful? It can help inspect convergence behavior or compare fitted models using their marginal log-likelihood trajectories.
Related
RidgeARDRegressionLinearRegressionGammaRegressor