Incremental-style linear regression
Large regression datasets
Scikit-Learn · Class
SGDRegressor
SGDRegressor fits a regularized linear regression model using stochastic gradient descent. It supports several loss functions and L1, L2, or elastic-net penalties.
How it works
SGDRegressor learns a linear regression model by repeatedly estimating the loss gradient from individual samples and updating parameters with a decreasing learning rate. A configurable penalty regularizes the coefficients, supporting L2, L1, or elastic-net behavior.
Syntax
class sklearn.linear_model.SGDRegressor(loss='squared_error', *, penalty='l2', alpha=0.0001, l1_ratio=0.15, fit_intercept=True, max_iter=1000, tol=0.001, shuffle=True, verbose=0, epsilon=0.1, random_state=None, learning_rate='invscaling', eta0=0.01, power_t=0.25, warm_start=False, average=False)Parameters
| Name | Type | Description |
|---|---|---|
loss | str, default='squared_error' | Loss function: squared_error, huber, epsilon_insensitive, or squared_epsilon_insensitive. |
penalty | str, default='l2' | Regularization penalty: l2, l1, or elasticnet. |
alpha | float, default=0.0001 | Regularization strength. |
l1_ratio | float, default=0.15 | Elastic-net mixing parameter. |
fit_intercept | bool, default=True | Whether to estimate the intercept. |
max_iter | int, default=1000 | Maximum number of passes over the training data for fit. |
tol | float or None, default=0.001 | Stopping criterion. |
shuffle | bool, default=True | Whether to shuffle training data after each epoch. |
verbose | int, default=0 | Verbosity level. |
epsilon | float, default=0.1 | Epsilon in epsilon-insensitive losses. |
random_state | int, RandomState instance, or None, default=None | Controls pseudo-randomness used for shuffling and updates. |
learning_rate | str, default='invscaling' | Learning-rate schedule. |
Returns
An SGDRegressor estimator; fit returns the fitted estimator, predict returns predicted target values, and score returns the coefficient of determination.
Key points
- Fits a linear regression model while minimizing a regularized empirical loss.
- Uses sample-wise gradient estimates rather than computing each update from the whole dataset.
- Updates use a learning-rate schedule whose strength decreases over time.
- Supports L2, L1, and elastic-net coefficient penalties.
- The squared-error option corresponds to ordinary least-squares loss.
- Huber loss reduces emphasis on fitting outliers compared with squared error.
- max_iter defines the maximum number of training epochs.
Common mistakes
- max_iter limits passes through the data for fit, but does not control partial_fit.
- Disabling fit_intercept requires the input features and targets to be centered beforehand.
- Provide dense floating-point feature arrays.
Example: Fit and predict with SGDRegressor
from sklearn.linear_model import SGDRegressor
X = [[0], [1], [2], [3]]
y = [0, 1, 2, 3]
model = SGDRegressor(random_state=0, max_iter=1000, tol=1e-3)
model.fit(X, y)
print(model.predict([[4]]))Creates a stochastic-gradient linear regressor, fits it to one-feature data, and predicts a target for a new sample.
When to use it
Robust regression with Huber loss
Data containing influential outliers
L1 or elastic-net regularization
Models requiring coefficient shrinkage or sparsity
Interview questions
QHow does SGDRegressor train a model?
SGDRegressor is a linear regressor trained by stochastic gradient descent. It estimates the loss gradient one sample at a time and updates the model using a decreasing learning-rate schedule.
Follow-up: How does regularization affect the model? It adds a penalty that shrinks coefficients toward zero, using L2, L1, or a mixture of both.
QWhat loss functions does SGDRegressor support?
The loss parameter selects the regression objective. The documented choices are squared error, Huber, epsilon-insensitive, and squared epsilon-insensitive loss; Huber reduces the influence of sufficiently large residuals.
Follow-up: When might Huber be preferable? It can be useful when outliers should have less influence than under ordinary squared-error fitting.
QHow do max_iter and tol affect training?
max_iter sets the maximum number of epochs for fit, while tol supplies a stopping tolerance. Training can stop when the loss fails to improve sufficiently for the configured number of consecutive epochs.
Follow-up: Does max_iter govern partial_fit? No; the documentation explicitly limits its effect to fit, not partial_fit.
Related
SGDClassifierRidgeLassoElasticNetLinearRegression