Outlier-resistant regression
Regression data contains a small number of extreme target or measurement errors.
Scikit-Learn · Class
HuberRegressor
HuberRegressor is an L2-regularized linear regression model that reduces the influence of outliers. It switches from squared to absolute loss when scaled residuals exceed epsilon.
How it works
HuberRegressor jointly estimates coefficients, an intercept, and a scale. It uses squared loss for observations whose scaled residual is within epsilon and absolute loss for larger residuals, while applying L2 regularization to the coefficients.
Syntax
class sklearn.linear_model.HuberRegressor(*, epsilon=1.35, max_iter=100, alpha=0.0001, warm_start=False, fit_intercept=True, tol=1e-05)Parameters
| Name | Type | Description |
|---|---|---|
epsilon | float, default=1.35 | Controls the point where squared loss changes to absolute loss; must be greater than 1.0. |
max_iter | int, default=100 | Maximum number of iterations for the optimization solver. |
alpha | float, default=0.0001 | L2 regularization strength applied to the coefficients. |
warm_start | bool, default=False | Whether to reuse the solution from a previous fit. |
fit_intercept | bool, default=True | Whether to calculate the intercept. |
tol | float, default=1e-05 | Optimization stopping tolerance. |
Returns
A fitted HuberRegressor estimator from fit; predictions from predict are target values.
Key points
- It combines linear regression with L2 regularization and reduced sensitivity to outlying observations.
- Residuals below the epsilon threshold use squared loss.
- Residuals beyond the threshold use absolute loss, limiting the effect of extreme errors.
- Training estimates coefficients, the intercept, and a residual-scale parameter together.
- The epsilon hyperparameter controls the robustness-versus-squared-loss trade-off.
Common mistakes
- Reducing epsilon increases robustness, but changes which residuals receive the absolute-loss treatment.
- Feature-scale differences are not handled automatically, so scale features when their magnitudes differ substantially.
Example: Robust regression with an outlier
from sklearn.linear_model import HuberRegressor
X = [[1.0], [2.0], [3.0], [20.0]]
y = [1.1, 2.0, 3.2, 3.0]
model = HuberRegressor().fit(X, y)
print(model.coef_)
print(model.predict([[4.0]]))Fits a robust linear model and predicts a target for a new feature value.
When to use it
Robust linear baseline
A linear relationship is expected, but ordinary least squares is too sensitive to anomalous observations.
Regularized modeling with downweighted outliers
A regularized linear model is needed while retaining partial influence from outliers.
Interview questions
QWhat problem does HuberRegressor solve?
HuberRegressor is an L2-regularized linear regression estimator designed to reduce the influence of outliers. It uses squared loss for small scaled residuals and absolute loss for large scaled residuals.
Follow-up: How does epsilon affect behavior? A smaller epsilon classifies more observations as outliers and makes the estimator more robust, while a larger value makes it closer to ordinary squared-loss regression.
QWhy is the scale parameter included?
The model optimizes coefficients, the intercept, and a scale parameter. The scale makes the residual threshold relative to the response scale, so the same epsilon can preserve robustness when the target is rescaled.
Follow-up: What regularization remains? The objective includes an alpha-weighted squared L2 penalty on the coefficient vector, not on the intercept.
QHow does it differ from Ridge and other robust regressors?
HuberRegressor differs from Ridge because it applies a linear loss to samples treated as outliers. Unlike RANSAC and Theil-Sen, it does not discard outliers; it gives them less influence.
Follow-up: When might another robust estimator be preferable? RANSAC or Theil-Sen can be alternatives, especially when the sample count is very large relative to the feature count or when their distinct robustness assumptions fit better.
Related
RidgeRANSACRegressorTheilSenRegressorSGDRegressor