Nonlinear regression
Predicting a continuous target when relationships may be nonlinear.
Scikit-Learn · Class
SVR
SVR is scikit-learn’s epsilon-support vector regression estimator. It supports linear and nonlinear kernels, with C controlling regularization and epsilon defining the no-penalty prediction tube.
How it works
SVR fits an epsilon-support vector regression model using a selectable kernel. The C parameter controls regularization strength, while epsilon defines a tube around the predictions in which errors are not penalized; the default RBF kernel can model nonlinear relationships.
Syntax
SVR(*, kernel='rbf', degree=3, gamma='scale', coef0=0.0, tol=0.001, C=1.0, epsilon=0.1, shrinking=True, cache_size=200, verbose=False, max_iter=-1)Parameters
| Name | Type | Description |
|---|---|---|
kernel | {'linear', 'poly', 'rbf', 'sigmoid', 'precomputed'} or callable | Kernel type: linear, poly, rbf, sigmoid, precomputed, or a callable; default is rbf. |
degree | int | Polynomial-kernel degree; default is 3. |
gamma | {'scale', 'auto'} or float | Kernel coefficient for rbf, poly, and sigmoid; default is scale. |
coef0 | float | Independent kernel-function term; default is 0.0. |
tol | float | Stopping-criterion tolerance; default is 1e-3. |
C | float | Regularization parameter; default is 1.0. |
epsilon | float | Epsilon-tube width; default is 0.1. |
shrinking | bool | Whether to use the shrinking heuristic; default is True. |
cache_size | float | Kernel-cache size in MB; default is 200. |
Returns
An SVR estimator whose fit method learns from training data and whose predict method produces regression outputs.
Key points
- SVR is the scikit-learn estimator for epsilon-support vector regression.
- C and epsilon are the model's central free parameters.
- The estimator is implemented on top of libsvm.
- kernel chooses the algorithm's kernel type; the default is rbf.
- gamma controls the kernel coefficient for RBF, polynomial, and sigmoid kernels.
- cache_size sets the amount of memory reserved for the kernel cache.
Common mistakes
- Training can become impractical as the sample count grows because the documented fit complexity is more than quadratic.
- C must be greater than zero; zero or negative values are invalid.
- epsilon cannot be negative.
Example: Fit and predict with SVR
from sklearn.svm import SVR
X = [[0], [1], [2], [3]]
y = [0.0, 1.1, 1.9, 3.2]
model = SVR(kernel="rbf", C=10.0, epsilon=0.1)
model.fit(X, y)
print(model.predict([[1.5]]))Creates an epsilon-SVR model, fits it to one-feature training data, and predicts a target for a new sample.
When to use it
Kernel-based regression
Fitting regression with a chosen kernel representation.
Epsilon-insensitive fitting
Modeling data where small prediction deviations should be ignored.
Interview questions
QWhat is SVR?
SVR is epsilon-support vector regression. It uses a kernel, with RBF as the default, and its two principal free parameters are C and epsilon.
Follow-up: What does epsilon do? It defines the prediction tube in which deviations from the target do not incur training-loss penalties.
QHow do C and epsilon differ?
C is the regularization parameter, and regularization strength varies inversely with C. Increasing C generally permits less regularization, while decreasing it applies more.
Follow-up: Which parameter must be strictly positive? C must be strictly positive; epsilon instead only needs to be non-negative.
QHow does the kernel parameter affect SVR?
The kernel selects the transformation used by the algorithm; supported choices include linear, polynomial, RBF, sigmoid, precomputed, or a callable. A callable is used to precompute the kernel matrix.
Follow-up: Why might SVR be unsuitable for a large dataset? Its fit time complexity is more than quadratic in the number of samples.
Related
LinearSVRNuSVRSVCsklearn.svm