Local regression
Predicting a numeric outcome from nearby feature vectors.
Scikit-Learn · Class
KNeighborsRegressor
KNeighborsRegressor predicts continuous targets from the nearest training samples. It supports uniform, inverse-distance, or callable weighting and configurable neighbor-search algorithms and distance metrics.
How it works
After fitting, the estimator finds the nearest training samples for each query and interpolates their target values. The weights setting controls whether neighbors contribute equally, according to inverse distance, or through a custom weighting function; the neighbor-search algorithm and Minkowski metric determine how proximity is computed.
Syntax
class sklearn.neighbors.KNeighborsRegressor(*, n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='minkowski', metric_params=None, n_jobs=None)Parameters
| Name | Type | Description |
|---|---|---|
n_neighbors | int | Number of neighbors used by default for kneighbors queries; default 5. |
weights | {'uniform', 'distance'}, callable or None | Prediction weighting: uniform, distance, a callable, or None; default uniform. |
algorithm | {'auto', 'ball_tree', 'kd_tree', 'brute'} | Neighbor-search method: auto, ball_tree, kd_tree, or brute; default auto. |
leaf_size | int | Leaf size passed to the tree-based search structures; default 30. |
p | float | Power parameter for the Minkowski metric; default 2. |
metric | str or callable | Distance metric used for neighbor searches; default minkowski. |
metric_params | dict or None | Additional keyword arguments for the metric; default None. |
n_jobs | int or None | Number of parallel jobs for neighbor search; default None. |
Returns
A fitted KNeighborsRegressor instance from fit; predictions from predict are target estimates for the supplied data.
Key points
- This class is a supervised regressor that uses nearby training samples to estimate continuous targets.
- Each prediction is formed by interpolating target values belonging to the nearest training points.
n_neighborsspecifies the default count of neighbors considered for neighbor queries.weightsdetermines how each selected neighbor influences the prediction.- Distance weighting gives nearer observations greater influence through inverse-distance weights.
- The
algorithmoption controls the method used to locate nearest neighbors. pis the Minkowski-distance power parameter.- Values of
pother than the common special cases produce the corresponding Minkowski distance.
Common mistakes
- The
pvalue changes the Minkowski distance: 1 gives Manhattan distance, while 2 gives Euclidean distance. algorithm="auto"delegates the neighbor-search method choice to the estimator.
Example: Distance-weighted nearest-neighbor regression
from sklearn.neighbors import KNeighborsRegressor
X = [[0], [1], [2], [3]]
y = [0, 1, 2, 3]
model = KNeighborsRegressor(n_neighbors=2, weights="distance")
model.fit(X, y)
print(model.predict([[1.5]]))Fits a two-neighbor regressor and predicts the target at a new point using distance-based neighbor weights.
When to use it
Distance-weighted estimation
Giving nearby observations stronger influence than distant ones.
Metric-based regression
Using a custom distance metric for domain-specific similarity.
Interview questions
QHow does KNeighborsRegressor produce a prediction?
KNeighborsRegressor predicts by local interpolation of target values from the nearest training samples. With uniform weights, every selected neighbor contributes equally; with distance weights, closer samples contribute more.
Follow-up: How would you make the estimator emphasize nearby observations? Use weights="distance", or supply a callable that maps distances to weights.
QWhat does n_neighbors control?
n_neighbors sets how many neighbors are used by default for neighbor queries. A smaller value makes predictions more local, while a larger value incorporates more training observations.
Follow-up: What trade-off does changing this value create? Smaller neighborhoods can be more sensitive to local variation, whereas larger neighborhoods smooth predictions over a broader region.
QWhich neighbor-search algorithms can be selected?
The algorithm parameter selects among auto, ball_tree, kd_tree, and brute. auto attempts to choose the most appropriate method from the values passed to fit.
Follow-up: Which distance family is used by default? The default metric is Minkowski with p=2, corresponding to Euclidean distance.
Related
KNeighborsClassifierRadiusNeighborsRegressorNearestNeighborsBallTreeKDTree