Local pattern classification
Classifying observations where nearby points tend to share labels.
Scikit-Learn · Class
KNeighborsClassifier
KNeighborsClassifier implements supervised classification by voting among a sample’s nearest neighbors. It supports uniform or distance-based voting and configurable neighbor-search algorithms and distance metrics.
How it works
The estimator stores labeled training samples and classifies a query by voting among its nearest neighbors. Neighbor influence can be uniform or based on inverse distance, while the selected distance metric and search algorithm control neighborhood computation.
Syntax
class sklearn.neighbors.KNeighborsClassifier(*, n_neighbors=5, weights='uniform', algorithm='auto', leaf_size=30, p=2, metric='minkowski', metric_params=None, n_jobs=None)Parameters
| Name | Type | Description |
|---|---|---|
n_neighbors | int | Number of neighbors used by default for kneighbors queries; default 5. |
weights | {'uniform', 'distance'}, callable or None | Prediction weighting: uniform, distance, a callable, or None; default uniform. |
algorithm | {'auto', 'ball_tree', 'kd_tree', 'brute'} | Neighbor-search algorithm; default auto. |
leaf_size | int | Leaf size passed to BallTree or KDTree; default 30. |
p | float | Power parameter for the Minkowski metric; default 2. |
metric | str or callable | Distance metric used for computation; default minkowski. |
metric_params | dict or None | Additional metric arguments; default None. |
n_jobs | int or None | Number of parallel jobs; default None. |
Returns
A fitted KNeighborsClassifier instance; prediction methods return class labels or probability estimates as described on the API page.
Key points
- It is a supervised classifier that assigns labels through a k-nearest-neighbor vote.
- n_neighbors defaults to 5 and determines the neighborhood size used by default.
- weights controls how each neighbor contributes to the predicted class.
- Distance weighting gives closer training points greater influence.
- algorithm selects how nearest neighbors are computed and defaults to auto.
- The default Minkowski metric becomes Euclidean distance when p equals 2.
- leaf_size is passed to tree-based neighbor structures and can affect construction, query speed, and memory.
Common mistakes
- The Minkowski power p must be positive.
- Choose leaf_size experimentally because its best value depends on the problem.
Example: Distance-weighted nearest-neighbor classification
from sklearn.neighbors import KNeighborsClassifier
X = [[0], [1], [2], [3]]
y = [0, 0, 1, 1]
model = KNeighborsClassifier(n_neighbors=3, weights="distance")
model.fit(X, y)
print(model.predict([[1.1]]))Fits a classifier on one-dimensional training data, then predicts the class of a new sample using three distance-weighted neighbors.
When to use it
Distance-weighted prediction
Giving nearby observations stronger influence than farther observations.
Configurable neighbor search
Using tree or brute-force neighbor search according to data and performance needs.
Interview questions
QHow does KNeighborsClassifier make a prediction?
KNeighborsClassifier predicts by applying a vote among the nearest training samples. With uniform weights, every neighbor contributes equally; with distance weights, closer samples contribute more.
Follow-up: How would you make nearby observations matter more? Set weights="distance", which uses inverse-distance weighting.
QWhat does n_neighbors control?
n_neighbors sets the default number of neighbors used for neighbor queries. Increasing it generally includes a broader local region in the vote, while decreasing it makes the decision more local.
Follow-up: What should be validated rather than assumed? Tune n_neighbors using validation data because the appropriate value depends on the dataset.
QHow do metric and p affect neighborhood selection?
The default metric is Minkowski, and p=2 gives the standard Euclidean distance. Setting p=1 produces Manhattan distance, while other positive p values produce other Minkowski distances.
Follow-up: How can computation be selected? algorithm="auto" lets the estimator choose among supported neighbor-search approaches based on the fitted data and parameters.
Related
KNeighborsRegressorRadiusNeighborsClassifierNearestNeighborsBallTreeKDTree