1. kNN Algorithm
K Nearest Neighbors is a supervised learning algorithm that predicts the new instances by grouping them together with most similar cases. This is also called instance based or memory based supervised learning. These learning methods work by memorizing the labeled examples in the training set and use those memorized examples to classify new objects.
KNN can be used for both classification and regression problems. The kNN classification or regression task can be broken down into 3 primary functions
- Calculate the distance between any two points
- Find the nearest neighbors on pairwise distances
- Majority vote on a class labels based on the nearest neighbor list
-
Model Complexity
-
n_neighbors- Number of nearest neighbors (k) to consider. - Default =
5.
-
-
Model Fitting
- Metric - distance between data points.
- Default: Minkowski distance with power parameter = 2 (Euclidean)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=3)
from sklearn.neighbors import KNeighborsRegressor
knn = KNeighborsRegressor(n_neighbors=3)
knn.fit(X_train, y_train)
knn.predict(X_test)
knn.score(X_test, y_test)
- A complete hands on example of implementing knn-algorithm to build an Object Recognition System
- KNN classification example on a synthetic dataset [ipynb] [py]
- KNN Regression Example on a synthetic Dataset [ipynb] [py]