Machine Learning Algorithms from Scratch

This repository contains various links for repo for implementations of machine learning algorithms from scratch using Python. These implementations are designed to demonstrate the core principles and workings of each algorithm without relying on specialized libraries. Each project is implemented in a Jupyter Notebook for easy understanding and experimentation.

Projects Included

1. Linear Regression from Scratch with Gradient Descent

This project demonstrates the implementation of linear regression using gradient descent. It covers the basic principles of linear regression, cost functions, and gradient descent optimization techniques.

2. Multivariate Regression from Scratch with Gradient Descent

This project extends linear regression to handle multiple features (multivariate regression). It includes the implementation of gradient descent for optimizing the regression model with multiple predictors.

3. Regression Trees from Scratch

This project implements regression trees from scratch. It covers the creation of regression trees, splitting criteria, and the recursive building of the tree structure.

4. KMeans from Scratch

This project implements the KMeans clustering algorithm from scratch using Python. The notebook demonstrates how to perform clustering on synthetic data generated using the make_blobs function from Scikit-learn.

5. Image Segmentation using KMeans

This project demonstrates image segmentation using the KMeans clustering algorithm. It includes steps for preprocessing the image, applying KMeans clustering, and visualizing the segmented image.

6. Face Recognition from Scratch with KNN

This project showcases a face recognition system implemented from scratch using the K-Nearest Neighbors (KNN) algorithm. It includes steps for capturing and recognizing faces using a webcam.

7. Naive Bayes from Scratch

This project implements the Naive Bayes classifier from scratch. It covers the principles of Naive Bayes classification, including prior and conditional probabilities, and evaluation of the model.

8. Decision Trees from Scratch

This project implements Decision Trees from scratch. It involves building a Decision Tree classifier, calculating entropy and information gain for splitting nodes, and making predictions based on tree traversal.

Requirements

Each project requires Python and several standard libraries such as NumPy, pandas, and Matplotlib. Install the required packages using:

pip install numpy pandas matplotlib

Feel free to explore and experiment with the implementations. If you have any questions or suggestions, please open an issue or submit a pull request.