In this repo, selected Machine Learning Algorithms in Python language are presented. They are developed by myself mostly in 2015. For more info, please feel free to contact me on miticm@gmail.com.
In particular, the developed algorithms include:
-
Linear Regression:
- Univariate and Multivariate Batch Gradient Descent;
- Stochastic Gradient Descent;
- Normal Equation for multivariate regression.
-
Logistic/Softmax Regression:
- Unregularized and Regularized Logistic Regression optimized by BFGS algorithm;
- Softmax (Multiclass Logistic) Regression on MNIST dataset optimized by BFGS algorithm.
-
Neural Networks:
- Neural Network trained using Backpropagation Algorithm;
- Neural Network trained using Backpropagation Algorithm and mini-batch Stochastic Gradient Descent;
- Neural Network trained using Backpropagation Algorithm and mini-batch Stochastic Gradient Descent with Momentum.
-
Clustering:
- KMeans Algorithm for IRIS dataset (3 clusters);
- General KMeans Algorithm (tested on IRIS, WINE and SEGMENT dataset).
-
Evolutionary Algorithms:
- Accelerated Particle Swarm Optimization;
- Ant Colony Optmization, early work, developed back in 2014;
- Firefly Algorithm, also developed in 2014.
The codes are pretty much self-explanatory, so the comments are not given in the huge amount. Please double check the directory in which you save the files. If necessary, modify the code so that appropriate path to the directory is included. Together with Python codes, I provided a datasets on which a particular algorithm was tested (you should put them in the same directory where the codes are). And I would definitely appreciate if you drop me an e-mail in case you find bugs or have a suggestion on how to improve anything. :)
Personal note: These algorithms are developed for the purpose of learning and improvement of my Python skills. I have never studied programming, so everything I learned is a product of persistence, perseverance, and huge motivation.
Copyright (c) by Dr. Marko Mitic under the MIT Licence.