BASIC ML

---

Content

• About the project
• Project structure
• Functionalities that are implemented
• Usage
• What's next
• Supporting developers

---

About the project

Repository holds a library of most common algorithms used manually without using the help of sklearn or other Machine Learning libraries. Only used libraries are:

• built-in libraries that belong to Python
• numpy
• scipy - only used for dendrogram function in scipy.cluster.hierarchy
• plotly
• tqdm - for showing progress of training
• autograd - for derivation of functions
• jupyterlab (as development environment and to call all other scripts)

Those can be installed via:

pip install numpy scipy plotly tqdm autograd jupyterlab

---

Project structure

The project is structured as follows:

• assets/ - containing all graphics that are shown here
• library/ - containing the respective scripts for all subtasks:
  • classic_learning/ - contains all classic Machine Learning algorithms
  • deep_learning/ - contains Deep Learning using Neural Networks
  • reinforcement_learning/ - contains Reinforcement Learning algorithms
  • utils/ - contains different kind of helper functions and classes
• Test.ipynb - a Jupyter Notebook containing a test implementation of all provided functionality

---

Functionalities that are implemented

• classic_learning

  • Clustering Clustering by sklearn

  

  • Dimension Reduction Algorithms (LDA, PCA and ICA) Dimension Reduction by sklearn

  

  • Gaussian Mixture Models with Expectation Maximization Algorithm GMM with EM by sklearn

  

  • Gaussian Processes Regression GP by sklearn

  

  • Linear Regression (with single- and multi-dimensional data support) Linear Regression by sklearn

  

• deep_learning
  • Deep Learning using Neural Networks [containing Convolution-, Pooling-, Dense-, Flatten-, Dropout- and ReLU-Layer] Deep Learning by Tensorflow

  

• reinforcement_learning - see in general: Reinforcement Learning Tutorial by PythonProgramming

  • Action Value Iteration for Reinforcement Learning - Action Value Iteration by Denny Britz & Co
  • Q-Learning for Reinforcement Learning - Q-Learning by Denny Britz & Co

  

  • Genetic Algorithm for DataSet manipulation Genetic algorithm by sklearn

  

  • Hidden Markov Models HMM by sklearn

  

• utils
  • Data Preprocessing for DataSet manipulation (containing a MinMaxScaler and a train_test_split()-function) Preprocessing by sklearn

  

---

Usage

In general all classes and functions can be used exactly as those which are implemented in sklearn with a training(), a predict() and a score() - if possible - function.

Algorithms that work exactly as describe above, respectively in their sklearn documentation:

• Linear Regression -> Regressor
• Clustering -> Classifier
• Dimension Reduction - train(), predict() and score() function are fit(), fit_transform() and transform() respectively
• Gaussian Mixture Models with Expectation Maximization Algorithm -> Classifier
• Gaussian Processes -> Regressor
• Deep Learning using Neural Networks - has its one score() function as being the loss() function in training. Workwise/Usage in the same way as Tensorflow Implementation

Algorithms with different work-wise:

• Reinforcement Learning --> since there is no prediction in the workwise of RL, there is no such function implemented. Furthermore there is no (or not yet) train() function implemented, since the user is obliged to self-decide whether or not to use Q-Learning or Action-Value-Iteration.
• Hidden Markov Models --> as they need a sequence to be trained and initial states and observations, the class is used slightly different to sklearn-typical work wise. You have to provide a sequence to all of the implemented algorithms, further instructions can be found on top of the class description in the hmm.py script.
• Genetic Algorithm for DataSet manipulation --> since there is no prediction in the workwise of GA, there is no such function implemented. The train() function gives back the best 'subdataset' that exists (in the original dataset or mutated from that)

---

What's next

• working on the Deep Learning module
• go through all the algorithms and add further variety

---

Supporting developers

• Alexander Christoph