Statistical Machine Learning Basics

This repository contains implementation of classifiers and linear regression methods. These codes were written for Statistical Machine Learning class (STAT 775) offered by Prof. Raul Rojas in Summer of 2016.

Classifiers:

1. Fisher Discriminant
2. Perceptrons
3. Neural Network (single hidden layer). For math and conventions used in the code see book [2] section 7.3,7.2

Regressions:

1. Linear regression based on ordinary least squares (OLS)
2. Subset selection method for linear regression
3. Logistic Regression
4. Ridge Regression

Dependencies:

1. numpy for data structure,
2. scikit-learn for computing error metrics and pca, and
3. matplotlib for data visualization.

The dataset for training and testing can be found in [1].

Reference

1. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. T Hastie, R Tibshirani, Jerome Friedman
2. Neural Networks, A Systematic Introduction. R. Rojas