lightML.jl

lightML.jl is a collection of reimplementation of general machine learning algorithm in Julia. I will update it for learning purpose.

Why?

This project is targeting people who want to learn internals of ml algorithms or implement them from scratch. The code is much easier to follow than the optimized libraries and easier to play with. All algorithms are implemented in Julia.

To-do List

• Support vector machine(SVM)
• Linear regression
• Logistic regression
• Lasso regression
• Ridge regression
• spectral clustering
• K-Nearst neighbors
• K-Means
• Naive bayes
• Neural network
• Hidden Markov Model
• Gaussian discriminant analysis
• Gaussian mixture model
• Decision tree
• Random Forests
• AdaBoost
• Gradient Boosting trees
• PCA
• SVD
• LDA
• Factorization machines
• Restricted Boltzmann machine
• t-Distributed Stochastic Neighbor Embedding

API

SVM example

include("src/MLsvm.jl")
X, y = dat.make_classification(n_samples=1200, n_features=10,n_informative=5,random_state=1111,n_classes=2, class_sep=1.75,)

# Convert y to {-1, 1}
y = (y * 2) - 1
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8,
                                                        rand_seed=1111)

for kernel in ["linear", "rbf"]
    model = svm(X_train, y_train, max_iter=500, kernel=kernel, C=0.6)
    train(model)
    predictions = predict(model,X_test)
    println("Classification accuracy $(kernel): $(accuracy(y_test, predictions))")
end

kmeans

function test_kmeans_random()
    X, y = make_blo()
    clu = length(unique(y))
    @show clu
    model = Kmeans(k=clu,init="random")
    train!(model,X)
    predict!(model)
    plot_!(model)
end

LDA

function test_LDA()
    X_train, X_test, y_train, y_test = make_cla()
    model = LDA()
    plot_in_2d(model, X_train, y_train)
end