Manifoldron: Direct Space Partition via Manifold Discovery

Colab Demo | ArXiv | Datasets
This respository includes implementations on Manifoldron: Direct Space Partition via Manifold Discovery in which we propose a new type of machine learning models referred to as Manifoldron that directly derives decision boundaries from data and partitions the space via manifold structure discovery. The experimental results on 9 small and 11 large datasets demonstrate that the proposed Manifoldron performs competitively compared to the mainstream machine learning models.

       

Fig. 1 The key steps and the pipeline of the Manifoldron.

Pre-requisites:

• Windows (runned on windows 10, can also run on Ubuntu with the required packages)
• Intell CPU (runned on 12 cores i7-8700 CPU @ 3.20GHZ)
• Python=3.7 (Anaconda), numpy=1.18.5, pandas=0.25.3, scikit-learn=0.22.1, scipy=1.3.2, matplotlib=3.1.1.

Folders

classification: this directory contains the implementations on classfication tasks;
regression: this directory contains implementations on simple regression tasks;
fancy_manifoldron: this directory includes implementations on 3D complex manifolds.

Dataset Preparation

All datasets are publicly available from python scikit-learn package, UCI machine learning repository, Kaggle, and Github: circle, glass, ionosphere, iris, moons, parkinsons, seeds, spirals, wine, banknote, breast, chess, drug, letRecog, magic04, nursery, satimage, semeion, tic-tac-toe, and usps5. Most datasets can also be directly obtained from our shared google drive. https://drive.google.com/drive/folders/14VHR8H7ucp0Loob1PS9yrgTtE9Jm0wsK?usp=sharing.
All datasets need to put under the 'classification/data/' folder to run the Manifoldron on specific datasets.

Running Experiments

Classification on fancy manifolds

Please first go to 'fancy_manifoldron/manifold generation/' to generate complex manifolds in matlab, copy the generated dataset to 'fancy_manifoldron/comparision/', then run the .py file for classification.

>> matlab PlotDoubleCurlySpiral.m            # generate complex manifolds
>> copy [generated txt data path] [fancy_manifoldron/comparision/]    # move the generated data to disired path
>> python TwoCurlySpirals_manifold.py        # the manifoldron on complex manifolds

Fig. 2 Complex manifolds.

Tab. 2 Results on complex manifolds.

Classification

As a demo, the below scripts show how different versions of the Manifoldron run on tic-tac-toe data.

>> python manifoldron_base.py       # the base manifoldron
>> python manifoldron_bagging.py    # the manifoldron with feature bagging
>> python manifoldron_parallel.py   # the manifoldron with parallel computation

If you would like to run the Manifoldron on other representative classification datasets, please go to 'classification/' folder and run cooresponding .py file. We also provide a classification demonstration on the 2D spiral dataset: https://colab.research.google.com/drive/1fK4OD27BYvmVdOorjartuf1CcaQe5PPK?usp=sharing

Experiment Results

Tab. 1 Classification results of the Manifoldron and its competitors.

Regression

Please go to 'regression/' folder and then run cooresponding .py file to run the manifoldron as regressor.

>> python regressor_function1.py       # the manifoldron regressor.