Python implementation of deep forest method : gcForest
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is even with pylablanche:master.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
GCForest.py version and status correction Sep 19, 2017
LICENSE License updated Mar 15, 2017
README.md README Sep 18, 2017
gcForest_tuto.ipynb notebook compatibility checked on 19th Sep 2017 Sep 19, 2017
requirements.txt requirements.txt Mar 14, 2017

README.md

Deep Forest in Python

Status : not under active development

What's New

version 0.1.6 : corrected max_features=1 for the completely random forest (correction thanks to sevenguin).
version 0.1.5 : remove layer when accuracy gets worse (behavior corrected thanks to felixwzh).
version 0.1.4 : faster slicing method.

Presentation

gcForest is a deep forest algorithm suggested in Zhou and Feng 2017 ( https://arxiv.org/abs/1702.08835 ). It uses a multi-grain scanning approach for data slicing and a cascade structure of multiple random forests layers (see paper for details).

The present gcForest implementation has been first developed as a Classifier and designed such that the multi-grain scanning module and the cascade structure can be used separately. During development I've paid special attention to write the code in the way that future parallelization should be pretty straightforward to implement.

You can find the official release of the code used in Zhou and Feng 2017 here.

Prerequisites

The present code has been developed under python3.x. You will need to have the following installed on your computer to make it work :

  • Python 3.x
  • Numpy >= 1.12.0
  • Scikit-learn >= 0.18.1
  • jupyter >= 1.0.0 (only useful to run the tuto notebook)

You can install all of them using pip install :

$ pip3 install -r requirements.txt

Using gcForest

The syntax uses the scikit learn style with a .fit() function to train the algorithm and a .predict() function to predict new values class. You can find two examples in the jupyter notebook included in the repository.

from GCForest import *
gcf = gcForest( **kwargs )
gcf.fit(X_train, y_train)
gcf.predict(X_test)

Saving and Loading Models

Using sklearn.externals.joblib you can easily save your model to disk and load it later. Just proceed as follow :
To save :

from sklearn.externals import joblib
joblib.dump(gcf, 'name_of_file.sav')

To load :

joblib.load('name_of_file.sav')

Notes

I wrote the code from scratch in two days and even though I have tested it on several cases I cannot certify that it is a 100% bug free obviously. Feel free to test it and send me your feedback about any improvement and/or modification!

Known Issues

Memory comsuption when slicing data There is now a short naive calculation illustrating the issue in the notebook. So far the input data slicing is done all in a single step to train the Random Forest for the Multi-Grain Scanning. The problem is that it might requires a lot of memory depending on the size of the data set and the number of slices asked resulting in memory crashes (at least on my Intel Core 2 Duo).
The memory consumption when slicing data is more complicated than it seems. A StackOverflow related post can be found here. The main problem is the non-contiguous aspect of the sliced array with the original data forcing a copy to be made in memory.

OOB score error During the Random Forests training the Out-Of-Bag (OOB) technique is used for the prediction probabilities. It was found that this technique can sometimes raises an error when one or several samples is/are used for all trees training.
A potential solution consists in using cross validation instead of OOB score although it slows down the training. Anyway, simply increasing the number of trees and re-running the training (and crossing fingers) is often enough.

Built With

  • PyCharm community edition
  • memory_profiler library

License

This project is licensed under the MIT License (see LICENSE for details)

Early Results

(will be updated as new results come out)

  • Scikit-learn handwritten digits classification :
    training time ~ 5min
    accuracy ~ 98%