Given a human-generated pipeline (HI-pipeline) for an ML task, HAIPipe introduces a reinforcement learning based approach to search an optimized ML-generated pipeline (ML-pipeline) and adopts an enumeration-sampling strategy to carefully select the best performing combined pipeline(HAI-pipeline).
This code is written in Python. Requirements include
- python = 3.8.12
- NumPy = 1.19.5
- pandas = 1.1.3
- torch = 1.9.0
- Scikit-learn = 0.23.2
You can install multiple packages:
pip install -r requirements.txt
python example.py
The file example.py is an example. Modify it according to your configuration.
- The input contains 1 notebook, 1 CSV file, some information about the ML task (including model and label_index)
- The output is the best HAI-program, the accuracy of HI-pipeline and the accuracy of HAI-pipeline.
You can put new data in the folder $hybridpipe/data/and organize them as:
hybridpipe
├── data
│ ├── dataset
│ │ └── new_folder #input dataset(csv file)
│ └── notebook #input notebook
├── MLPipeGen
└── HybridPipeGenThen modify the input in example.py.
The whole dataset and notebooks are put in the follow link: https://www.dropbox.com/s/reqenlqzggpx8bk/haipipe_datasets.zip?dl=0