ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation

Zihao Tang¹, Zheqi Lv¹, Shengyu Zhang¹，Fei Wu¹, Kun Kuang¹

¹Zhejiang University

Official Pytorch Implementation for the research paper titled "ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation".

Installation

Clone this repository and install the required packages:

git clone https://github.com/IshiKura-a/ModelGPT.git
cd ModelGPT

conda create -n ModelGPT python=3.8
conda activate ModelGPT
conda install pytorch torchvision torchaudio pytorch-cuda=12.0 -c pytorch -c nvidia

pip install -r requirements.txt

Download datasets:

• Office-31
• GLUE Benchmark: already installed by pip requirements
• Tabular Datasets: already installed by pip requirements

Baseline

For baseline, simply run the file in the folder baseline. For example, to run baseline for nlp, run:

python -m baseline.glue

Train

To replicate our results, run main_lora_nlp.py, main_img_cls.py, main_tabular.py for nlp, cv and tabular datasets individually, like:

python main_lora_nlp.py

Hyperparameter settings are embedded into these files. Readers can also refer to Appendix A.

Citation

We warmly welcome any discussion in this emerging field! If you are interested in our work, you can star our project and cite our paper:

@article{tang2024modelgpt,
  title={ModelGPT: Unleashing LLM's Capabilities for Tailored Model Generation},
  author={Tang, Zihao and Lv, Zheqi and Zhang, Shengyu and Wu, Fei and Kuang, Kun},
  journal={arXiv preprint arXiv:2402.12408},
  year={2024}
}