Amalgamating Multi-Task Models with Heterogeneous Architectures (AmalMTH)

This is the implementation repository of Versatile Common Feature Consolidator (VENUS), the solution for AmalMTH, AAAI 2024.

File listing

• main.py : Code for training VENUS
• model.py : Supporting models
• utils.py : Supporting utility functions
• requirements.txt : Library requirements

Instructions

Prepared folders:

• data : directory to place training data
• teachers : directory to place teacher models
• output : directory for student model as outputs

The datasets and the teachers we use in our paper are available here: https://drive.google.com/drive/folders/1ShzRZF2ARXTnyfEaS-uRdVehX7bt8wlJ?usp=drive_link

Run script as:

python main.py -backbone resnet -tname t0_densenet t1_resnet -t_modelname densenet resnet -data-name data1 -t_tasks '2 0' '2 3 1' -ep 1

Parameters:

• Required:

  • -backbone : the backbone model for the student model e.g., resnet
  • -tname : the name of teacher models e.g., t0_densenet, t1_resnet
  • -t_modelname : the architecture of each teacher e.g., densenet, resnet
  • -t_tasks : teachers’ tasks e.g., t0: '2 0', t1: '2 3 1'
  • -dataname : unlabelled data for training the student

• Hyperparameters:

  • -lr : learning rate, default 0.01
  • -ep : epochs, default 50
  • -bs : batch size, default 16