Typo in the code and other questions

[Adam-fei]

Hi, thank you for your open source. I have some questions during reproduce the paper:

1. Typo in the code
   ./models/utt_fusion_model.py line18, 'lexical' should be 'visual'
   ./data/multimodal_dataset.py line70, 'proveee' should be 'process'

2. Some small questions:

   • why you choose BaseModel(ABC) as the parents class instead of nn.Module?
     some convenient method like model.to(device) and model.parameters() cannot be used.
     (In the code you used for-loop to apply these method to each layer, but it seems too complex)
   • In ./scripts, I did not find the file to train the baseline model with augmented data.
     But there is a ./data_aug.sh which run the train_miss model with augmented data. Is this a miswritting?

Thx!

[FatDs-lrc]

Hi, thank you for your open source. I have some questions during reproduce the paper:

1. Typo in the code
   ./models/utt_fusion_model.py line18, 'lexical' should be 'visual'
   ./data/multimodal_dataset.py line70, 'proveee' should be 'process'

2. Some small questions:

   • why you choose BaseModel(ABC) as the parents class instead of nn.Module?
     some convenient method like model.to(device) and model.parameters() cannot be used.
     (In the code you used for-loop to apply these method to each layer, but it seems too complex)
   • In ./scripts, I did not find the file to train the baseline model with augmented data.
     But there is a ./data_aug.sh which run the train_miss model with augmented data. Is this a miswritting?

Thx!

Thanks for your attention! I'm one of the authors of this work.
For your question:

1. Thanks! They are typo errors in the project, we will fix it soon.
2. (1) BaseModel is the class which defines the training process, which includes loading a batch of data from dataloader, the forward caculation of the networks(which is nn.Module) as well as the backpropagation, maybe the name "Trainer" or "pipeline" could fit it better. Indeed, the name "Model" could be misleading, we will change the naming strategy of this code structure in the future work.
   (2) The "scripts/data_aug.sh" is the training script of augmented baseline, "train_miss.py" is the scripts that evaluate model on missing modality data.

[Adam-fei]

Thanks for your attention! I'm one of the authors of this work.
For your question:

1. Thanks! They are typo errors in the project, we will fix it soon.
2. (1) BaseModel is the class which defines the training process, which includes loading a batch of data from dataloader, the forward caculation of the networks(which is nn.Module) as well as the backpropagation, maybe the name "Trainer" or "pipeline" could fit it better. Indeed, the name "Model" could be misleading, we will change the naming strategy of this code structure in the future work.
   (2) The "scripts/data_aug.sh" is the training script of augmented baseline, "train_miss.py" is the scripts that evaluate model on missing modality data.

Thank you for your kindly reply :)

As for the "scripts/data_aug.sh", the command in this file is "python train_miss.py" but as my understanding, the baseline training should be "python train_baseline.py" ?

Secondly, I have a question about the "eval" in file "train_miss.py". Line45 the result of "AVL" is evaluated. But in "data/multimodal_miss_dataset.py" line74 the related code is commented out and the list in line76 did not contain "AVL" also. Should this be fixed?