Training Logs

[WangHelin1997]

Hi authors, thanks for the great work!
Is there any log files in the training stage?
I didn't find it.

[kkoutini]

Hi! thank you!
We relay on sacred logging in a MongoDB, for details please see here (there other sacred loggers possible such as file storage)
For example, to store in mongodb_server:27000:audioset21_balanced (server:port:database_name)

 python ex_audioset.py with trainer.precision=16 models.net.arch=passt_deit_bd_p16_384 -p -m mongodb_server:27000:audioset21_balanced -c "PaSST base"

There is also an option to add tensorboard trainer.use_tensorboard_logger (not tested yet)
Example:

 python ex_audioset.py with trainer.precision=16 models.net.arch=passt_deit_bd_p16_384  trainer.use_tensorboard_logger=True -p

We will try to add weights&biases soon.
Let me know if you face any issues

[WangHelin1997]

Hi! Thanks for your help.
I found that MongoDB needs a server which is now difficult for me to set up one (I did not know why).
Instead, I use the following command:

python ex_audioset.py with trainer.precision=16 models.net.arch=passt_deit_bd_p16_384 -p -F My_DIr -c "PaSST base"

And I got these files in My_Dir:

My_dir
  1
    config.json 
    cout.txt�
    info.json
    metrics.json�
    run.json
 _sources
...

Is it right?

Also, I tried:

python ex_audioset.py with trainer.precision=16 models.net.arch=passt_deit_bd_p16_384  trainer.use_tensorboard_logger=True -p

It seems not work (without saved tensorboard files).

[WangHelin1997]

Hi, by using this I can get the summary log file:

python ex_audioset.py with trainer.precision=16 models.net.arch=passt_deit_bd_p16_384  trainer.use_tensorboard_logger=True -p

Thanks.