Many warnings during training

[KinWaiCheuk]

During training, I get tons of warning for Sequential, ConvStack, LSTM and etc.
The warning message is like this

"type " + obj.name + ". It won't be checked "
/opt/conda/envs/onset_model/lib/python3.6/site-packages/torch/serialization.py:292: UserWarning: Couldn't retrieve source code for container of type LSTM. It won't be checked for correctness upon loading.

Is it normal?

I created a new conda environment to run this code and installed all the dependencies via requirements.txt.

[jongwook]

I haven't seen those errors, but only similar ones when I load a model from pickled checkpoints. I suspect for some reasons Python is not loading the source code of pytorch (Sequential and LSTM) and onsets-and-frames (ConvStack) from your file system.

It should be harmless though. Source code (accessed via inspect.getsourcefile) is only used for safety check.

[KinWaiCheuk]

As for the training, we train on 500,000 iterations, and each iteration contains 8 spectrograms. Am I right?

On my system, it shows that 70 hours are required to finish 500,000 iterations. Is it a reasonable speed? Or is it too slow?

I am asking the warnings because I am not sure if the training time is affected by it or not.

[jongwook]

The speed looks fine. It took about a week for me to train for 1M iterations on a single 1080 Ti.