Mixed Precision Training

[Zadagu]

Hi,

first of all I want to thank you for your great work. I'm using SRUs for speech enhancement, they do very well on a reasonable computational cost.

I would like to know if there is a possibility to train SRUs in mixed precision mode?
I tried to enable it, by setting precision=16 in the pytorch lightning trainer, but that didn't do the trick.

Kind of regards,
Zadagu

[taolei87]

@Zadagu SRU / SRU++ can work with pytorch native mixed precision training.

See this for example:
https://github.com/asappresearch/sru/blob/3.0.0-dev/experiments/srupp_experiments/train_enwik8.py#L250