fix #1751

[qmpzzpmq]

register ctc loss_fn as float function
@ShigekiKarita

[ShigekiKarita]

I have already fix this issue in #1777 for float16 training (not only for apex.amp). However, notifying apex.amp for ignoring CTC also looks good to me.

[ShigekiKarita]

I found that apex.amp already register torch.ctc_loss as FP32_FUNCS. Can you check your local apex already done this?
https://github.com/NVIDIA/apex/blob/ca00adace51ae4d82d2ff5cfc1ef1c9174ff6aaa/apex/amp/lists/functional_overrides.py#L67

[qmpzzpmq]

Yes, but I found only register the torch.ctc is not enough, there still some error, You can check it by your self.

[ShigekiKarita]

OK. Please ignore circle CI errors. I think they are not related to this PR.

[ShigekiKarita]

Thanks!

[kamo-naoyuki]

amp.init() is old api.
https://nvidia.github.io/apex/amp.html#for-users-of-the-old-amp-api

and register_float_function should be done before initialize, but is amp.register_float_function really necessary? It seems meaningless if casting.

[kamo-naoyuki]

@qmpzzpmq Please fix.