Improve loss overflow logs

[Quentin-Anthony]

This is a small QoL PR that resolves two small issues:

1. The loss overflow code is duplicated across Zero-3 and Zero-1/2
2. The loss overflow log messages don't handle fp16 hysteresis, leading to confusing messages where the loss scale doesn't change such as those reported in [loss OVERFLOW] Several Issues #1599:

[deepscale] OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 65536

With this PR, loss overflow logs instead look like:

...
OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 3. Reducing hysteresis to 2
OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, but hysteresis is 2. Reducing hysteresis to 1
OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 65536, reducing to 32768
OVERFLOW! Rank 0 Skipping step. Attempted loss scale: 32768, reducing to 16384
...

Fixes: #1599

@stas00 @jeffra @tjruwase

[stas00]

Thank you for adding these improvements, @Quentin-Anthony!

You can add:

Fixes: https://github.com/microsoft/DeepSpeed/issues/1599

to your OP and it'll automatically close that issue on merge.

[stas00]

I'm looking into the breakage on HF side

[stas00]

ok, fixed on HF Transformers side, @Quentin-Anthony could you please run:

git commit --allow-empty -m "Trigger CI"
git push

to re-run the CI? Thank you!

[stas00]

excellent. thank you for re-testing, @Quentin-Anthony!

the amd CI job seems to fail everywhere.