Empty ZeRO3 partition cache

[tjruwase]

API to free GPU memory consumed by zero3 partition cache.
Fix #3025

[stas00]

oh, sorry, could this new method be added to the API docs please? Thank you, Tunji!

Perhaps something like:

By default at the end of the training some parameters will remain unpartitioned and use up some gpu memory. This is done on purpose as an optimization should you resume training again. If you'd like to clear out the cached parameters that use up gpu memory, you can call:

deepspeed_engine.empty_partition_cache()

as soon as the training has finished.

[stas00]

Thank you for adding the doc - looks great, Tunji!