use int64 steps, check for NaN actions

[chriselion]

Proposed change(s)

• Use int64 for tensorflow policy steps, to avoid int32 overflow after 2**31 steps.
• Also use int64 tensor for torch policy steps. The int32 overflow problem doesn't exist in torch since it use float tensors by default. But if setting the global step to a large number and then call get_current_step() where the step count is converted to int, the conversion will cause numerical error and the result will not be as expected.
• Check for NaN actions before they get sent back to Unity. Currently, we only check the observations, but if we send NaN actions, we'll very likely get NaN observations and "blame" the observations.
• After this change, old Tensorflow model will become incompatible and cannot be loaded by new code. Torch is not affected.

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

• Bug fix
• New feature
• Code refactor
• Breaking change
• Documentation update
• Other (please describe)

Checklist

• Added tests that prove my fix is effective or that my feature works
• Updated the changelog (if applicable)
• Updated the documentation (if applicable)
• Updated the migration guide (if applicable)

Other comments

[chriselion]

There's a few ways to do this; we were using np.dot for observations at one point:

ml-agents/ml-agents-envs/mlagents/envs/brain.py

Line 245 in 81acfaf

d = np.dot(np_obs, np_obs)

but that morphed to np.mean during some refactoring:
https://github.com/Unity-Technologies/ml-agents/pull/3022/files#diff-614d11115c4d74319295ea34b44a3b7d8ab40b43865c1c3e351a8f6c9602c673R92

np.sum is one of the recommended approaches in the stackoverflow thread.

[chriselion]

Just raising for now; there's not much the user can do about it. We could use np.nan_to_num but that would probably trigger every step and give bad results.

[vincentpierre]

Do not forget to cherry pick on release_10 if you want these changes in the next release