Skip to content

Conversation

@dongruoping
Copy link
Contributor

Proposed change(s)

  • Add the int overflow/ nan action checks for torch

  • Use int tensor for global_step for torch: the int32 overflow problem doesn't happen in torch since torch use float tensors by default. But there's a subtle bug in this: when setting the global step to a very large number and then call get_current_step() where the step count is converted to int, there's some numerical error and the result will not be as expected, i.e.

policy.set_step(2 ** 31 - 1)
assert policy.get_current_step() == 2 ** 31 - 1  <---this will fail

This could be a potential bug when trying to resume training from a large step in torch.

  • Added tests for int overflow check, verified both TF and torch failed on master and passed with fix in this PR

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

  • Bug fix
  • New feature
  • Code refactor
  • Breaking change
  • Documentation update
  • Other (please describe)

Checklist

  • Added tests that prove my fix is effective or that my feature works
  • Updated the changelog (if applicable)
  • Updated the documentation (if applicable)
  • Updated the migration guide (if applicable)

Other comments

@dongruoping dongruoping merged commit bcc0ba0 into MLA-1503-int-overflow-nan Nov 12, 2020
@delete-merged-branch delete-merged-branch bot deleted the MLA-1503-int-overflow-nan-torch branch November 12, 2020 23:35
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants