Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RLlib]: Fix torch None conversion #26863

Merged
merged 1 commit into from
Jul 23, 2022
Merged

Conversation

Rohan138
Copy link
Contributor

@Rohan138 Rohan138 commented Jul 22, 2022

PyTorch does not support dtype=object anymore, which is what np.asarray(None) returns. This breaks save_checkpoint and load_checkpoint for algorithms with pytorch >= 1.12.0.

Repro script:

from ray.rllib.algorithms.dqn import DQNConfig
import os
import shutil

checkpoint_dir = "/tmp/cartpole/torch/"
shutil.rmtree(checkpoint_dir, ignore_errors=True)
os.makedirs(checkpoint_dir, exist_ok=True)
config = (
    DQNConfig()
    .environment(env="CartPole-v0")
    .framework("torch")
    .rollouts(num_rollout_workers=4)
)
algo = config.build()
random = algo.save_checkpoint(checkpoint_dir)
algo.load_checkpoint(random)

Related issue number

Checks

  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@avnishn
Copy link
Member

avnishn commented Jul 22, 2022

This is fine with me pending tests passing. Thanks for submitting this!

@sven1977 sven1977 merged commit 97bcf38 into ray-project:master Jul 23, 2022
yaxife pushed a commit to alipay/ant-ray that referenced this pull request Jul 26, 2022
…h_tensor`. (ray-project#26863)

Signed-off-by: nanqi.yxf <nanqi.yxf@antgroup.com>
klwuibm pushed a commit to yuanchi2807/ray that referenced this pull request Jul 27, 2022
Catch-Bull pushed a commit to alipay/ant-ray that referenced this pull request Jul 27, 2022
…h_tensor`. (ray-project#26863)

Signed-off-by: Catch-Bull <burglarralgrub@gmail.com>
franklsf95 pushed a commit to franklsf95/ray that referenced this pull request Aug 2, 2022
…h_tensor`. (ray-project#26863)

Signed-off-by: Frank Luan <lsf@berkeley.edu>
gramhagen pushed a commit to gramhagen/ray that referenced this pull request Aug 15, 2022
…h_tensor`. (ray-project#26863)

Signed-off-by: Scott Graham <scgraham@microsoft.com>
gramhagen pushed a commit to gramhagen/ray that referenced this pull request Aug 15, 2022
Stefan-1313 pushed a commit to Stefan-1313/ray_mod that referenced this pull request Aug 18, 2022
…h_tensor`. (ray-project#26863)

Signed-off-by: Stefan van der Kleij <s.vanderkleij@viroteq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants