[BUG] Running into Failed to import torchrl C++ binaries for local install

[jborbik]

Describe the bug

I tried installing both tensordict and torchrl from latest master and v0.5.0 and I am running into Failed to import torchrl C++ binaries

To Reproduce

I am installing TorchRL with the following script:

#!/usr/bin/env bash
set -Eeuo pipefail

eval "$(conda "shell.$(basename "${SHELL}")" hook)"

# cd to the script location
cd "$(dirname "${BASH_SOURCE[0]}")"

conda create -y --name quantized-offline-rl python=3.10
conda activate quantized-offline-rl

# pytorch
conda install pytorch==2.4.1 torchvision==0.19.1 pytorch-cuda=12.4 -c pytorch -c nvidia -y

# torchrl
pip install -U ninja
pip install cmake
cd submodules/tensordict
python setup.py develop
cd ../rl
python setup.py develop
cd ../..

I have tried with torchrl and tensordict checked out to v0.5.0 and latest master. In both cases if I run:

python -c 'from torchrl.envs.libs.gym import GymEnv'

I get:

/home/jorbik/Projects/project/submodules/rl/torchrl/data/replay_buffers/samplers.py:37: UserWarning: Failed to import torchrl C++ binaries. Some modules (eg, prioritized replay buffers) may not work with your installation. If you installed TorchRL from PyPI, please report the bug on TorchRL github. If you installed TorchRL locally and/or in development mode, check that you have all the required compiling packages.

If I replace the local installation steps with

pip install tensordict==0.5.0
pip install torchrl==0.5.0

the import works without any errors

Expected behavior

No C++ extension errors on import

System info

Describe the characteristic of your environment:

• Describe how the library was installed (pip, source, ...) - in the bug description
• Python version - 3.10
• Versions of any other relevant libraries - pytorch 2.4.1, torchvision 0.19.1

import torchrl, numpy, sys
print(torchrl.__version__, numpy.__version__, sys.version, sys.platform)

/home/jorbik/Projects/project/submodules/rl/torchrl/data/replay_buffers/samplers.py:37: UserWarning: Failed to import torchrl C++ binaries. Some modules (eg, prioritized replay buffers) may not work with your installation. If you installed TorchRL from PyPI, please report the bug on TorchRL github. If you installed TorchRL locally and/or in development mode, check that you have all the required compiling packages.
  warnings.warn(EXTENSION_WARNING)
>>> print(torchrl.__version__, numpy.__version__, sys.version, sys.platform)
0.5.0+9565398 2.0.1 3.10.15 (main, Oct  3 2024, 07:27:34) [GCC 11.2.0] linux

Additional context

I have tried running the code outside of any git repositories as advised in the README

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have read the documentation (required)
• I have provided a minimal working example to reproduce the bug (required)

[carschandler]

I've been getting this as well on torchrl-nightly for Python 3.12.7 (whose current match for {torchrl,tensordict}-nightly is v2024.10.21) for both Linux and Windows. Haven't tried running anything to see if it's actually an issue yet, but seems concerning at the least lol.

[vmoens]

Hello!
I see in the example that you're not doing a git clone - I presume the repo is already there. Have you tried doing python setup.py clean before running the setup.py develop?
I will have a look at the nightlies!

[vmoens]

I've been getting this as well on torchrl-nightly for Python 3.12.7 (whose current match for {torchrl,tensordict}-nightly is v2024.10.21) for both Linux and Windows. Haven't tried running anything to see if it's actually an issue yet, but seems concerning at the least lol.

Are you using pytorch nightlies? Maybe i can clarify the error message - with nightlies, the pytorch installed must also be the latest nightly built

[carschandler]

Are you using pytorch nightlies? Maybe i can clarify the error message - with nightlies, the pytorch installed must also be the latest nightly built

Ah, I'm not running the pytorch nightlies. There was no mention of it being required in the docs, and the dependencies for tensordict-nightly and torchrl-nightly in the pypi index do not specify anything about nightly, so I ended up with 2.5.1.

If they are required, it would be greatly helpful to explicitly state so in all parts of the documentation/README that mention nightly builds.

[vmoens]

Ah, I'm not running the pytorch nightlies. There was no mention of it being required in the docs, and the dependencies for tensordict-nightly and torchrl-nightly in the pypi index do not specify anything about nightly, so I ended up with 2.5.1.

In #2468 I made the warning more explicit - I hope that helps!

Open to edit the doc/README too ofc

[carschandler]

Thanks! I'd love to drop some edits on the README/docs but I’m not super well-versed in the ecosystem yet... I'd like to confirm that the PyTorch nightlies are not available on pypi. If that's the case, then I think it's best to just add a note under the pip instructions for nightly tensordict and torchrl mentioning that the nightly build of PyTorch is also required, then refer users to the PyTorch install page where they can determine how to install it on their own. What do you think?