Can't use tch-rs in Jupyter Hub

[sergree]

Hello. I have a problem with evcxr.
I can't use tch-rs with it.

My enviroment is: Jupyter Hub, I'm not an administrator, just regular user.
I can't set global env vars for my user, because my provider uses DockerSpawner (.bashrc / .profile exports do not work).

So

1. At first, i'm trying to set local env var LIBTORCH_USE_PYTORCH=1 to use LibTorch from local PyTorch installation in evcxr notebook :

std::env::set_var("LIBTORCH_USE_PYTORCH", "1");

2. I'm verifying it in the next cell:

std::env::var("LIBTORCH_USE_PYTORCH")
// It prints: Ok("1")

3. Next i'm adding my local library using dep in the next cell, it compiles my code (takes few minutes):

:dep my_awesome_lib = { path = "my_awesome_lib" }

4. My env var is still here:

std::env::var("LIBTORCH_USE_PYTORCH")
// Still prints: Ok("1")

5. Next i'm calling my lib training code that uses tch-rs and it immidiately shows me this error:

While processing instruction `Ok("LOAD_AND_RUN /tmp/.tmpA9iuWP/target/x86_64-unknown-linux-gnu/debug/deps/libcode_6.so run_user_code_5")`, got error: Message("libtorch_cpu.so: cannot open shared object file: No such file or directory")
Subprocess terminated with status: exit status: 99

6. I know that it can be fixed by setting "LIBTORCH_USE_PYTORCH" env var to "1", i already set it in (1) but if i check it again:

std::env::var("LIBTORCH_USE_PYTORCH")
// !!! IT PRINTS Err(NotPresent) NOW

Could you please tell me how to solve this problem? Is that possible? Thank you.

[bjorn3]

I think the missing LIBTORCH_USE_PYTORCH at the end is the result of Subprocess terminated with status: exit status: 99 and a subsequent restart of the repl process with a fresh environment.

As for the libtorch_cpu.so: cannot open shared object file: No such file or directory, it looks like the build script of tch-rs reads LIBTORCH_USE_PYTORCH an determines which library to have the program link against based on this: https://github.com/LaurentMazare/tch-rs/blob/dca12b647ade36414c09359bade840d5092087d6/torch-sys/build.rs#L206 std::env::set_var("LIBTORCH_USE_PYTORCH", "1"); only affects the repl process itself, not the cargo invocation used to build your program and it's dependencies. To set that env var for cargo you need to start the evcxr jupyter kernel itself with this env var set, which as you say is not possible for you. @davidlattimore maybe we can have a :build_env command for setting env vars for the cargo invocation?

[davidlattimore]

Seems like a reasonable option to have. I've pushed a change to github that adds the :build_env command.

[sergree]

Thank you very much for your help! 🙏🙏🙏