Can't use tch-rs in Jupyter Hub (Part 2)

[sergree]

Hello! Thank you for your help with #350.

But i'm still have some weird problem.
Reminder from the #350:

I have a problem with evcxr.
I can't use tch-rs with it.

My enviroment is: Jupyter Hub, I'm not an administrator, just regular user.
I can't set global env vars for my user, because my provider uses DockerSpawner (.bashrc / .profile exports do not work).

Current problem is, if i run this cell (i'm trying to set LD_LIBRARY_PATH in both ways):

:build_env LD_LIBRARY_PATH=/home/jovyan/libtorch/lib:/home/jovyan/libtorch/lib
std::env::set_var("LD_LIBRARY_PATH", "/home/jovyan/libtorch/lib:/home/jovyan/libtorch/lib");
my_library_that_uses_tch_rs::train_with_cuda(0)

(I have correct Libtorch in /home/jovyan/libtorch)
(https://github.com/LaurentMazare/tch-rs#error-loading-shared-libraries)

It still gives me this error:

While processing instruction `Ok("LOAD_AND_RUN /tmp/.tmpYj5rvn/target/x86_64-unknown-linux-gnu/debug/deps/libcode_12.so run_user_code_5")`, got error: Message("libtorch_cpu.so: cannot open shared object file: No such file or directory")
Subprocess terminated with status: exit status: 99

And if i call this cell next:

std::env::var("LD_LIBRARY_PATH")

It gives me another weird value (that comes from the enviroment that i can't control):

Ok("/usr/local/nvidia/lib:/usr/local/nvidia/lib64")

Any ideas? Is it possible to fix it? Thank you for your time! 🤝

[bjorn3]

Looks like glibc only reads LD_LIBRARY_PATH at startup: https://github.com/bminor/glibc/blob/780c33920281c5d5cc42a1e578bdc9218e675405/elf/dl-support.c#L300 As such we did need to introduce another command which sets an env var for the repl process and also make sure the repl process gets restarted and thus lose all defined variables.

[davidlattimore]

I've added :env and :restart commands that when used together should let you set LD_LIBRARY_PATH when the subprocess starts

[sergree]

It works! Thank you @davidlattimore