Cannot pixi run tools installed in path with spaces

[nchachereau]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

cd ~
mkdir 'Sample Dir'
cd 'Sample Dir'
pixi init
pixi add python
pixi run pydoc

Issue description

When you install a package that puts an executable into (e.g.) .pixi/envs/default/bin, you should be able to run it using pixi run. This works as long as the path does not include any space. If there is a space in the path, the shebang line includes the path as it is, but the interpreter name must not contain blanks.

$ head -n1 .pixi/envs/default/bin/pydoc
#!/home/nico/Sample Dir/.pixi/envs/default/bin/python3.12

On my system (Linux 6.8.0-40-generic, x86_64, Ubuntu 22.04.4 LTS), the reproducible example results in the following error:
Error launching 'pydoc': No such file or directory (os error 2).

That's the gist of it. I have done some research, I hope this could help in solving the problem:

• There is a further strange thing: when the path is long and contains spaces, the shebang is prefixed by /usr/bin/env and followed by a mangled path:

  $ head -n1 code/mes-contributions/pixi/fixing-bugs/trying\ to\ create\ an\ extremely\ long\ path\ is\ not\ so\ easy/bug/.pixi/envs/default/bin/pydoc
  #!/usr/bin/env trying to create an extremely long path is not so easy/bug/.pixi/envs/default/bin/python3.12

  This leads to a slightly different error:

  $ pixi run pydoc
  /usr/bin/env: 'trying to create an extremely long path is not so easy/bug/.pixi/envs/default/bin/python3.12': No such file or directory
  /usr/bin/env: use -[v]S to pass options in shebang lines
  

• Interestingly, when a long path does not include any space, the shebang is different from when the path is short (probably as a way of dealing with the maximum length of this line). When the path is short enough, the shebang is the absolute path of the interpreter (#!/home/user/SampleDir/.pixi/envs/default/bin/python3.12). When it is long, it is prefixed by /usr/bin/env and uses just the filename of the interpreter (#!/usr/bin/env python3.12), which only works when we use pixi run (running the script directly, e.g. .pixi/envs/default/bin/pydoc errors out: /usr/bin/env: 'python3.12': No such file or directory).

• pip has adopted following shell/python hybrid shebang as a solution:

  $ python3 -m venv .venv
  $ head .venv/bin/pip
  #!/bin/sh
  '''exec' "/home/nico/code/mes-contributions/pixi/fixing-bugs/trying to create an extremely long path is not so easy/bug/.venv/bin/python3" "$0" "$@"
  ' '''
  

  (it looks like it does this with any package it installs)

• This bug is related to Can't install pypi-dependencies into path with spaces. #1421, but different: installing works, and you could run the executable manually, e.g.: .pixi/envs/default/bin/python .pixi/envs/default/bin/pydoc. (Obviously I have been using pydoc as a mere minimal example here, it is more relevant for pypi packages that install tools, say black or marimo).

Expected behavior

cd ~
mkdir "Sample Dir"
cd "Sample Dir"
pixi init
pixi add python
pixi run pydoc
# Works

[nchachereau]

Additional notes: naively introducing pip's solution apparently does not work with pixi. One can then run the script directly, but pixi run pydoc errors out: Error launching 'pydoc': Permission denied (os error 13).

I also tried to make a symlink to the directory, without spaces in its path, but the path that gets written into the shebang is the canonical path with symlinks resolved.

[Hofer-Julian]

Thanks for the thorough investigation @nchachereau!

[ruben-arts]

Hi @nchachereau

Thanks for the reproducer! And the tip on what pip does.

I've checked and pixi does a different thing from conda

The shebang in the conda installed binaries are: #!/usr/bin/env python3.12 but in pixi they are #!/home/rarts/Sample Dir/.pixi/envs/default/bin/python3.12. Thus this problem doesn't occur with a conda environment but does with a pixi one.

Asking @baszalmstra and @wolfv for their knowledge on the implementation details.

[nchachereau]

Thanks for the news. The conda solution would seem appropriate for pixi, as it already works like that when the path is long but without spaces.

[wolfv]

Great. Yes, @ruben-arts you identified the issue correctly. I'll take a look at our shebang replacement logic in rattler. Should be fixed today!

[nchachereau]

I can confirm it works now, thank you!