Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot pixi run tools installed in path with spaces #1990

Closed
2 tasks done
nchachereau opened this issue Sep 5, 2024 · 6 comments
Closed
2 tasks done

Cannot pixi run tools installed in path with spaces #1990

nchachereau opened this issue Sep 5, 2024 · 6 comments
Labels
🐞 bug Something isn't working

Comments

@nchachereau
Copy link

nchachereau commented Sep 5, 2024

Checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

cd ~
mkdir 'Sample Dir'
cd 'Sample Dir'
pixi init
pixi add python
pixi run pydoc

Issue description

When you install a package that puts an executable into (e.g.) .pixi/envs/default/bin, you should be able to run it using pixi run. This works as long as the path does not include any space. If there is a space in the path, the shebang line includes the path as it is, but the interpreter name must not contain blanks.

$ head -n1 .pixi/envs/default/bin/pydoc
#!/home/nico/Sample Dir/.pixi/envs/default/bin/python3.12

On my system (Linux 6.8.0-40-generic, x86_64, Ubuntu 22.04.4 LTS), the reproducible example results in the following error:
Error launching 'pydoc': No such file or directory (os error 2).

That's the gist of it. I have done some research, I hope this could help in solving the problem:

  • There is a further strange thing: when the path is long and contains spaces, the shebang is prefixed by /usr/bin/env and followed by a mangled path:

    $ head -n1 code/mes-contributions/pixi/fixing-bugs/trying\ to\ create\ an\ extremely\ long\ path\ is\ not\ so\ easy/bug/.pixi/envs/default/bin/pydoc
    #!/usr/bin/env trying to create an extremely long path is not so easy/bug/.pixi/envs/default/bin/python3.12

    This leads to a slightly different error:

    $ pixi run pydoc
    /usr/bin/env: 'trying to create an extremely long path is not so easy/bug/.pixi/envs/default/bin/python3.12': No such file or directory
    /usr/bin/env: use -[v]S to pass options in shebang lines
    
  • Interestingly, when a long path does not include any space, the shebang is different from when the path is short (probably as a way of dealing with the maximum length of this line). When the path is short enough, the shebang is the absolute path of the interpreter (#!/home/user/SampleDir/.pixi/envs/default/bin/python3.12). When it is long, it is prefixed by /usr/bin/env and uses just the filename of the interpreter (#!/usr/bin/env python3.12), which only works when we use pixi run (running the script directly, e.g. .pixi/envs/default/bin/pydoc errors out: /usr/bin/env: 'python3.12': No such file or directory).

  • pip has adopted following shell/python hybrid shebang as a solution:

    $ python3 -m venv .venv
    $ head .venv/bin/pip
    #!/bin/sh
    '''exec' "/home/nico/code/mes-contributions/pixi/fixing-bugs/trying to create an extremely long path is not so easy/bug/.venv/bin/python3" "$0" "$@"
    ' '''
    

    (it looks like it does this with any package it installs)

  • This bug is related to Can't install pypi-dependencies into path with spaces. #1421, but different: installing works, and you could run the executable manually, e.g.: .pixi/envs/default/bin/python .pixi/envs/default/bin/pydoc. (Obviously I have been using pydoc as a mere minimal example here, it is more relevant for pypi packages that install tools, say black or marimo).

Expected behavior

cd ~
mkdir "Sample Dir"
cd "Sample Dir"
pixi init
pixi add python
pixi run pydoc
# Works
@nchachereau nchachereau added the 🐞 bug Something isn't working label Sep 5, 2024
@nchachereau
Copy link
Author

nchachereau commented Sep 5, 2024

Additional notes: naively introducing pip's solution apparently does not work with pixi. One can then run the script directly, but pixi run pydoc errors out: Error launching 'pydoc': Permission denied (os error 13).

I also tried to make a symlink to the directory, without spaces in its path, but the path that gets written into the shebang is the canonical path with symlinks resolved.

@Hofer-Julian
Copy link
Contributor

Thanks for the thorough investigation @nchachereau!

@ruben-arts
Copy link
Contributor

Hi @nchachereau

Thanks for the reproducer! And the tip on what pip does.

I've checked and pixi does a different thing from conda

The shebang in the conda installed binaries are: #!/usr/bin/env python3.12 but in pixi they are #!/home/rarts/Sample Dir/.pixi/envs/default/bin/python3.12. Thus this problem doesn't occur with a conda environment but does with a pixi one.

Asking @baszalmstra and @wolfv for their knowledge on the implementation details.

@nchachereau
Copy link
Author

Thanks for the news. The conda solution would seem appropriate for pixi, as it already works like that when the path is long but without spaces.

@wolfv
Copy link
Member

wolfv commented Oct 2, 2024

Great. Yes, @ruben-arts you identified the issue correctly. I'll take a look at our shebang replacement logic in rattler. Should be fixed today!

@nchachereau
Copy link
Author

I can confirm it works now, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants