Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to compile standalone gazelle binary with rules_python plugin #1455

Closed
Whoaa512 opened this issue Oct 3, 2023 · 5 comments · Fixed by #1485
Closed

Unable to compile standalone gazelle binary with rules_python plugin #1455

Whoaa512 opened this issue Oct 3, 2023 · 5 comments · Fixed by #1485

Comments

@Whoaa512
Copy link
Contributor

Whoaa512 commented Oct 3, 2023

Context:
I work within a repo where we require building a static gazelle binary per platform that is checked into git LFS. The precompiled binaries are then used for pre-commit hooks and can be run in CI environments without the need to build from source with bazel.

Issue:
I'm having issues compiling gazelle with the rules_python package because it makes use of runfiles.

The runfiles however is not portable and short of patching the parser.go I don't see a clear path to compiling this gazelle plugin to be a portable go binary.

Is this intentional or could there be an alternative that allows for this?

@aignas
Copy link
Collaborator

aignas commented Oct 4, 2023

The gazelle plugin depends on a python interpreter and a few python files that are executed using the said interpreter, so creating a standalone binary would not be possible.

The fact that gazelle needs to:

  1. Parse the python imports and
  2. Understand what imports are from stdlib

makes it very difficult to achieve without having a Python interpreter.

@aignas
Copy link
Collaborator

aignas commented Oct 4, 2023

Just to add here what has been discussed over Slack, https://github.com/kluctl/go-embed-python seems to be like an interesting way to embed the Python interpreter into a go binary and we could potentially use this project to include our own Python toolchain.

@groodt
Copy link
Collaborator

groodt commented Oct 9, 2023

Possible duplicate of py_binary with hermetic toolchain requires a system interpreter
(or, at very least, related to it)

@aignas
Copy link
Collaborator

aignas commented Oct 15, 2023

#1485 Should fix the single binary part of the issue, but it would still need system python, which needs to be addressed via #691.

github-merge-queue bot pushed a commit that referenced this issue Oct 17, 2023
The runtime dependencies of Gazelle Python extension makes it hard to
distribute Gazelle binaries: we have to preserve the runfiles structure
and distribute it with Gazelle binaries.

Instead, we can build a single Python zip file that comes a built-in
interpreter, and embed the zip file into the Go binary in compile time
and avoid the runtime dependency.

Fixes #1455

---------

Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com>
@Whoaa512
Copy link
Contributor Author

Thank you! This is perfect for my needs. Agree that #691 needs to be addressed but this will remove the need for a genrule where I basically create a runfiles dir next to my pre-compiled gazelle.

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants