feat(py_console_script_binary)!: entry points with custom dependencies

[aignas]

Add py_console_script_binary, a macro/rule that allows better customization of how
entry points are generated. Notable features of it are:

• It allows passing in additional dependencies, which makes it easier for plugin
  dependencies to be added to tools such as pylint or sphinx.
• The underlying py_binary rule can be passed in, allowing custom rules,
  such as the version-aware rules, to be used for the resulting binary.
• Entry point generation is based upon a wheel's entry_points.txt file. This helps
  avoid loading external repositories unless they're actually used, allows entry
  points to have better version-aware support, and allows bzlmod to provide a
  supportable mechanism for entry points.

Because the expected common use case is an entry point for our pip generated repos,
there is special logic to make that easy and concisely do. Usage of
py_console_script_binary is not tied to our pip code generation, though, and users can
manually specify dependencies if they need to.

BREAKING CHANGE: This is a breaking change, but only for bzlmod users. Note that
bzlmod support is still beta. Bzlmod users will need to replace using entry_point
from requirements.bzl with loading py_console_script_binary and defining the
entry point locally:

load("@rules_python//python/entry_points:py_console_script_binary.bzl, "py_console_script_binary")

py_console_script_binary(name="foo", pkg="@mypip//pylint")

For workspace users, this new macro is available to be used, but the old code is still
present.

Fixes #1362
Fixes #543
Fixes #979
Fixes #1262
Closes #980
Closes #1294
Closes #1055

[aignas]

Ideally, I would love to reuse the implementation for py_binary and have entry_point as an actual rule, but genrule can also work here initially.

@rickeylev, @groodt, this may have some rough edges still (I am wondering if the workaround around the sys.path entries yields to correct behaviour, but I am not sure if there is a better solution yet), but it is ready for initial feedback. The benefit of this approach is that it does not require very clever tricks explored in #1294 in order to expose spoke contents via the hub repo and we can just use the stuff exposed in the hub repo directly.

[chrislovecnm]

@aignas I'm getting the same error on Windows. Not sure if this helps:

PS C:\Users\chris\Workspace\rules_python\examples\bzlmod\bazel-bin\entry_point> c:\\b\\r4du22do\\execroot\\_main\\bazel-out\\x64_windows-fastbuild\\bin\\entry_point\\entry_point_test.exe.runfiles\\_main\\entry_point\\yamllint.exe -v
Traceback (most recent call last):
  File "C:\Users\chris\AppData\Local\Temp\Bazel.runfiles_mbclvpp2\runfiles\_main\entry_point\yamllint.py", line 12, in <module>
    from yamllint.cli import run
  File "C:\Users\chris\AppData\Local\Temp\Bazel.runfiles_mbclvpp2\runfiles\_main\entry_point\yamllint.py", line 12, in <module>
    from yamllint.cli import run
ModuleNotFoundError: No module named 'yamllint.cli'; 'yamllint' is not a package
PS C:\Users\chris\Workspace\rules_python\examples\bzlmod\bazel-bin\entry_point

Although running pylint.exe does work.

[rickeylev]

FYI: I have some pubic API-surface feedback to give, but gtg at the moment.

Skimming the PR, I'll also likely have some implementation feedback, but need to look more closely.

[chrislovecnm]

Overall looks good! I know that @rickeylev has some input as well. Mostly my comments are about documentation. I want to include more documentation to simplify code maintenance and our users' lives.

@rickeylev and I have talked about how much we appreciate all of the work you have done!

[chrislovecnm]

@aignas are we fixing #1055 as well?

[aignas]

I'll be going on vacation from tomorrow, so will not respond to questions for a week and a bit.

[rickeylev]

Ah, I see you've gone through the trouble of implementing the "read from entry points config file" approach I originally was thinking to defer on :).

Well, ok, but lets we need to put some more thought into the design and better document what parts of an entry point config we do and don't support. Lets have that discussion on #979 instead of this PR. I'll post further thoughts on it there. I've tried to refrain from over-commenting on things relating to that.

[aignas]

I'd be grateful if somebody could enlighten me as to why the Windows tests are failing.

[rickeylev]

I'd be grateful if somebody could enlighten me as to why the Windows tests are failing.

Given the error, it looks like the entry point's executable (the "windows launcher" wrapper thing) is running, and that the launcher is failing to find...something. There's a lot of something it could be :(. Do you have a windows machine?

The basic way the windows+zip mechanism work is...

A native windows executable is created. This is called the "windows launcher". This binary has the path to python interpreter to use embedded into it by the py_binary build process. That interpreter path is uses comes from...it's supposed to be the toolchain. However, I seem to recall that --python_path could override it (and might have on windows?), but I'm having trouble finding the logic that did that.

Additionally, it looks like, in the launcher source code, it does if <desired path doesn't exist>: use python.exe.

A few ideas:

1. Strip down the test to just call the entry point without any args. That eliminates any of the other args being passed as potential issues.
2. Try running subprocess.run("python.exe", shell=True). It's not clear to me if it's invoking python.exe via a shell (which would trigger PATH searching), or directly (e.g. via exec, which wouldn't perform PATH searching. I have to assume the former; I don't see how it could have worked without PATH lookup.
3. The launcher source seems to indicate it has a special --print_launcher_command flag
4. Replace pylint with a simple example. This eliminates any issues due to pylint.
5. Check the runfiles environment; nested binaries can interact poorly sometimes
6. I see the test changes the current working directory? If something is looking for a path relative to the execroot, that could confuse things.

The windows launcher source code is here: https://github.com/bazelbuild/bazel/tree/master/src/tools/launcher

To see how the launcher is built, see here. Note that is the starlarkified code from Bazel head. The error is Bazel 6, which would be using the Java impl. The two should be functionally the same, though.

[rickeylev]

Alrighty, almost there!

Comments have been addressed; ready to merge