Enable custom py_binary stub_template attribute (via py_runtime) #16806

fahhem · 2022-11-19T22:47:53Z

Fixes #137, but unlike my approach in #6632, this adds an attribute to py_runtime rather than to py_binary

Open to suggestions on the attribute name and documentation

src/main/starlark/builtins_bzl/common/python/py_runtime_rule.bzl

rickeylev · 2022-11-21T20:49:56Z

src/main/starlark/builtins_bzl/common/python/py_runtime_rule.bzl

+            allow_single_file = True,
+            default = DEFAULT_STUB_SCRIPT_TEMPLATE,
+            doc = """
+The stub script template file to use. Should have %python_binary%,


I'd like for the variables to be more formally documented, but I think this is OK for now.

Can you please add 3 pieces of doc, though?

This template, after expansion, becomes the executable file used to start the process, so it is responsible for initial bootstrapping actions such as
finding the Python interpreter, runfiles, and constructing an environment
to run the intended Python application.

And also:

While this attribute is currently optional, it will become required when the Python rules are moved out of Bazel itself.

And also:

The exact variable names expanded is an unstable API and is subject to change. The API will become more stable when the Python rules are moved out of Bazel itself.

(also, you can omit such doc in the java impl; I'll be deleting it in a couple weeks. I've just been waiting for the Starlark version to be released within Google to serve as a spot-check before deleting the Java impl)

Why would the attribute become required? Is the plan to remove the bootstrap template that I'm moving to bazel_tools?

Added the comments verbatim for now

Is the plan to remove the bootstrap template that I'm moving to bazel_tools?

Yep! The plan is to remove all this py rule stuff from Bazel itself and move it into rules_python instead. The pieces to start this are almost all in place.

src/main/java/com/google/devtools/build/lib/rules/python/PyRuntimeInfo.java

src/main/java/com/google/devtools/build/lib/bazel/rules/python/BazelPyRuleClasses.java

src/main/starlark/builtins_bzl/common/python/py_runtime_rule.bzl

src/main/java/com/google/devtools/build/lib/starlarkbuildapi/python/PyRuntimeInfoApi.java

src/main/starlark/builtins_bzl/common/python/providers.bzl

rickeylev · 2022-11-22T18:32:34Z

src/test/java/com/google/devtools/build/lib/packages/util/BazelMockPythonSupport.java

@@ -39,6 +39,7 @@ public void setup(MockToolsConfig config) throws IOException {
    addTool(config, "tools/python/toolchain.bzl");
    addTool(config, "tools/python/utils.bzl");
    addTool(config, "tools/python/private/defs.bzl");
+    addTool(config, "tools/python/python_bootstrap_template.txt");


note to self: this will probably need to also be added to the internal MockPythonSupport version of this class when importing

fahhem · 2022-11-22T19:21:25Z

Thanks Richard! I replied to the questions, and I'll follow up on the actionable comments today

fahhem · 2022-11-23T19:36:02Z

Done with the review, thank you @rickeylev !

src/main/java/com/google/devtools/build/lib/starlarkbuildapi/python/PyRuntimeInfoApi.java

src/main/starlark/builtins_bzl/common/python/py_runtime_rule.bzl

src/main/java/com/google/devtools/build/lib/rules/python/PyRuntimeInfo.java

fahhem · 2022-11-30T22:05:23Z

Seems like the main change needed was just to switch to ctx.file and then fix the two places I had workarounds to "support" ctx.attr being not a FileApi? I've done the above and it looks like the tests are passing, are we good to merge?

src/main/starlark/builtins_bzl/common/python/providers.bzl

rickeylev · 2022-12-02T01:45:34Z

src/main/java/com/google/devtools/build/lib/starlarkbuildapi/python/PyRuntimeInfoApi.java

+              allowedTypes = {
+                @ParamType(type = FileApi.class),
+              },


Does this permit None being passed?

I added NoneType since I think we should allow None to be passed in

rickeylev · 2022-12-02T01:46:33Z

src/main/java/com/google/devtools/build/lib/starlarkbuildapi/python/PyRuntimeInfoApi.java

@@ -46,6 +47,7 @@
 public interface PyRuntimeInfoApi<FileT extends FileApi> extends StarlarkValue {

  static final String DEFAULT_STUB_SHEBANG = "#!/usr/bin/env python3";
+  static final String DEFAULT_BOOTSTRAP_TEMPLATE = "//tools/python:python_bootstrap_template.txt";


Should this have "@bazel_tools"? Or is that implicit in this context for some reason?

I think it was, but it's not used here anymore so deleted it

Actually added it back, it's used (but via the subclass PyRuntimeInfo, that's why I didn't find it via rg). I added a comment that it should be used with getToolsLabel() to use it correctly

src/main/java/com/google/devtools/build/lib/rules/python/PyRuntimeInfo.java

rickeylev · 2022-12-14T23:43:07Z

@fahhem just bumping this in case it got lost

fahhem · 2022-12-21T20:51:07Z

Sorry, needed to get something else done for work, but coming back to this now.

Fixes bazelbuild#137, but unlike my approach in bazelbuild#6632, this adds an attribute to `py_runtime` rather than to `py_binary` Open to suggestions on the attribute name and documentation

rickeylev · 2022-12-21T23:37:28Z

LGTM.

fahhem · 2023-01-06T04:00:13Z

Thank you @rickeylev! I assume this will be merged by copybara or something in the coming weeks? Do I need to do anything to get it into the next 6.x release?

rickeylev · 2023-01-06T04:37:07Z

Heyas,

Just got back from vacation the other day. The Bazel-pr-importer-team (that's who the "awaiting-pr-merge" label notifies) starting importing it, but some failures popped up so I had to fix those first. I'm now re-importing it and it's looking good; I expect to submit it tomorrow or Monday, and it'll show up on github automatically shortly thereafter.

This is just a small quality of life improvement. This is originally from PR #16806; it was split off to make submitting the main change of that PR easier. PiperOrigin-RevId: 500222350 Change-Id: I274f527340086a131d81f11a123bde240c87d34c

Fixes #137, but unlike my approach in #6632, this adds an attribute to `py_runtime` rather than to `py_binary` Closes #16806. PiperOrigin-RevId: 500248760 Change-Id: Ic39b0e9f8ae8063c3dedd1f67feece2e69de6306

…y_runtime on bazel latest This PR fixes a small bug where the bootstrap template runtime file was not being passed to Py Binary. This broke the bootstrap template feature on bazel latest authored in #16806 Closes #18103. PiperOrigin-RevId: 525042380 Change-Id: I8b8a200634eb98e156b4d8ba6b2e5baef5d06c73

…y_runtime on bazel latest This PR fixes a small bug where the bootstrap template runtime file was not being passed to Py Binary. This broke the bootstrap template feature on bazel latest authored in bazelbuild#16806 Closes bazelbuild#18103. PiperOrigin-RevId: 525042380 Change-Id: I8b8a200634eb98e156b4d8ba6b2e5baef5d06c73

fahhem requested review from lberki and comius as code owners November 19, 2022 22:47

fahhem marked this pull request as draft November 19, 2022 22:58

lberki requested review from rickeylev and removed request for lberki November 21, 2022 08:07

rickeylev requested changes Nov 21, 2022

View reviewed changes

fahhem marked this pull request as ready for review November 22, 2022 07:50

fahhem requested review from rickeylev and removed request for comius November 22, 2022 07:51

fahhem mentioned this pull request Nov 22, 2022

Enable custom py_binary stub_template attribute #6632

Closed

sgowroji added team-Rules-Python Native rules for Python awaiting-review PR is awaiting review from an assigned reviewer labels Nov 22, 2022

rickeylev requested changes Nov 22, 2022

View reviewed changes

fahhem requested a review from rickeylev November 22, 2022 21:42

fahhem force-pushed the custom_stub_runtime branch from e89f988 to ac137e5 Compare November 23, 2022 19:37

rickeylev requested changes Nov 28, 2022

View reviewed changes

fahhem force-pushed the custom_stub_runtime branch from 361aeb9 to 9dc72d6 Compare November 30, 2022 21:47

fahhem requested a review from rickeylev November 30, 2022 22:05

fahhem mentioned this pull request Dec 1, 2022

Allow custom stub template for py_binary's #137

Closed

rickeylev requested changes Dec 2, 2022

View reviewed changes

fahhem added 3 commits December 21, 2022 12:52

Enable custom py_binary stub_template attribute (via py_runtime)

c6b5adc

Fixes bazelbuild#137, but unlike my approach in bazelbuild#6632, this adds an attribute to `py_runtime` rather than to `py_binary` Open to suggestions on the attribute name and documentation

ctx.attr => ctx.file fixes so many things

8f221ed

rickeylev review: removing unused code and using TOOLS_REPO

2e4d998

fahhem force-pushed the custom_stub_runtime branch from 9dc72d6 to 2e4d998 Compare December 21, 2022 21:00

fahhem requested a review from rickeylev December 21, 2022 21:01

add back DEFAULT_BOOTSTRAP_TEMPLATE string

32d6193

rickeylev approved these changes Dec 21, 2022

View reviewed changes

rickeylev added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Dec 23, 2022

copybara-service bot closed this in 0696ba3 Jan 7, 2023

sgowroji removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Jan 9, 2023

fahhem deleted the custom_stub_runtime branch January 21, 2023 01:59

ankit-agarwal1999 mentioned this pull request Apr 14, 2023

[Bootstrap Template] Fixed bootstrap template not being passed to py_binary from py_runtime on bazel latest #18103

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable custom py_binary stub_template attribute (via py_runtime) #16806

Enable custom py_binary stub_template attribute (via py_runtime) #16806

fahhem commented Nov 19, 2022

rickeylev Nov 21, 2022

fahhem Nov 22, 2022

rickeylev Nov 22, 2022

rickeylev Nov 22, 2022

fahhem commented Nov 22, 2022

fahhem commented Nov 23, 2022

fahhem commented Nov 30, 2022

rickeylev Dec 2, 2022

fahhem Dec 21, 2022

rickeylev Dec 2, 2022

fahhem Dec 21, 2022

fahhem Dec 21, 2022

rickeylev commented Dec 14, 2022

fahhem commented Dec 21, 2022

rickeylev commented Dec 21, 2022

fahhem commented Jan 6, 2023

rickeylev commented Jan 6, 2023

Enable custom py_binary stub_template attribute (via py_runtime) #16806

Enable custom py_binary stub_template attribute (via py_runtime) #16806

Conversation

fahhem commented Nov 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fahhem commented Nov 22, 2022

fahhem commented Nov 23, 2022

fahhem commented Nov 30, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rickeylev commented Dec 14, 2022

fahhem commented Dec 21, 2022

rickeylev commented Dec 21, 2022

fahhem commented Jan 6, 2023

rickeylev commented Jan 6, 2023