Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(py_console_script_binary)!: entry points with custom dependencies #1363

Merged
merged 86 commits into from
Aug 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
b4ff67b
feat! add entry_point macro to rules_python
aignas Aug 5, 2023
20263bb
fixup pip_parse_vendored example
aignas Aug 5, 2023
e0e64fe
fixup tests
aignas Aug 5, 2023
c7bb137
fixup: use regex for testing the pylint output
aignas Aug 5, 2023
76fedaf
fixup: add pylint-print to Windows requirements in the example
aignas Aug 5, 2023
85d9bab
fixup windows requirements in the bzlmod example once more
aignas Aug 5, 2023
72d7ad5
fixup: add transitions
aignas Aug 5, 2023
4a6ce9e
doc: Update docs with examples
aignas Aug 5, 2023
f976b4a
comment: rename to py_entry_point_binary.bzl
aignas Aug 5, 2023
2463ad0
comment: disallow main
aignas Aug 5, 2023
b77ef08
comment: default to the pkg name for the script if not specified
aignas Aug 5, 2023
cf2f481
pass the binary rule as a parameter
aignas Aug 5, 2023
1d20bd8
fixup rename
aignas Aug 5, 2023
6d320fe
comment: change the macro name
aignas Aug 5, 2023
0f979f5
comment: simplify the entry_point with transition inclusion
aignas Aug 5, 2023
8ada8f9
comment: revert the inference of the script name because the pkg labe…
aignas Aug 5, 2023
32ff458
chore: update docs
aignas Aug 5, 2023
a04f6b5
fixup: add debugging output when import for the entry_point fails
aignas Aug 5, 2023
026281f
docs: rename py_entry_point_binary docs
aignas Aug 6, 2023
04f715a
refactor: reimplement the genrule as a simple rule
aignas Aug 7, 2023
054f93a
Rewrite the use of genrule to a private rule that uses the python too…
aignas Aug 7, 2023
82e756a
chore: add bzl_library targets and rework doc generation
aignas Aug 7, 2023
e09e72c
chore: remove a TODO note
aignas Aug 7, 2023
470acbe
refactor: move and rename to //python/entry_point:py_console_script_b…
aignas Aug 10, 2023
b82178a
refactor: generate docs from //python/entry_point:py_console_script_b…
aignas Aug 10, 2023
0330ca7
refactor: move to the new import
aignas Aug 10, 2023
a1a05dd
chore: migrate twine_publisher to py_console_script_binary
aignas Aug 11, 2023
8a18304
test: fix test importpath after moving the Py files
aignas Aug 11, 2023
d60abf8
comment: move tests to '//tests'
aignas Aug 20, 2023
cdf8903
comment: remove shebang as it is not used
aignas Aug 20, 2023
8b92e29
comment: use py_binary for entry_point generation tool
aignas Aug 20, 2023
5c0431b
comment: restrict .bzl visibility to the docs subpackage
aignas Aug 20, 2023
39a57a4
comment: validate that srcs is not passed in
aignas Aug 20, 2023
59e9b1d
comment: drop unnecessary kwargs.pop for main
aignas Aug 20, 2023
861a8fd
comment: prepend internal targets with '_'
aignas Aug 20, 2023
9d48df0
comment: remove shebang from the generator as it is not used
aignas Aug 20, 2023
45149df
comment: add the py_console_script_binary_bzl to build_test
aignas Aug 20, 2023
d204b2b
revert: "migrate twine_publisher to py_console_script_binary"
aignas Aug 20, 2023
db8f3ac
comment: move things around
aignas Aug 20, 2023
df6660a
comment: rename _impl for easier grepping
aignas Aug 20, 2023
ce157dc
comment: re-raise the exception on ImportError
aignas Aug 20, 2023
3d382d4
comment: sort console scripts before raising an exception
aignas Aug 20, 2023
b47848b
Simplify the second test
aignas Aug 20, 2023
4be2cac
comment: stop using __file__ in the test
aignas Aug 20, 2023
38d3cd3
comment: fixup bzlmod error message
aignas Aug 20, 2023
d3b1c7f
comment: add a todo for extras in entry_points #1383
aignas Aug 20, 2023
db1aa77
comment: improve error handling logic
aignas Aug 20, 2023
9939ba2
comment: update the docstring for the py_console_script_gen
aignas Aug 20, 2023
45b93c2
Add a note on map_each
aignas Aug 20, 2023
22a24ed
comment: add progress message and mnemonic
aignas Aug 20, 2023
1883ad3
chore: remove duplicated docs
aignas Aug 20, 2023
197f5de
comment: clarify docs
aignas Aug 20, 2023
933643b
comment: dig a little deeper as to why we need to alter sys.path
aignas Aug 20, 2023
841aab2
test: split the entry_point tests to each file to ensure minimal sys.…
aignas Aug 20, 2023
530a8f9
fixup: use the correct env variable for PYLINTHOME
aignas Aug 20, 2023
923c93d
fix: add back the sys.path workaround and add clearer documentation a…
aignas Aug 20, 2023
a9a3e62
comment: revert the deprecation warning
aignas Aug 20, 2023
f5cfd7b
fixup! fix: add back the sys.path workaround and add clearer document…
aignas Aug 20, 2023
5bca053
feat: add better auto-detection of console_script
aignas Aug 20, 2023
7dbf322
feat: allow the user to supply his own entry_points_txt config
aignas Aug 20, 2023
48b52ea
doc: clarify docs
aignas Aug 20, 2023
0915a57
chore: add a changelog entry
aignas Aug 22, 2023
9e5d47d
test: exclude the pylint_with_deps test on windows
aignas Aug 22, 2023
e40a9ad
chore: rebase changelog entry after a release
aignas Aug 23, 2023
79e3732
fix: do not chdir in the subprocess.run
aignas Aug 23, 2023
c06096c
bzlmod: rewrite the example for linting files with pylint
aignas Aug 23, 2023
1fc498a
doc: add a note about the removal of the entry_point macro for bzlmod…
aignas Aug 23, 2023
e23cca6
docs: improve docs
aignas Aug 23, 2023
b7d3895
comment: move tests to a separate directory
aignas Aug 23, 2023
c153693
test: use a regular file instead of write_file skylib rule
aignas Aug 23, 2023
b9db878
comment: add a note on the subprocess.run in entry_point tests
aignas Aug 24, 2023
3020e21
comment: improve wording of docs
aignas Aug 24, 2023
b0aa698
comment: move imports to the right location.
aignas Aug 24, 2023
606e08b
comment: improve entry_points_txt docs
aignas Aug 24, 2023
87a9e07
fix: support Label as the pkg target
aignas Aug 24, 2023
e76b286
comment: improve the CHANGELOG
aignas Aug 24, 2023
7851fbb
comment: improve progress_message
aignas Aug 24, 2023
f43df6c
comment: improve internal docstring
aignas Aug 24, 2023
c978708
comment: improve a replacement suggestion for bzlmod users
aignas Aug 24, 2023
8f9eb1e
fix: do not normalize strings when guessing entry_point name
aignas Aug 24, 2023
ed0e82e
fix a docstring reference to entry_points.txt
aignas Aug 24, 2023
924a8dd
fix: detect sys.flags.safe_path on newer Python versions
aignas Aug 24, 2023
3480c12
pre-commit: update deleted-packages
aignas Aug 24, 2023
ff8d1f4
fixup tests
aignas Aug 24, 2023
58f3c89
Update python/private/py_console_script_binary.bzl
rickeylev Aug 25, 2023
51cf7e5
Update python/private/py_console_script_binary.bzl
rickeylev Aug 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
# This lets us glob() up all the files inside the examples to make them inputs to tests
# (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it)
# To update these lines, run tools/bazel_integration_test/update_deleted_packages.sh
build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_point,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_point,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points
query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_install,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,tests/compile_pip_requirements,tests/compile_pip_requirements_test_from_external_workspace,tests/ignore_root_user_error,tests/pip_repository_entry_points

test --test_output=errors

Expand Down
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@ A brief description of the categories of changes:
* Particular sub-systems are identified using parentheses, e.g. `(bzlmod)` or
`(docs)`.

## Unreleased

### Added

* (bzlmod, entry_point) Added
[`py_console_script_binary`](./docs/py_console_script_binary.md), which
allows adding custom dependencies to a package's entry points and customizing
the `py_binary` rule used to build it.

### Removed

* (bzlmod) The `entry_point` macro is no longer supported and has been removed
in favour of the `py_console_script_binary` macro for `bzlmod` users.

## [0.25.0] - 2023-08-22

### Changed
Expand Down
11 changes: 11 additions & 0 deletions docs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ _DOCS = {
"pip_repository": "//docs:pip-repository",
"py_cc_toolchain": "//docs:py_cc_toolchain-docs",
"py_cc_toolchain_info": "//docs:py_cc_toolchain_info-docs",
"py_console_script_binary": "//docs:py-console-script-binary",
"python": "//docs:core-docs",
}

Expand Down Expand Up @@ -128,6 +129,16 @@ stardoc(
],
)

stardoc(
name = "py-console-script-binary",
out = "py_console_script_binary.md_",
input = "//python/entry_points:py_console_script_binary.bzl",
target_compatible_with = _NOT_WINDOWS,
deps = [
"//python/entry_points:py_console_script_binary_bzl",
],
)

stardoc(
name = "packaging-docs",
out = "packaging.md_",
Expand Down
87 changes: 87 additions & 0 deletions docs/py_console_script_binary.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 1 addition & 4 deletions examples/bzlmod/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -113,10 +113,7 @@ pip.parse(
"@whl_mods_hub//:wheel.json": "wheel",
},
)

# NOTE: The pip_39 repo is only used because the plain `@pip` repo doesn't
# yet support entry points; see https://github.com/bazelbuild/rules_python/issues/1262
use_repo(pip, "pip", "pip_39")
use_repo(pip, "pip")

bazel_dep(name = "other_module", version = "", repo_name = "our_other_module")
local_path_override(
Expand Down
20 changes: 0 additions & 20 deletions examples/bzlmod/entry_point/BUILD.bazel

This file was deleted.

33 changes: 33 additions & 0 deletions examples/bzlmod/entry_points/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
load("@python_versions//3.9:defs.bzl", py_console_script_binary_3_9 = "py_console_script_binary")
load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary")

# This is how you can define a `pylint` entrypoint which uses the default python version.
py_console_script_binary(
name = "pylint",
pkg = "@pip//pylint",
visibility = ["//entry_points:__subpackages__"],
)

# We can also specify extra dependencies for the binary, which is useful for
# tools like flake8, pylint, pytest, which have plugin discovery methods.
py_console_script_binary(
name = "pylint_with_deps",
pkg = "@pip//pylint",
# Because `pylint` has multiple console_scripts available, we have to
# specify which we want if the name of the target name 'pylint_with_deps'
# cannot be used to guess the entry_point script.
script = "pylint",
visibility = ["//entry_points:__subpackages__"],
deps = [
# One can add extra dependencies to the entry point.
"@pip//pylint_print",
],
)

# A specific Python version can be forced by using the generated version-aware
# wrappers, e.g. to force Python 3.9:
py_console_script_binary_3_9(
name = "yamllint",
pkg = "@pip//yamllint:pkg",
visibility = ["//entry_points:__subpackages__"],
)
63 changes: 63 additions & 0 deletions examples/bzlmod/entry_points/tests/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
load("@bazel_skylib//rules:run_binary.bzl", "run_binary")
load("@rules_python//python:defs.bzl", "py_test")

# Below are targets for testing the `py_console_script_binary` feature and are
# not part of the example how to use the feature.

# And a test that we can correctly run `pylint --version`
py_test(
name = "pylint_test",
srcs = ["pylint_test.py"],
data = ["//entry_points:pylint"],
env = {
"ENTRY_POINT": "$(rlocationpath //entry_points:pylint)",
},
deps = ["@rules_python//python/runfiles"],
)

# Next run pylint on the file to generate a report.
run_binary(
name = "pylint_report",
srcs = [
":file_with_pylint_errors.py",
],
outs = ["pylint_report.txt"],
args = [
"--output-format=text:$(location pylint_report.txt)",
"--load-plugins=pylint_print",
# The `exit-zero` ensures that `run_binary` is successful even though there are lint errors.
# We check the generated report in the test below.
"--exit-zero",
"$(location :file_with_pylint_errors.py)",
],
env = {
# otherwise it may try to create ${HOME}/.cache/pylint
"PYLINTHOME": "./.pylint_home",
},
tool = "//entry_points:pylint_with_deps",
)

py_test(
name = "pylint_deps_test",
srcs = ["pylint_deps_test.py"],
data = [
":pylint_report",
"//entry_points:pylint_with_deps",
],
env = {
"ENTRY_POINT": "$(rlocationpath //entry_points:pylint_with_deps)",
"PYLINT_REPORT": "$(rlocationpath :pylint_report)",
},
deps = ["@rules_python//python/runfiles"],
)

# And a test to check that yamllint works
py_test(
name = "yamllint_test",
srcs = ["yamllint_test.py"],
data = ["//entry_points:yamllint"],
env = {
"ENTRY_POINT": "$(rlocationpath //entry_points:yamllint)",
},
deps = ["@rules_python//python/runfiles"],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""
A file to demonstrate the pylint-print checker works.
"""

if __name__ == "__main__":
print("Hello, World!")
72 changes: 72 additions & 0 deletions examples/bzlmod/entry_points/tests/pylint_deps_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Copyright 2023 The Bazel Authors. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import os
import pathlib
import subprocess
import tempfile
import unittest

from python.runfiles import runfiles


class ExampleTest(unittest.TestCase):
def __init__(self, *args, **kwargs):
self.maxDiff = None

super().__init__(*args, **kwargs)

def test_pylint_entry_point(self):
rlocation_path = os.environ.get("ENTRY_POINT")
assert (
rlocation_path is not None
), "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool"

entry_point = pathlib.Path(runfiles.Create().Rlocation(rlocation_path))
self.assertTrue(entry_point.exists(), f"'{entry_point}' does not exist")

# Let's run the entrypoint and check the tool version.
#
# NOTE @aignas 2023-08-24: the Windows python launcher with Python 3.9 and bazel 6 is not happy if we start
# passing extra files via `subprocess.run` and it starts to fail with an error that the file which is the
# entry_point cannot be found. However, just calling `--version` seems to be fine.
proc = subprocess.run(
[str(entry_point), "--version"],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
self.assertEqual(
"",
proc.stderr.decode("utf-8").strip(),
)
self.assertRegex(proc.stdout.decode("utf-8").strip(), "^pylint 2\.15\.9")

def test_pylint_report_has_expected_warnings(self):
rlocation_path = os.environ.get("PYLINT_REPORT")
assert (
rlocation_path is not None
), "expected 'PYLINT_REPORT' env variable to be set to rlocation of the report"

pylint_report = pathlib.Path(runfiles.Create().Rlocation(rlocation_path))
self.assertTrue(pylint_report.exists(), f"'{pylint_report}' does not exist")

self.assertRegex(
pylint_report.read_text().strip(),
"W8201: Logging should be used instead of the print\(\) function\. \(print-function\)",
)


if __name__ == "__main__":
unittest.main()
Loading