release_2.18.0

❗❗❗ This project is no longer being released via PyPI, and is instead released via GitHub Releases. See https://github.com/pantsbuild/pants/releases

Individuals and companies can now sponsor Pants financially.

Pants is an open-source project that is not owned or controlled by any one company or organization, and does incur some expenses. These expenses are managed by Pants Build, a non-profit that was established for this purpose. This non-profit's only source of revenue is sponsorship by individuals and companies that use Pants.

We offer formal sponsorship tiers for companies, as well as individual sponsorships via GitHub.

What's New

Highlights

• The improvements to AWS Lambda and Google Cloud Function support in 2.17 have continued in 2.18, including simpler configuration and support for AWS Lambda Layers.
• Many more options can now be controlled when building Docker images.
• Significant feature work and bug fixes to "deploy" backends like Helm and Terraform.

Keep reading to see the details and what's also included.

Overall

The pants launcher binary is now required to run Pants. It must be version 0.10.0 or greater: run PANTS_BOOTSTRAP_VERSION=report pants to check and SCIE_BOOT=update pants to upgrade to the latest. Pants 2.18 and later are not distributed via PyPI, and thus the launcher is required to coordinate downloading the appropriate versions from GitHub Releases.

The [cli.alias] section of pants.toml now supports defining flag-like aliases. For example: --all-changed = "--changed-since=HEAD --changed-dependents=transitive".

The paths goal now supports passing groups of targets to --from and/or --to.

The help goal now supports showing help for an environment variable. For example: pants help PANTS_FIX_ONLY.

Additional bug fixes:

• The client now detects changes in pants configuration that require restarting the pants daemon.
• Sequences of goals that mutate files on disk and then immediately re-read them now no longer fail spuriously if the second goal happens faster than the OS file-change notification is received. For instance, pants fix lint :: would sometimes run the lint goal on the files before the fix changes were applied.
• More reliable detection of when optimised hard-linking can be used for creating sandboxes. For instance, no longer hitting errors when a single device is mounted multiple times.

The deprecation for referring to a "secondary" owner target via the file name has expired, and so this is no longer supported. For example, if path/to/main.py is the entry point of the path/to:binary pex_binary target, an invocation like pants package path/to/main.py must now use the target address instead: pants package path/to:binary.

BUILD files

Several bug fixes:

• Better error messages when a BUILD file refers to an unknown value.
• __defaults__ can now be used in the same BUILD file as environment targets like local_environment.

Performance

Memory usage has been reduced, particularly for the Go backend.

A best-effort limit has been placed on the number of open cache file handles (approximately 1000), reducing the need for ulimit adjustments.

Remote caching/execution

Remote caching using the gRPC remote execution API now supports mTLS, via the new remote_client_certs_path and remote_client_key_path options in the [GLOBAL] section.

Backends

Several backends that were intended to be included in previous releases were accidentally missing from the released pants artifacts, these are now available in 2.18:

• pants.backend.experimental.openapi.codegen.java: Enables generating Java from OpenAPI.
• pants.backend.experimental.python.framework.django: Enables better support for projects using Django.
• pants.backend.experimental.swift: Enables core Swift support.
• pants.backend.python.providers.experimental.pyenv.custom_install: Enables customising how the pyenv provider builds a Python interpreter.

In addition, the documentation of backends now lists all backends, including preview ones that were not previously advertised. Please go check it out, try backends relevant to you and let us know how they go for your use cases, report bugs and/or submit fixes!

Docker

More options can be passed when building images (see and docs for more details:

• on the [docker] subsystem: build_hosts, build_no_cache
• on the docker_image target: build_network, build_platform, extra_build_hosts

Additional features and bug fixes:

• Registries with the same address no longer silently overwrite each other, and instead now emit an error.
• The secrets field now supports expanding ~ to refer to the current user's home directory.
• The image_tags field now has an explanatory error if it is incorrectly empty.

Helm

The helm_unittest_tests target has had several improvements:

• support for being added by the tailor goal (this can be disabled with the new [helm].tailor_unittests option)
• support for snapshot testing via the new generate-snapshots goal

The new pants.backend.experimental.helm.check.kubeconform backend allows running Kubeconform on helm_chart and helm_deployment targets as part of the check goal.

Deprecations:

• The helm_deployment target should now specify the new chart field to explicitly indicate which chart is being deployed.
• The create_namespace field for helm_deployment will not be supported in future, use the passthrough argument --create-namespace instead.
• The {env.VAR_NAME} interpolation syntax some fields of helm_deployment has been replaced by the more general env build file function; for instance "text-{env.VAR_NAME}" can become f"text-{env('VAR_NAME')}".
• The [helm].tailor option has been renamed to tailor_charts.

Additional features and bug fixes:

• Relative target addresses for docker_images can be passed like helm_deployment(values={"some_image": ":target-name"}), and are expanded in the same way as non-relative addreses path/to:target-name.
• If the passthrough --debug argument is specified, like pants experimental-deploy path/to:helm-deployment -- --debug, that flag is now passed to all Helm invocations pants does.
• The Helm binary has been upgraded to 3.12.3 by default.
• The new version field to helm_chart allows overriding the version in Chart.yaml.

JavaScript

The node_build_script function now supports extra_env_vars to specify additional environment variables when executing.

Node-based tools can now install from a user-provided 'resolve'/package.json, similar to Python tools.

The dependency inference for JavaScript has been rewritten into Rust, and thus runs much faster.

Python

The functions-as-a-service (FaaS) backends, pants.backend.awslambda.python and pants.backend.google_cloud_function.python have seen more improvements in 2.18:

• Deprecation: the python_awslambda target has been renamed to python_aws_lambda_function.
• Deprecation: the [lambdex].layout = "zip" value in pants.toml is now the default, and the [lambdex] section should be removed.
• New: the python_aws_lambda_layer target can be used to build Layers for AWS Lambda.
• New: the runtime field is no longer necessary when the interpreter constraints unambiguously refer to an entire single major version. For instance, with [python].interpreter_constraints = ["==3.11.*"], one doesn't need to specify runtime="python3.11" to python_aws_lambda_function or runtime="python311" to python_google_cloud_function, as it will be inferred.
• New: for AWS Lambda targets, the complete_platforms argument now has default values provided by Pants, which gives more reliable builds, especially when building on a different platform than the AWS Lambda itself. This change also means that only one of the complete_platforms or platforms fields should be provided.

The new, faster parser for dependency inference is now the default, so the [python-infer].use_rust_parser option can be removed from pants.toml, if it is being explicitly set to true. The old parser is deprecated and will be removed: follow the warnings in 2.17 to move to the new parser.

The new pants.backend.experimental.python.typecheck.pytype backend allows running the pytype type checker as part of the check goal.

Additional features and bug fixes:

• The tailor goal now supports automatically creating python_requirements targets for in pyproject.toml files that follow PEP 621 to specify dependencies.
• Imports within a with contextlib.suppress(ImportError): ... construct are now considered weak imports.
• The global coverage report now correctly handles namespaced packages.
• pex_binary targets that consume python_distribution targets now work with just the distributions, not their transitive dependencies (the files that are used to build the distribution).
• Pants now emits an error if isort cannot read its configuration file (NB. isort itself emits warnings for this, but they would not be visible when running isort under the default Pants configuration).
• The pants.backend.python.typecheck.mypy backend is now more resilient to its cache being removed.

Some deprecations have expired and the supporting code has been removed:

• the "old style" tool lockfiles are no longer supported, and instead customising the versions/installation of tools like black, mypy and pytest should configure a "normal" resolve
• using pants export without a --resolve flag is no longer supported, always pass that flag

Go

Memory usage has been reduced.

Go processes are now cached using the full version of the Go compiler (for example, go1.20.6) so that artifacts are not incorrectly reused between different patch versions.

Building with coverage no longer has an exponential blow-up when preparing the build requests.

JVM

The deprecation for the excludes field on jvm_artifact has expired, use exclusions instead.

The coursier tool now uses v2.1.6 by default, and, for this version, supports a native binary on ARM macOS (Apple Silicon).

The deploy_jar target now supports an exclude_files field.

NEW: Rust

The new pants.backend.experimental.rust backend has been added, with support for using rustfmt to format Rust source files.

Scala

The new scala_artifact target can be used to track third-party artifacts from Scala, with Pants then understanding how Scala artifacts include the Scala version in their coordinates.

Shell

The shellcheck binary for the pants.backend.shell.lint.shellcheck backend is now fetched from vscode-shellcheck, and thus now supports AArch64 macOS (Apple Silicon) natively.

The deprecations for the pre-2.16 target names have expired, and thus these have been removed:

• experimental_shell_command: use shell_command
• experimental_run_shell_command: use run_shell_command

Terraform

The new terraform_deployment target allows configuring a terraform_module to be deployed with the experimental-deploy goal.

A new pants.backend.experimental.terraform.lint.tfsec backend has been added to run the tfsec linter as part of the lint goal.

The check goal now runs terraform init to ensure modules are installed before validating.

Visibility

The pants.backend.experimental.visibility backend is now also runs as a linter, such as when running pants lint ::, to report all visibility violations at one time.

Visibility rules about dependencies in the root of the repository are now supported.

Plugin API Changes

Pants no longer publishes wheels to PyPI. The only supported way to depend on pants itself is the pants_requirements target, which now supports a new version_spec field if one needs to depend on a different version of Pants than the currently executing one.

The include_special_cased_deps flag (on DependenciesRequest, TransitiveTargetsRequest, and CoarsenedTargetsRequest) is replaced with should_traverse_deps_predicate. If set to True, use should_traverse_deps_predicate=pants.engine.target.AlwaysTraverseDeps() instead. See #19272 and #19387 for details.

The TestResult stdout and stderr fields have been renamed to stdout_bytes and stderr_bytes, and store bytes instead of str, to handle non-UTF-8 data without crashing.

Other changes:

• Plugins can now declare additional requirements in a requirements.txt file next to register.py.
• Pants only supports running on Python 3.9 now.
• Named caches can now used digits in addition to lower-case letters and underscores.
• The pants.core.goals.package.TraverseIfNotPackageTarget type can be used with the new should_traverse_deps_predicate to treat packageable targets as "leaves" when finding dependencies; for instance, stopping at a pex_binary without also surfacing the python_sources that are used for it.

The deprecations for many request types has expired (follow instructions in warnings in 2.17 to resolve), so they are now removed: AllAssetTargetsRequest, AllTargetsRequest, BashBinaryRequest, CatBinaryRequest, CGoCompilerWrapperScriptRequest, ChmodBinaryRequest, DiffBinaryRequest, DjangoAppsRequest, DockerBinaryRequest, GitBinaryRequest, GunzipBinaryRequest, MkdirBinaryRequest, PythonBinary, ReadlinkBinaryRequest, TarBinaryRequest, UnzipBinaryRequest, ZipBinaryRequest.

Full Changelog

For the full changelog, see the individual GitHub Releases for this series: https://github.com/pantsbuild/pants/releases