Pants 2 is a fast, scalable, user-friendly build system for codebases of all sizes. It's currently focused on Python, Go, Java, Scala, Kotlin, Shell, and Docker, with support for other languages and frameworks coming soon.
Individuals and companies can now sponsor Pants financially.
Pants is an open-source project that is not owned or controlled by any one company or organization, and does incur some expenses. These expenses are managed by Pants Build, a non-profit that was established for this purpose. This non-profit's only source of revenue is sponsorship by individuals and companies that use Pants.
We offer formal sponsorship tiers for companies, as well as individual sponsorships via GitHub.
- Test retries for flaky Python tests.
- Using the
parametrizeto set multiple fields at once when generating new groups of targets. buildxsupport in the Docker backend.- New backends for running semgrep and openapi-format.
Keep reading to see the details and what's also included.
The .pids/ internal directory is now under .pants.d/ by default, rather that adjacent to it (this can be controlled via the [GLOBAL].pants_subprocessdir option).
The new [system-binaries] subsystem allows controlling where Pants searches for binaries like bash, zip, tar that is uses as part of normal operation. Set the system_binaries_paths as appropriate, potentially using the <PATH> special value. This can also be set on specific environments using the system_binaries_system_binary_paths field.
Additional features and fixes:
- New versions of the
get-pants.shscript install to~/.local/bin, not~/bin. - Environment variables that aren't valid UTF-8 no longer cause Pants to crash (note,
PANTS_...environment variables that Pants needs to read but aren't UTF-8 will be ignored). - Reference documentation for subsystem options includes how to set them in
pants.toml, in addition to the environment variable and CLI arguments. - Environments are now more resilient to being (partially) defined via macros, although this is not recommended.
- Docker environments can now pull public images without credentials, and can now handle docker images without Python.
- The
fixgoal now partitions files across multiple processes in a way that's more similar to the partitioning of thelintgoal, which can work around some spurious linting failures.
The parametrize helper now supports parametrizing multiple fields together, in lock step. For instance:
# Creates two targets:
#
# example:tests@parametrize=py2
# example:tests@parametrize=py3
python_test(
name="tests",
source="tests.py",
**parametrize("py2", interpreter_constraints=["==2.7.*"], resolve="lock-a"),
**parametrize("py3", interpreter_constraints=[">=3.6,<3.7"], resolve="lock-b"),
)In addition, parametrize can now be used with a single value (field=parametrize("for-consistency")), and now works on target generations in more cases (particularly those with "plugin" fields).
Some processes are run with more consistent "color" configuration which may result in more cache hits: Pytest, Helm unittest and shunit2 processes now always run with colored output enabled. If [GLOBAL].colors is not enabled, Pants removes the coloring before displaying.
Large files that need to be stored to a remote cache are now streamed to it directly from disk, in a way that should reduce overhead and increase concurrency (#19711).
The deprecation has expired for the [GLOBAL].remote_store_chunk_upload_timeout_seconds and [GLOBAL].remote_cache_read_timeout_millis options. Use [GLOBAL].remote_store_rpc_timeout_millis and [GLOBAL].remote_cache_rpc_timeout_millis instead.
The docker backend now has explicit support for using buildx (and thus BuildKit), via the new use_buildx option. This also allows adding few additional fields for docker_image targets, for buildx-specific functionality:
cache_fromandcache_tofor using external cache storage backendsbuild_platformfor configuring multi-platform buildsoutputto control how/wherebuildxoutputs an image when runningpants package ::
JVM lock-files are more reliable, handling some cases of Coursier not including dependencies when expected.
Using shading_rules is now supported on deploy_jar targets defined in directories of the build root.
The Kotlin analyzer now runs using the Zulu JDK, to be able to run natively on Apple Silicon (arm64 macOS).
Dependency inference for Java now supports code using recently added syntax like sealed class & permits, switch expressions and yield.
Scala 3 is now supported when using the BSP in an IDE.
The new pants.backend.experimental.openapi.lint.openapi_format backend supports running openapi-format to format openapi_... targets.
The pants.backend.codegen.protobuf.python backend now supports:
protobuf_sourcestargets that are part of multiple Python resolves, likepython_resolve=parametrize("first-resolve", "second-resolve").[python-infer].ambiguity_resolution = "by_source_root"to infer dependencies without warnings or errors in more cases.
Version 24.4 of the protoc compiler is included as a known version (but is not the default). It can be selected by setting [protoc].version = "24.4". This version has native support for Apple Silicon (arm64 macOS).
Buf (pants.backend.codegen.protobuf.lint.buf) now correctly ignores buf.work.yaml, buf.lock, buf.gen.lock when running as a linter.
Deprecation: Using the platforms field on pex_binary is now deprecated. It is not a recommended way to build for platforms other than the local one, as it is a lossy abbreviation of the target platform and often leads to binaries that don't work. To replace this, either use complete_platforms to continue cross-compiling or use an environment matching the target platform.
Python tests can now be attempted multiple times, if they fail. This can help with limiting the impact of flaky tests. For instance, to require tests to fail three times before the overall pants test goal fails, set [test].attempts_default = 3. In this case, a very flaky test might fail on the first and second runs but finally pass on the third run, and thus the overall Pants invocation would pass.
Pants now uses version 2.1.148 of the PEX CLI by default, which includes support for Python 3.12. To use Python 3.12, you will need to set [python].pip_version to 23.2 or newer (or latest), and ensure all of your tools use resolves generated with this new version (Pants' built-in ones are not, yet).
Python interpreters provided by ASDF are now searched by default, in addition those provided by pyenv and on the system PATH, by being included in the [python-bootstrap].search_path option.
Many more "groups" of PyPI dependencies have default module mappings that guide how an import statement in code matches particular python_requirements. All packages from the azure-..., django-..., google-cloud-..., opentelemetry-instrumentation-..., oslo-... ecosystems are now matched to modules matching their usual conventions. In addition, packages starting with python-... now have that python- prefix removed by default: for instance, a package like python-example-name will be implicitly assumed to be imported like import example_name. The value of any module_mapping field can thus be tweaked (hopefully simplified!).
The [generate-lockfiles].diff option is now True by default, so running pants generate-lockfiles now pretty-prints any differences in the dependencies and their versions.
Additional features and fixes:
- Additional arguments can be passed to the IPython REPL, like
pants repl --shell=ipython :: -- -i helloworld/main.py. - Additional arguments can be passed to the underlying PEX invocations when building FaaS artifacts (AWS Lambda, Google Cloud Functions) via the new
pex3_venv_create_extra_argsfield. For instance, if dependencies have packaged files in unexpected locations, passingpex3_venv_create_extra_args=["--collisions-ok"]can side-step collision errors. - The
pip_versionfield no longer restricts the values, and so allows more flexibility when upgrading the PEX CLI if it supports newer Pip versions. - Running tests or
pants runa source file now make binaries provided bypython_requirementavailable for execution as normal processes (for instance, usingsubprocess.run(["name-of-binary", ...])). - The
PATHenvironment variable can now be set using[subprocess-environment].env_vars - Import statements within a
with contextlib.suppress(ImportError)context are now weak, and so no longer warn if not provided by a dependency. - Computing an aggregate coverage sandbox report is now more reliable in the presence of name collisions in files created via code-generation (including
relocated_files). - Pyright (
pants.backend.experimental.python.typecheck.pyright) now runs with an appropriately initialised environment and so runs more reliably (especially in transient CI environments, where "named caches" might not be preserved). - Ruff (
pants.backend.experimental.python.lint.ruff) now properly respects anyexcludeconfiguration fromruff.tomlorpyproject.toml.
The following code has been removed, because deprecations have expired:
- Support for building FaaS artifacts (AWS Lambda, Google Cloud Functions) via Lambdex has been fully replaced with the new
ziplayout: remove the[lambdex]section and itslayoutsetting frompants.toml. - Passing both
runtimeandcomplete_platformsto a FaaS target (python_aws_lambda_function,python_google_cloud_function) is now longer supported, ascomplete_platformsfully overridesruntime. - Dependency inference now only supports running with the new Rust-based parser. That is, the
[python-infer].use_rust_parseroption cannot be disabled and is now unnecessary.
The new pants.backend.experimental.tools.semgrep backend supports running Semgrep as a local linter on all files that Pants knows about (any target with a source or sources field). Configuration files are automatically discovered.
The experimental_test_shell_command target now obeys its environment field, and thus runs the command within the specified environment instead of the host.
The check goal runs terraform validate on terraform_modules targets by default, again. For modules for which this doesn't work and/or isn't desirable, use the new skip_terraform_validate field. (This restores the behaviour from Pants 2.17 and earlier, with the addition of the skip field, based on feedback about Pants 2.18.0 which switched to only validating terraform_deployments.)
terraform_deployment's var_files field now supports files in a different directory to the root_module.
Transitive dependencies are now automatically included, in addition to direct dependencies.
Processes, especially tests, can now be run with retries on failure using ProcessWithRetries (#20378). In addition, to support tests that have been retried multiple times, TestResult.from_fallible_process_result now accepts a tuple of multiple FallibleProcessResults, instead of just one. (#19760).
Some @rules may now need to pass canonical_name_suffix in preparation for "call by name". (#19755)
The new TransitivelyExcludeDependenciesRequest can be used to emulate transitive excludes (!!<address>). (#20080)
The TestResult.stdout and TestResult.stderr fields are now removed, in favour of stdout_bytes and stderr_bytes. (#19768)
For the full changelog, see the individual GitHub Releases for this series: https://github.com/pantsbuild/pants/releases