Skip to content

Commit

Permalink
Update docs site for version 2.22.0.dev0 (#198)
Browse files Browse the repository at this point in the history
  • Loading branch information
WorkerPants committed Apr 29, 2024
1 parent 60e91b4 commit 40f04a7
Show file tree
Hide file tree
Showing 167 changed files with 1,722 additions and 1,453 deletions.
18 changes: 8 additions & 10 deletions docs/docs/contributions/development/setting-up-pants.mdx
Expand Up @@ -61,32 +61,30 @@ To free up space, run `rm -rf src/rust/engine/target`.
Warning: this will cause Rust to recompile everything.
:::

## Step 3: (Optional) Set up a Git hook
## Step 3: Set up a pre-push Git Hook

We have a [Git hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) that runs some useful checks and lints when you `git commit`.
We have a [Git Hook](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) that runs some useful checks and lints when you `git push`. Running this locally can prevent easily avoidable CI failures such as whitespace or linting issues.

To install this, run:

```bash
$ build-support/bin/setup.sh
```

If you commit frequently as part of your workflow you may find it annoying to have these run every time, so installing this hook is not required.

You can manually run the pre-commit check with:
You can manually run the pre-push check with:

```bash
$ build-support/githooks/pre-commit
$ build-support/githooks/pre-push
```

The [Rust-compilation](./developing-rust.mdx) affecting `MODE` flag is passed through to the hooks, so to run the commit hooks in "debug" mode, you can do something like:
The [Rust-compilation](./developing-rust.mdx) affecting `MODE` flag is passed through to the hooks, so to run the githooks in "debug" mode, you can do something like:

```bash
$ MODE=debug git commit ...
$ MODE=debug git push ...
```

:::note How to temporarily skip the pre-commit checks
Use `git commit --no-verify` or `git commit -n` to skip the checks.
:::note How to temporarily skip the pre-push checks
Use `git push --no-verify` or `git push -n` to skip the checks.
:::

## Configure your IDE (optional)
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/contributions/development/style-guide.mdx
Expand Up @@ -13,7 +13,7 @@ Most of Pants' style is enforced via Black, isort, Docformatter, Flake8, and MyP

```bash
$ pants --changed-since=HEAD fmt
$ build-support/githooks/pre-commit
$ build-support/githooks/pre-push
```

:::note Tip: improving Black's formatting by wrapping in `()`
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/contributions/index.mdx
Expand Up @@ -74,8 +74,8 @@ Periodically, you will want to run MyPy and the autoformatters and linters:
# Format un-committed changes
$ pants --changed-since=HEAD fmt

# Run the pre-commit checks, including `check` and `lint`
$ build-support/githooks/pre-commit
# Run the pre-push checks, including `check` and `lint`
$ build-support/githooks/pre-push
```

See our [Style guide](./development/style-guide.mdx) for some Python conventions we follow.
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/contributions/releases/release-process.mdx
Expand Up @@ -70,7 +70,7 @@ The release commit is the commit that bumps the VERSION string. For `dev`/`a0` r

If this is the first dev release in a new series:

1. Create a new file in ` src/python/pants/notes`, e.g. create `src/python/pants/notes/2.9.x.md`.
1. Create a new file in `docs/notes`, e.g. create `docs/notes/2.9.x.md`.
1. Copy the title and template over from the prior release, e.g. `2.8.x.md`.

### Bump the VERSION
Expand Down
8 changes: 6 additions & 2 deletions docs/docs/python/integrations/aws-lambda.mdx
Expand Up @@ -7,10 +7,10 @@ Create a Lambda with Python code.

---

Pants can create a Lambda-compatible zip file from your Python code, allowing you to develop your Lambda functions and layers in your repository instead of using the online Cloud9 editor.
Pants can create a Lambda-compatible zip file or directory from your Python code, allowing you to develop your Lambda functions and layers in your repository instead of using the online Cloud9 editor.

:::note FYI: how Pants does this
Under-the-hood, Pants uses the [PEX](https://github.com/pex-tool/pex) project, to select the appropriate third-party requirements and first-party sources and lay them out in a zip file, in the format recommended by AWS.
Under-the-hood, Pants uses the [PEX](https://github.com/pex-tool/pex) project, to select the appropriate third-party requirements and first-party sources and lay them out in a zip file or directory, in the format recommended by AWS.
:::

## Step 1: Activate the Python AWS Lambda backend
Expand Down Expand Up @@ -58,6 +58,10 @@ Pants will use [dependency inference](../../using-pants/key-concepts/targets-and

You can optionally set the `output_path` field to change the generated zip file's path.

:::tip Using layout
Use [layout](../../../reference/targets/python_aws_lambda_function.mdx#layout) to determine whether to build a `.zip` file or a directory
:::

:::caution Use `resource` instead of `file`
`file` / `files` targets will not be included in the built AWS Lambda artifacts because filesystem APIs like `open()` would not load them as expected. Instead, use the `resource` and `resources` target. See [Assets and archives](../../using-pants/assets-and-archives.mdx) for further explanation.
:::
Expand Down
18 changes: 14 additions & 4 deletions docs/docs/python/integrations/google-cloud-functions.mdx
Expand Up @@ -7,10 +7,10 @@ Create a Cloud Function with Python.

---

Pants can create a Google Cloud Function-compatible zip file from your Python code, allowing you to develop your functions in your repository.
Pants can create a Google Cloud Function-compatible zip file or directory from your Python code, allowing you to develop your functions in your repository.

:::note FYI: how Pants does this
Under-the-hood, Pants uses the [PEX](https://github.com/pex-tool/pex) project, to select the appropriate third-party requirements and first-party sources and lay them out in a zip file, in the format recommended by Google Cloud Functions.
Under-the-hood, Pants uses the [PEX](https://github.com/pex-tool/pex) project, to select the appropriate third-party requirements and first-party sources and lay them out in a zip file or directory, in the format recommended by Google Cloud Functions.
:::

## Step 1: Activate the Python Google Cloud Function backend
Expand Down Expand Up @@ -59,6 +59,10 @@ Pants will use [dependency inference](../../using-pants/key-concepts/targets-and

You can optionally set the `output_path` field to change the generated zip file's path.

:::tip Using layout
Use [layout](../../../reference/targets/python_google_cloud_function.mdx#layout) to determine whether to build a `.zip` file or a directory
:::

:::caution Use `resource` instead of `file`
`file` / `files` targets will not be included in the built Cloud Function because filesystem APIs like `open()` would not load them as expected. Instead, use the `resource` / `resources` target. See [Assets and archives](../../using-pants/assets-and-archives.mdx) for further explanation.
:::
Expand Down Expand Up @@ -91,9 +95,15 @@ A better solution is to work with the dependencies to stop them from packaging f

## Step 4: Upload to Google Cloud

You can use any of the various Google Cloud methods to upload your zip file, such as the Google Cloud console or the [Google Cloud CLI](https://cloud.google.com/functions/docs/deploying/filesystem#deploy_using_the_gcloud_tool).
You can use any of the various Google Cloud methods to upload your zip file or directory, such as the Google Cloud console or the [Google Cloud CLI](https://cloud.google.com/functions/docs/deploying/filesystem#deploy_using_the_gcloud_tool).

You must specify the `--entry-point` as `handler`. This is a re-export of the function referred to by the `handler` field of the target.

You must specify the handler as `handler`. This is a re-export of the function referred to by the `handler` field of the target.
For example, if using `layout="flat"`:

```
gcloud functions deploy --source=dist/project/cloud_function --entry-point=handler --trigger-topic=<TOPIC> --runtime=python38 <FUNCTION_NAME>
```

## Advanced: Using PEX directly

Expand Down
6 changes: 6 additions & 0 deletions docs/docs/python/integrations/pyoxidizer.mdx
Expand Up @@ -7,6 +7,12 @@ Creating Python binaries through PyOxidizer.

---

:::caution (March 17, 2024) PyOxidizer is not actively maintained
As mentioned in [PyOxidizer issue #741](https://github.com/indygreg/PyOxidizer/issues/741) by indygreg, development on PyOxidizer is effectively stagnant. Please take this into consideration before using PyOxidizer in a new project, as this may affect PyOxidizer support within Pants as well.

> tl;dr shifting personal priorities have resulted in me de-prioritizing PyOxidizer (and other open source projects). At this point in time, the future of PyOxidizer is uncertain, possibly dead.
:::

PyOxidizer allows you to distribute your code as a single binary file, similar to [Pex files](../overview/pex.mdx). Unlike Pex, these binaries include a Python interpreter, often greatly simplifying distribution.

See our blog post on [Packaging Python with the Pants PyOxidizer Plugin](https://blog.pantsbuild.org/packaging-python-with-the-pyoxidizer-pants-plugin/) for more discussion of the benefits of PyOxidizer.
Expand Down
6 changes: 3 additions & 3 deletions docs/docs/python/overview/lockfiles.mdx
Expand Up @@ -195,7 +195,7 @@ pytest = "3rdparty/python/pytest.lock"

[pytest]
install_from_resolve = "pytest" # Use this resolve's lockfiles.
requirements =["//3rdparty/python:pytest"] # Use these requirements from the lockfile.
requirements = ["//3rdparty/python:pytest"] # Use these requirements from the lockfile.
```

Then set up the resolve's inputs:
Expand Down Expand Up @@ -242,7 +242,7 @@ Rather than repeat the same requirement in two different resolves, you can point

```toml title="pants.toml"
[pytest]
install_from_resolve = python-default
install_from_resolve = "python-default"
```

Of course, you have to ensure that this resolve does in fact provide appropriate versions of the tool.
Expand All @@ -251,7 +251,7 @@ As above, you will want to point `requirements` to the subset of targets represe

```toml title="pants.toml"
[pytest]
install_from_resolve = python-default
install_from_resolve = "python-default"

requirements = [
"//3rdparty/python#pytest",
Expand Down
Expand Up @@ -76,7 +76,6 @@ The `name` field defaults to the directory name. So, this target has the address
python_sources()
```

You can refer to this target with either `helloworld/greet:greet` or the abbreviated form `helloworld/greet`.
:::

:::note Use `//:tgt` for the root of your repository
Expand Down
Expand Up @@ -88,7 +88,7 @@ To read and write the cache to `/path/to/cache`, you will need to configure `pan

```toml
[GLOBAL]
remote_store_provider = "experimental-file"
remote_provider = "experimental-file"
remote_store_address = "file:///path/to/cache"
remote_cache_read = true
remote_cache_write = true
Expand Down
2 changes: 1 addition & 1 deletion docs/docs/using-pants/restricted-internet-access.mdx
Expand Up @@ -101,7 +101,7 @@ url_template = "https://my.custom.host/bin/protoc/{version}/{platform}/protoc.zi

For simplicity, we used the original value for `url_platform_mapping`, meaning that we set up our hosted URL to store the macOS x86 binary at `.../osx-x86_64/protoc.zip`, for example. You can override the option `url_platform_mapping` if you want to use different values.

Occasionally, new Pants releases will upgrade to new versions of these binaries, which will be mentioned in the "User API Changes" part of the changelog [https://github.com/pantsbuild/pants/tree/master/src/python/pants/notes](https://github.com/pantsbuild/pants/tree/master/src/python/pants/notes). When upgrading to these new Pants releases, you should download the new artifact and upload a copy to your host.
Occasionally, new Pants releases will upgrade to new versions of these binaries, which will be mentioned in the "User API Changes" part of the changelog [https://github.com/pantsbuild/pants/tree/main/docs/notes](https://github.com/pantsbuild/pants/tree/main/docs/notes). When upgrading to these new Pants releases, you should download the new artifact and upload a copy to your host.

:::note Asking for help
It's possible that Pants does not yet have all the mechanisms it'll need to work with your organization's specific networking setup, which we'd love to fix.
Expand Down
25 changes: 23 additions & 2 deletions docs/docs/using-pants/validating-dependencies.mdx
Expand Up @@ -9,6 +9,8 @@ Validating your code's dependencies.

Visibility rules are the mechanism by which to control who may depend on whom. It is an implementation of Pants's dependency rules API. With these rules a dependency between two files (or targets) may be set for entire directory trees down to single files. A target may be selected not only by its file path but also by target type, name, and tags.

NB: Visibility rules are applied on a low level, so they can inspect metadata provided directly on a target and via `__defaults__`, but anything applied via `overrides` is opaque to the visibility rules. For example, a rule that selects tagged targets can find tags provided directly on the target using the `tags` field (like `target(tags=[...])`) or via `__defaults__` (like `__defaults__({target:dict(tags=[...])})`) but any tags in `overrides` (like `target(overrides={"foo": dict(tags=[...])})`) cannot be inspected by the visibility rules.

To jump right in, start with [enabling the backend](./validating-dependencies.mdx#enable-visibility-backend) and add some rules to your BUILD files.

## Example visibility rules
Expand Down Expand Up @@ -203,13 +205,13 @@ __dependencies_rules__(

The selector and rule share a common syntax (refered to as a **target rule spec**), that is a dictionary value with properties describing what targets it applies to. Together, this pair of selector(s) and rules is called a **rule set**. A rule set may have multiple selectors wrapped in a list/tuple and the rules may be spread out or grouped in any fashion. Grouping rules like this makes it easier to re-use/insert rules from macros or similar.

:::note An empty selector (`{}` or `""`) will never match anything and is as such pointless and will result in an error.
:::note An empty selector (`{}` or `""`) will never match anything and is as such pointless and is ignored.
For every dependency link, only a single set of rules will ever be applied. The first rule set
with a matching selector will be the only one used and any remaining rule sets are never
consulted.
:::

The **target rule spec** has four properties: `type`, `name`, `tags`, and `path`. From the above example, when determining which rule set to apply for the dependencies of `src/a/main.py` Pants will look for the first selector for `src/a/BUILD` that satisifies the properties `type=python_sources`, `tags=["apps"]`, and `path=src/a/main.py`. The selection is based on exclusion so only when there is a property value and it doesn't match the target's property it will move on to the next selector; the lack of a property will be considered to match anything. Consequently an empty target spec would match all targets, but this is disallowed and will raise an error if used because it is conceptually not very clear when reading the rules.
The **target rule spec** has four (five for rules) properties: `type`, `name`, `tags`, `path`, and `action` (only rules consult `action`). The `action` is one of `allow` (default if not specified), `warn`, or `deny`. From the above example, when determining which rule set to apply for the dependencies of `src/a/main.py` Pants will look for the first selector for `src/a/BUILD` that satisifies the properties `type=python_sources`, `tags=["apps"]`, and `path=src/a/main.py`. The selection is based on exclusion so only when there is a property value and it doesn't match the target's property it will move on to the next selector; the lack of a property will be considered to match anything. Consequently an empty target spec would match all targets, but this is disallowed and will raise an error if used because it is conceptually not very clear when reading the rules.

The values of a **target rule spec** supports wildcard patterns (or globs) in order to have a single selector match multiple different targets, as described in [glob syntax](./key-concepts/targets-and-build-files.mdx#glob-syntax). When listing multiple values for the `tags` property, the target must have all of them in order to match. Spread the tags over multiple selectors in order to switch from _AND_ to _OR_ as required. The target `type` to match against will be that of the type used in the BUILD file, as the path (and target address) may refer to a generated target it is the target generators type that will be used during the selector matching process.

Expand Down Expand Up @@ -289,6 +291,25 @@ The previous example, using this alternative syntax for the selectors, would loo
)
```

Similarily, the rules may also be expressed using the dict syntax:

```python
(
( # Selectors block
...
),
( # Grouping rules for readability
( # Deny rules
{"path": "tests/**", "action": "deny"}, # No tests
{"path": "src/*/*/**", "action": "deny"}, # Nothing deeply nested
),
( # Allow rules
{"path": "*"}, # Allow everything else (allow is default action)
),
)
)
```

### Glob syntax

The visibility rules are all about matching globs. There are two wildcards: the `*` matches anything except `/`, and the `**` matches anything including `/`. (For paths that is non-recursive and recursive globbing respectively.)
Expand Down
7 changes: 6 additions & 1 deletion docs/docs/writing-plugins/overview.mdx
Expand Up @@ -148,7 +148,11 @@ This will generate [`python_requirement` targets](../python/overview/third-party

The target generator dynamically sets the version downloaded to match your current `pants_version` set in `pants.toml`. Pants's [dependency inference](../using-pants/key-concepts/targets-and-build-files.mdx) understands imports of the `pants` module and will automatically add dependencies on the generated `python_requirement` targets where relevant.

If you do not want your plugin requirements to mix with your normal requirements, it's often a good idea to set up a dedicated "resolve" (lockfile) for your plugins. See [Third-party dependencies](../python/overview/third-party-dependencies.mdx) for more information. For example:
:::caution Plugins resolve
We strongly recommend to set up a dedicated "resolve" (lockfile) for your plugins. Pants ships as a monolithic application with a pinned set of dependencies, which can make it hard to combine with your project's dependencies.
:::

To set up a dedicated resolve for your plugins, update your `pants.toml` as follows:

```python title="pants.toml"
[python]
Expand All @@ -172,6 +176,7 @@ pants-plugins = ["==3.9.*"]
```

Then, update your `pants_requirements` target generator with `resolve="pants-plugins"`, and run `pants generate-lockfiles`. You will also need to update the relevant `python_source` / `python_sources` and `python_test` / `python_tests` targets to set `resolve="pants-plugins"` (along with possibly the `interpreter_constraints` field).
See [Third-party dependencies](../python/overview/third-party-dependencies.mdx) for more information.

## Publishing a plugin

Expand Down

0 comments on commit 40f04a7

Please sign in to comment.