Configure pants_ignore and bump pants to v2.14.0rc1

[cognifloyd]

Background

This is another part of introducing pants, as discussed in the TSC Meetings on 12 July 2022, 02 Aug 2022 and 06 Sept 2022. Pants has fine-grained per-file caching of results for lint, fmt (like black), test, etc. It also has lockfiles that work well for monorepos that have multiple python packages. With these lockfiles CI should not break when any of our dependencies or our transitive dependencies release new versions, because CI will continue to use the locked version until we explicitly relock with updates.

To keep PRs as manageable/reviewable as possible, introducing pants will take a series of PRs. I do not know yet how many PRs; I will break this up into logical steps with these goals:

• introduce pants to the st2 repo, and
• teach some (but not all) TSC members about pants step-by-step.

Other pants PRs include:

• Initialize pantsbuild #5713
• Configure pantsbuild source roots #5724
• Enable pantsbuild language backends #5725
• Adjust .gitignore to work better with pantsbuild #5726
• Add GHA workflow to validate pants BUILD files #5732

Overview of this PR

As described in #5732, we need to run ./pants tailor :: (ie step5 to generate metadata (BUILD) files.

If we ran that now, tailor would (a) hit some deprecation messages, and (b) add some metadata that we don't want.

So, this PR adjusts the pants config enough for tailor to do a satisfactory first pass on the st2 repo. I've divided these changes into 3 commits:

• bump pants version to 2.14.0rc1
• make pants ignore setup.py and dist_utils.py
• make pants tailor ignore python_requirements targets

Upgrade to pants 2.14.0rc1

pants 2.13 has deprecations that require explicit (temporary) config in pants.toml. But 2.14 defaults to the newer setting and deprecates the option in pants.toml. So, if we just start with 2.14, we can avoid the options churn.

Also, the latest work on the pants PoC branch used a new feature from a dev release of pants 2.14. But, I don't want to rely on dev releases, so I used 2.13.0rc0 in #5713. Since then 2.13.0 was released and 2.14 had two rc releases. So, it is safe enough to bump to 2.14 now.

I hope to finish bumping to the final 2.14.0 release as soon as it is released.

[GLOBAL].pants_ignore (in pants.toml)

Both of the other commits add entries to [GLOBAL].pants_ignore. This setting is described here:

Paths to ignore for all filesystem operations performed by pants (e.g. BUILD file scanning, glob matching, etc). Patterns use the gitignore syntax (https://git-scm.com/docs/gitignore). The pants_distdir and pants_workdir locations are automatically ignored. pants_ignore can be used in tandem with pants_ignore_use_gitignore; any rules specified here are applied after rules specified in a .gitignore file.

So, we use this to make sure pants ignores some files even though we have them checked into git.

In particular, we have pants ignore:

• setup.py and dist_utils.py:
  • because pants will handle building the wheel and generating setup.py if needed. So, we will be able to completely remove these files once we're done moving to pants.
  • because dist_utils.py has multiple copies across the repo, if we didn't ignore them, we would have to add explicit dependency metadata because pants can't tell which dist_utils setup imports.
• various requirements.txt files:
  • because pants will generate the setup.py files with only the requirements that each module actually imports (or where we add an explicit dep in a BUILD file if pants can't infer the dep. So, pants will obsolete all of our manual dep management (in-requirements.txt) and the script that generated the deps file for pip (requirements.txt).

If we did not have pants ignore these, ./pants tailor would add python_sources and python_requirements targets in BUILD files that are problematic in that we would have to add extra metadata to avoid ambiguous imports errors.

[rush-skills]

👍🏼