Testing Wheels generated by Pantsbuild

[cognifloyd]

To generate all of the st2 wheels, run this from the root of the StackStorm/st2 repo:

./pants package ::

Here, :: is like a recursive glob so that pants will package everything it knows about (based on BUILD metadata).

To build a subset of the wheels, replace :: with the path where it is defined. To build just st2common and the orquesta_runner, do:

./pants package st2common contrib/runners/orquesta_runner

Pants puts all packaged artifacts under the dist/ directory.

The package goal

The package goal (where goals refer to the thing(s) you want to do with pants: ./pants <goal(s)> <path(s)>) documentation is here:

https://www.pantsbuild.org/v2.16/docs/python-package-goal

The package goal makes various kinds of installable or distributable artifact. The only packageable artifacts in st2 are python_distribution artifacts, which includes sdists and wheels. In particular, it will generate wheels for all the components (defined in st2*/ like st2commom and st2api) and the runners (defined in contrib/runners/*_runner) when given the :: recursive target glob.

python_distribution

We use python_distribution in BUILD metadata to tell pants about each wheel we want it to generate. Documentation about building python distributions is here:

https://www.pantsbuild.org/v2.16/docs/python-distributions

Defining this BUILD metadata for st2 was the focus of these PRs:

• pants: Add st2_*_python_distribution() macros to pants-plugins/macros.py st2#5901
• Pants: add python_distribution BUILD metadata for runners st2#5907
• Pants: add python_distribution BUILD metadata for st2* components st2#5909
• Pants: add dependencies to the st2* python_distributions (except st2common/st2tests) st2#5928
• Pants: add dependencies to the st2tests python_distributions st2#5929

Dependencies defined in other PRs also have an affect on what gets included in the wheel.

Testing the wheels

Open or install the sdists and/or wheels. Possible things to check:

• Is everything in the wheel or sdist that should be there? Python? conf files?
• Are there any files in the wrong place?
• Is the metadata sane?
• Pants generates setup.py: does it look sane?
• Does it install?
• Anything else you can think of to inspect.

As you're looking at it, what kinds of tests would be good for the wheels? Maybe even write some tests for pants to run via pytest (make the python_test target depend on the relevant python_distribution target to get the wheel in the test sandbox.

What to do when the wheel is missing something

When something is missing from a wheel, look for something else that is included in the wheel that might need a manual dependency on whatever is missing. If there is nothing obvious inside the wheel that needs the dependency (eg because it is a public API in st2common) then add the dependency to the python_distribution metadata.

What to do when files are in the wrong place in the wheel

I suspect this happens when the files are owned by a resources or files target when they should probably be the other one. We might also need to use relocated_files to move them similar to how I added LICENSE to all the wheels.