Change python packaging system from setuptools to Poetry

[cflynn3]

Closes #379

[connortann]

[ Edited to consolidate comments]

This looks promising. I think there are still hangovers of the previous packaging setup that need to be migrated before this is ready for merging.

Project metadata

As I understand, python poetry is a replacement for setuptools. To avoid being in a messy state with duplication, I think if we switch to Poetry we should also remove the previous setup and use poetry to build the project.

This would mean moving the package metadata from setup.cfg to pyproject.toml , and also editing the relevant github CI workflows that do the build and deploy steps.

Versioning

I think we need a clear plan for versioning. Python Poetry manages the version using pyproject.toml, which is inconsistent with the current setup using setuptools-scm. If we use Poetry, we need to ensure it correctly picks up the version number from version control.

This could be one Poetry-compatible way of ensuring we still have PEP-440 compliant version strings that are single-sourced from version control:

https://pypi.org/project/poetry-dynamic-versioning/

Deployment

If we switch to using Poetry, we need to ensure that the deploy scripts used internally in bp to our scientific computing environments work correctly. At present, there are setuptools-specific build steps that this will break.

Outstanding concerns

• Complete migration away from setuptools
• Single source of PEP-440 version from git tags
• Appropriately open/restrictive version constraints for users
• CI build step using Poetry
• CI publish step using Poetry
• Ensure bp-internal computing environment build scripts work
• Ensure poetry not in same python env as resqpy

[connortann]

As per comments above, I think there are still setuptools remnants and the migration to poetry is not complete.

[connortann]

I think this "test" step should be unchanged - as the previous step builds a python wheel, we should be able to install that wheel in the usual way with pip in this step.

[connortann]

Should we use the existing pip install resqpy --find-links=dist/ here, to ensure we are running the tests against the built python wheel?

This step was originally intended to ensure that the python wheel functions correctly and can be pip-installed by end users.

[connortann]

FYI, I've made a PR on the "Install Poetry" GitHub Action to allow it to install plugins as well. However, would recommend waiting until Poetry 1.2 is officially released before using this.

snok/install-poetry#69

[connortann]

LGTM! Thanks for addressing all of my comments. @andy-beer , FYI this changes the release pipeline significantly so would be good to ensure the release process still works after this is merged.