ci: build/push python packages workflow, added

[consideRatio]

This workflow is meant to help us with automation to publish to PyPI. It won't work without configured github secrets within this repo for the PyPI project's dask-gateway and dask-gateway-server. For that, we would need to take the steps outlined in #484 which I copy paste here:

• PyPI release preparations
  We need deployment keys for both PyPI projects available to this repo's GitHub workflows (dask-gateway, dask-gateway-server). This can only be done by a PyPI project maintainer, currently that must be @jcrist.
  • Visit https://pypi.org/manage/account/token/ and create a token for one project
  • Visit https://github.com/dask/dask-gateway/settings/secrets/actions and create a GitHub actions secret including the package name, like pypi_token__dask_gateway and pypi_token__dask_gateway_server.
    It would be good to have an additional owner from the Dask team added to those PyPI projects.
  • Add one or more owner/maintainer from the Dask team to both projects, direct links are: dask-gateway on PyPI, dask-gateway-server on PyPI.

[jacobtomlinson]

This looks great. I'll leave this until we get unblocked on the credentials.

[jcrist]

Thanks Jacob and Erik, what are y'all's pypi usernames? I'll add you right now.

[martindurant]

@jcrist , that's great news. Please also update the conda-forge feedstock.

[jacobtomlinson]

Awesome thanks @jcrist. I am jacob.tomlinson and Erik is consideRatio. If you could merge conda-forge/dask-gateway-feedstock#22 also that would be great!

[jacobtomlinson]

Thanks for updating PyPI @jcrist. I've added @martindurant too.

[jcrist]

I'm not sure what your wheel building automation plans are, but note that dask-gateway-server has some native (go) code that will need to be fixed and checked by auditwheel. Previously I had a hacky buildscript I'd run locally for the osx & linux builds, but if I were to redo this today I'd just use cibuildwheel and be done with it. Apologies if y'all already know all this.

[consideRatio]

@jcrist thank you for clarifying this and suggesting a strategy to go about it, and thank you for your trust!

I have much to learn about Go but this seems like a good reason to learn some more. I'll rework this GitHub workflow as needed to make things work as they should.

Jim Is the hacky buildscript you had version controlled in this repo or something you could share in a gist or similar?

[jcrist]

The scripts are here: https://github.com/dask/dask-gateway/tree/main/dask-gateway-server/build_tools, but things should build fine with cibuildwheel, provided the golang compiler is added to the build images before running the setup.py. This could be done with https://cibuildwheel.readthedocs.io/en/stable/options/#before-all perhaps. Note that dask-gateway-server doesn't support windows, so a bash script for linux/osx should be sufficient.

[consideRatio]

Ah excellent, I'm confident I can come up with something sensible with this very helpful guidance!

I opened #496 to track the need to fix this properly, something I'd be happy to do.