Change workflows to be reusable

[jacobbieker]

Detailed Description

GitHub just released reusable workflows, in beta, https://docs.github.com/en/actions/learn-github-actions/reusing-workflows

It seems like we can reduce a lot of duplication in our repos by moving workflows, especially the PyPi publishing one, linter, and most of the test workflows to here, and just calling them in those repos.

Context

Currently, we have the exact same workflows copied and pasted to all of our new nowcasting repos, and if we want to make changes to one, we have to manually copy and past it to all the other ones to propagate the change.

ToDos

• Add Linter workflow (see d6744ad)
• Add Test workflow (pytest and pydoctest)
• Add Release workflow (see Change workflows to be reusable #6 (comment))

[JackKelly]

Sounds like a great idea! Thanks for letting us know about GitHub's new reusable workflows beta :)

[flowirtz]

Great idea to unify this across repos, I think that would make things a lot easier.

I've seen a few different tools across repos now and I think it would make sense to maybe decide on a standard set of things we want to run on all or most python repos. Would this be a sensible starting point?

• flake8
• black
• isort
• pytest

[JackKelly]

Sounds great!

We could maybe add pydoctest?

[jacobbieker]

Yeah, sounds good! I'd also go with adding pydoctest, and potentially the release scripts as well? Since they are all the exact same and just copied and pasted as well

[flowirtz]

Sounds good! Maybe it makes sense splitting these out into three groups:

Linters

• isort
• flake8
• black
• pydocstyle

Test

• pytest
• pydoctest

Release

• the release script @jacobbieker mentioned

[jacobbieker]

Yeah! That sounds good

[JackKelly]

SGTM! Thanks!

[flowirtz]

FYI I made this work for the linting workflow we described above. We now have:

• A reusable linting workflow https://github.com/openclimatefix/.github/blob/189d6ba939d4d6a671ca5aad6b5bfd7bde487301/.github/workflows/python-lint.yml
• A repository already using this workflow (nowcasting_api) https://github.com/openclimatefix/nowcasting_api/blob/8290262dc54869ebb9c28003f9f7ad46e78519e0/.github/workflows/lint.yml
• Brief docs that describe how to (re-)use these workflows https://github.com/openclimatefix/.github/blob/189d6ba939d4d6a671ca5aad6b5bfd7bde487301/README.md

[JackKelly]

Nice!

[jacobbieker]

Awesome!

[jacobbieker]

I've added a general test and release ones now too, and updated SatFlow to use them: openclimatefix/satflow@3c2ebc6

[flowirtz]

Awesome! Thanks!

Can we close this issue?

[jacobbieker]

Yeah! Just updated almost all the nowcasting repos to use it now too, so changes here should then affect all our repos

[flowirtz]

Awesome! Thanks so much, @jacobbieker!