Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adopt ci gha #4503

Merged
merged 20 commits into from
Jun 17, 2022
Merged

adopt ci gha #4503

merged 20 commits into from
Jun 17, 2022

Conversation

bjlittle
Copy link
Member

@bjlittle bjlittle commented Jan 14, 2022 •
edited
Loading

🚀 Pull Request

Description

This PR replaces our use of cirrus-ci and nox with gha (github actions)... finally 🥳

Although I introduced the use of nox, I'm personally not seeing the benefit and gha is easily filling that gap (plus gha is well understood by the community, whereas that isn't necessarily the case for nox and indeed cirrus-ci)

This PR is a side-effect of the work on #4480

Still to-do:

  • add cartopy caching
  • add conda environment caching
  • add pre-commit job
    • the gha pre-commit/action has been deprecated and replaced by the pre-commit.ci service, which is already activated on this repository and part of the current CI workflow
  • to nox or not/partially to nox
  • update documentation
  • add whatsnew entry

Consult Iris pull request check list

@bjlittle bjlittle marked this pull request as draft January 14, 2022 12:34
@bjlittle
Copy link
Member Author

bjlittle commented Jan 14, 2022 •
edited
Loading

@trexfeathers is nox an integral part of your vision for the benchmarking CLI and CI workflow?

I'm keen to see if we can drop nox, but what's your preference/our options?

@trexfeathers
Copy link
Contributor

trexfeathers commented Jan 14, 2022 •
edited
Loading

@trexfeathers is nox an integral part of your vision for the benchmarking CLI and CI workflow?

I'm keen to see if we can drop nox, but what's your preference/our options?

Yes

And I'd argue in Nox's/Tox's favour even without benchmarking. Codifying the steps one expects a user to take to properly set up an environment or run several steps is hugely appealing. Ever since you introduced it I've worked so much more smoothly by knowing I'm running the same steps as the CI runs ❤❤❤.

I don't know if you've seen this, but iris-esmf-regrid's method for refreshing the lockfiles is (IMO) much simpler and more flexible than Iris' set up, all thanks to Nox.

@trexfeathers
Copy link
Contributor

@trexfeathers is nox an integral part of your vision for the benchmarking CLI and CI workflow?

I'm keen to see if we can drop nox, but what's your preference/our options?

If you can implement another way of codifying environment creation/update that can be shared by GHA/other scripts (e.g. ASV)/users I would certainly be interested.

@trexfeathers
Copy link
Contributor

  • add pre-commit job
  • the gha pre-commit/action has been deprecated and replaced by the pre-commit.ci service, which is already activated on this repository and part of the current CI workflow

Big fan of this. I had wondered before but assumed I was missing something!

@trexfeathers
Copy link
Contributor

  • add conda environment caching

Some inspiration if needed. I think GitHub can share caches even between different workflows too(?), which would be neat.

@bjlittle
Copy link
Member Author

bjlittle commented Jan 14, 2022 •
edited
Loading

@trexfeathers Awesome, thanks.

Could I suggest flipping to tox? Note that, https://github.com/tox-dev/tox-conda is a thing.

That said, I think we should discuss this further, but not let it be a blocker for this PR.

For the moment, I'm going to reinstate the noxfile.py but only benchmarking will utilise it from CI, as-is. Lovely to hear that you're getting genuine benefit from nox, that's super useful to know 😄

Sound reasonable?

@trexfeathers
Copy link
Contributor

trexfeathers commented Jan 14, 2022 •
edited
Loading

Could I suggest flipping to tox? Note that, https://github.com/tox-dev/tox-conda is a thing.

Sure, if you think it's important. Is there something wrong with Nox that I'm unaware of? Making the switch will obviously take some work.

For the moment, I'm going to reinstate the noxfile.py but only benchmarking will utilise it from CI, as-is. Lovely to hear that you're getting genuine benefit from nox, that's super useful to know 😄

Sound reasonable?

I feel like I might be at odds with the team here. I feel it would be a serious backwards step to remove the ability for users, and other scripts, to recreate the same conditions that CI uses. Having everything calling to the same place for the last year has saved time in replication as well as avoiding a risk of divergence.

I'm definitely getting the feeling there's something fundamentally wrong with Nox that I haven't understood 😅. Having used it with GHA I don't see how it was ever a blocker to making this change?

@bjlittle bjlittle force-pushed the adopt-gha branch 5 times, most recently from 5499c1d to 23afe7d Compare January 15, 2022 00:07
@trexfeathers
Copy link
Contributor

Summary of an offline discussion with @bjlittle regarding Nox/Tox/something else:

  • Removal of Nox was a good tool to prompt a discussion.
    • There's no urgency to this PR being finished.
    • This PR does not depend on Nox's removal.
  • We agreed on the merit of having some common test setup / test running script(s) - not isolating this code within GHA files.
  • Discussion so far has highlighted everyone's different workflows. Documentation should acknowledge and allow for this.
    • @bjlittle almost always relies on CI to test any changes, so has got little value from Nox.
    • @trexfeathers regularly runs offline tests, and therefore appreciates Nox providing common conditions with CI.
    • We know @rcomer has had trouble in the past with Nox appearing to be the de-facto way of running tests; how to run a subset of the tests?
  • Nox or Tox:
    • Nox is easier and more accessible by being Pythonic.
    • Tox has more community acceptance - important for encouraging contributions.
    • Tox could also interface with Mamba (I think?), allowing parallel package download.
      (We don't benefit from faster resolution since we use lock files).
    • @trexfeathers: it doesn't matter if something is superior if few people are familiar with it. Consider switching to Tox.

@bjlittle
Copy link
Member Author

bjlittle commented Jan 17, 2022 •
edited
Loading

@trexfeathers Lovely summary, thanks 🍻

We can probably discuss this all in more detail on the dedicated GH Discussion Trial GitHub Actions vs Cirrus CI

Guilty as charged though. I am using this PR to "poke the bear", perhaps unnecessarily, but I think it's a healthy thing to do i.e., question how we use our current tooling and gauge if we're all roughly heading in the same direction.

In general, I think we are... but it's very worthwhile to make that an explicit agreement, but also clarify (in the docs) our tooling choices and explain how we support different developer workflows. That would be an awesome outcome.

And yeah, just to clarify, there is absolutely no rush to review/merge this PR (or take it out of draft atm)

@trexfeathers trexfeathers mentioned this pull request Apr 14, 2022
Merged
@bjlittle
Copy link
Member Author

bjlittle commented Jun 14, 2022 •
edited
Loading

I'm going to see if I can show this PR some love, given the current difficulties with cirrus-ci; the writing is clearly on the wall AFAIC

I'll rebase, but also reinstate the current use of nox. That will reduce the scope of this PR, and we can save the nox vs tox discussion for another day.

@pp-mo pp-mo mentioned this pull request Jun 14, 2022
Closed
@bjlittle bjlittle force-pushed the adopt-gha branch 2 times, most recently from 9a6c96c to 1930d93 Compare June 15, 2022 13:40
@bjlittle bjlittle marked this pull request as draft June 16, 2022 12:31
@bjlittle
Copy link
Member Author

@trexfeathers and @pp-mo Without polishing too much... I think this is looking less intimidating thanks to the composite actions.

Back to you guys 👍

@bjlittle bjlittle marked this pull request as ready for review June 16, 2022 15:13
trexfeathers
trexfeathers requested changes Jun 16, 2022
View reviewed changes
Copy link
Contributor

@trexfeathers trexfeathers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, nearly there

.github/workflows/composite/iris-data-cache/action.yml Outdated Show resolved Hide resolved
.github/workflows/composite/nox-cache/action.yml Outdated Show resolved Hide resolved
@trexfeathers
Copy link
Contributor

‼ Almost forgot: we need to chose a solution to the problem that GHA's cannot spawn other GHA's - a problem for our automatic lock-file updates. More detail at bjlittle/geovista#49

@bjlittle bjlittle removed the request for review from pp-mo June 17, 2022 09:33
@trexfeathers
Copy link
Contributor

Thanks @bjlittle. Into another brave new world...

@trexfeathers trexfeathers merged commit 724f1b9 into SciTools:main Jun 17, 2022
@pp-mo pp-mo mentioned this pull request Jun 17, 2022
Merged
@bjlittle bjlittle deleted the adopt-gha branch June 17, 2022 12:31
@bjlittle bjlittle mentioned this pull request Jun 21, 2022
Merged
tkknight added a commit to tkknight/iris that referenced this pull request Jun 21, 2022
@tkknight
Merge remote-tracking branch 'upstream/main' into dark_theme
f5005bd 
* upstream/main: (22 commits)
  Scripted Logo Generation, take two (SciTools#3935)
  adopt ci gha (SciTools#4503)
  pinned pydata-sphinx-theme until the docs are dark mode friendly. (SciTools#4796)
  Retire use of the New labels. (SciTools#4801)
  Add mesh details to cube printout. (SciTools#4778)
  `PolarStereographic` implementation (SciTools#4773)
  Fix whatsnew (SciTools#4784)
  Updated environment lockfiles (SciTools#4782)
  Bump peter-evans/create-pull-request from 4.0.3 to 4.0.4 (SciTools#4783)
  Redirect whatsnew to latest (SciTools#4727)
  Updated environment lockfiles (SciTools#4775)
  Cubelist contain only cubes -- resurrected (SciTools#4767)
  Developer documentation on available testing conveniences (SciTools#4600)
  Prevent truncation of cell method descriptions with nested brackets (SciTools#4436)
  Increase speed of iris.analysis.trajectory.interpolate with linear interpolation (SciTools#4366)
  Test warnings (SciTools#4764)
  don't use np.float or np.int (SciTools#4763)
  CPerf and SPerf Benchmark Fixes (SciTools#4758)
  Credit @rcomer on PR SciTools#4755 review (SciTools#4761)
  Permit fast percentile with mdtol=0 (SciTools#4755)
  ...
lbdreyer added a commit to lbdreyer/iris that referenced this pull request Sep 14, 2022
@lbdreyer @bjlittle
@github-actions @web-flow @trexfeathers @dependabot @rcomer @pre-commit-ci @pp-mo @wjbenfold @dennissergeev @tkknight @schlunma @stephenworsley @scitools-ci @krikru @ESadek-MO
Qa issue (#9)
edd6c33 
* 3.2 version and whats new (#4559)

* Reset whats new (#4563)

* reset whats new

* Specify minor in whats new

* update trove classifiers (#4564)

* Update version to 3.3.dev0 (#4565)

* Updated environment lockfiles (#4567)

Co-authored-by: Lockfile bot <noreply@github.com>

* New tool-agnostic ASV environment management (#4571)

* New tool-agnostic ASV env management.

* Benchmarking only build the latest Python.

* Increase benchmark accuracy by increasing rounds.

* Fix ASV rounds mistake.

* ASV clearer use of _interpolate_commands.

* adopt dependabot GHA (#4568)

* Bump peter-evans/create-pull-request from 3.8.2 to 3.12.1 (#4577)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3.8.2 to 3.12.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/052fc72b4198ba9fbc81b818c6e1859f747d49a8...f22a7da129c901513876a2380e2dae9f8e145330)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/stale from 4.0.0 to 4.1.0 (#4575)

Bumps [actions/stale](https://github.com/actions/stale) from 4.0.0 to 4.1.0.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v4.0.0...v4.1.0)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/script from 4 to 5.1.0 (#4576)

* gha: lockfiles labels and auto-pr details (#4578)

* Fix refresh lockfile worrkflow pull request title (#4579)

* New tool-agnostic ASV environment management (#4571)

* New tool-agnostic ASV env management.

* Benchmarking only build the latest Python.

* Increase benchmark accuracy by increasing rounds.

* Fix ASV rounds mistake.

* ASV clearer use of _interpolate_commands.

* Fix load_http bug, extend testing, and note to docs (#4580)

* Fix opendap bug, add docs and extra testing

* Add whats new entry

* Update docs/src/whatsnew/3.2.rst

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>

* Add warning box

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>

* Loading Benchmarks (#4477)

* Synthetic FF PP NetCDF and loading benchmarks.

* fix test (#4585)

* [pre-commit.ci] pre-commit autoupdate (#4587)

* [pre-commit.ci] pre-commit autoupdate

updates:
- [github.com/psf/black: 21.12b0 → 22.1.0](https://github.com/psf/black/compare/21.12b0...22.1.0)
- [github.com/asottile/blacken-docs: v1.12.0 → v1.12.1](https://github.com/asottile/blacken-docs/compare/v1.12.0...v1.12.1)
- [github.com/aio-libs/sort-all: v1.1.0 → v1.2.0](https://github.com/aio-libs/sort-all/compare/v1.1.0...v1.2.0)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Finalise whatsnew and version string update (#4588)

* docs linkcheck skip (#4590)

* Add missing commit to v3.2.x and update version number (#4593)

* fix trove classifier (#4324)

* update version to 3.2. post release

Co-authored-by: Bill Little <bill.james.little@gmail.com>

* Bump actions/script from 5.1.0 to 6 (#4586)

Bumps [actions/script](https://github.com/actions/script) from 5.1.0 to 6.
- [Release notes](https://github.com/actions/script/releases)
- [Commits](https://github.com/actions/script/compare/v5.1.0...v6)

---
updated-dependencies:
- dependency-name: actions/script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Yaml fixes + clarifications. (#4594)

* Yaml fixes + clarifications.

* Update .github/workflows/stale.yml

Co-authored-by: Ruth Comer <10599679+rcomer@users.noreply.github.com>

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>
Co-authored-by: Ruth Comer <10599679+rcomer@users.noreply.github.com>

* Overnight benchmarks (#4583)

* Utility class in netcdf loader should not be public. (#4592)

* Utility class in netcdf loader should not be public.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Rename container for better clarity.

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Stop using nc_time_axis.CalendarDateTime (#4584)

* stop using CalendarDateTime

* update dependencies

* stronger tests for _fixup_dates

* add whatsnew

* Update 3.2.rst whats new with section for bug fixes (#4604)

* Bump peter-evans/create-pull-request from 3.12.1 to 3.13.0 (#4607)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3.12.1 to 3.13.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/f22a7da129c901513876a2380e2dae9f8e145330...89265e8d24a5dea438a2577fdc409a11e9f855ca)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Support false-easting and false-northing when loading Mercator-projected data (#4524)

* Add extra pieces to get false easting and northing

* tests

* Add extra test file ref and cml

* lib/iris/tests/results/netcdf/netcdf_merc_false.cml

* Update cml for new test data

* Bump test data version for cirrus

* What's new

* Bump peter-evans/create-pull-request from 3.13.0 to 3.14.0 (#4608)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3.13.0 to 3.14.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/89265e8d24a5dea438a2577fdc409a11e9f855ca...18f7dc018cc2cd597073088f7c7591b9d1c02672)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Revert plotting-vs-y (#4601)

* revert plotting-vs-y

* whatsnew

* Bump actions/stale from 4.1.0 to 5 (#4612)

Bumps [actions/stale](https://github.com/actions/stale) from 4.1.0 to 5.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v4.1.0...v5)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 2 to 3 (#4611)

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Use crs.globe in _crs_distance_differentials() (#4605)

* Use crs.globe in _crs_distance_differentials(), add a relevant test, update whatsnew

* add a ref to the issue and PR

* add dennis (#4614)

* Final offline benchmark migration (#4562)

* Overnight benchmarks remove ambiguity between file and commit names. (#4620)

* purge deploy key (#4615)

* Remove no_clobber task from Refresh lockfiles Action (#4618)

* Remove no_clobber task from Refresh lockfiles Action

* review actions - add extra infor to pr message

* Scalar Scatter Plot (#4616)

* add failing test

* pass test

* initial review actions

* test 2nd arg scalar

* whatsnew

* Updated environment lockfiles (#4624)

Co-authored-by: Lockfile bot <noreply@github.com>

* Overnight benchmarks - find a valid issue assignee (#4627)

* Overnight benchmarks find a valid issue assignee.

* Overnight benchmarks safer check for a valid issue assignee.

* Votable Issues (#4617)

* Create test2.yml

* gha test

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* Update refresh-votable-issues.yml

* initial

* tidy

* updated whatsnew

* removed defunct ext links

* Update docs/src/votable_issues.rst

Co-authored-by: Ruth Comer <10599679+rcomer@users.noreply.github.com>

* added suggestion to subscribe to voted issues.

* Renamed votable to voted.  Layout tweaks to the voted issues page.

Co-authored-by: Ruth Comer <10599679+rcomer@users.noreply.github.com>

* Avoid dimensionality mismatch between data array and CF variables. (#4610)

* Sperf & Cperf Benchmarks (#4621)

* Update version to 3.2.1 and fix whatsnew (#4635)

* Improve Small Benchmark Accuracy (#4636)

* Remove another memory benchmark that's smaller than the noise level.

* Fixed import benchmarks.

* Updated GHA frequency and table name for Voted Issues (#4641)

* updated gha frequency and table name

* added new ignore for linkcheck

* use conda-lock -k explicit (#4644)

* Bump actions/cache from 2 to 3 (#4650)

Bumps [actions/cache](https://github.com/actions/cache) from 2 to 3.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](https://github.com/actions/cache/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump peter-evans/create-pull-request from 3.14.0 to 4 (#4652)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 3.14.0 to 4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/18f7dc018cc2cd597073088f7c7591b9d1c02672...d6d5519d05f5814158ef015b8448f2f74648c421)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix aggregated_by mdtol/masked constant problem (#4246)

* add failing test

* pass test

* whatsnew

* remove redundant copy call

* Make iris.tests.stock.simple_1d respect bounds arg (#4658)

* Make iris.tests.stock.simple_1d respect bounds arg

* What's new

* What's new

* Improved check for valid user in overnight BM run. (#4659)

* Preserve position when replacing Axes with GeoAxes (#4273)

* preserve position

* add test

* comment to clarify test

* whatsnew

* use original position

* Lazy Percentile Aggregator (#3901)

* implement lazy percentile

* fix map_complete_blocks tests

* fix legacy masked type no mask tests

* add test for map_complete_blocks on dask array

* handle axis expressed as sequence

* fix docstring

* add lazy_aggregate method and mdtol handling

* reshape for multi-dim collapse

* mdtol handling for added trailing dimension

* update aggregator instance docstring

* simplify array reshape

* factor out array reshaping as decorator

* reduce repetition in tests

* PERCENTILE multi-axis tests

* add tests for array wrangling

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* py3.7

* test_PERCENTILE.py tweaks

* cut down post_process tests

* clarify wrapper tests

* whatsnew

* revert py3.7 change

* Update lib/iris/analysis/__init__.py

Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>

* review: remove docstring kwarg; define _real_percentile

* review: blank line for note

* review: remove scalar_percent check

* review: test__axis_to_single_trailing.py

* review: simplify structure

* review: _percentile docstring args

* review: remove axis description from _percentile docstring

* review: don't use rollaxis

* review: generalize LazyMixin to also work for non-lazy.  Rename

* review: move Test_lazy_aggregate tests into MaskedCalcMixin

* review: generalise MaskedCalcMixin

* review: remove stray as_lazy_data calls

* review: use mixins for Test_aggregate

* review: move multi axis tests to their own mixin

* review: use MultiAxisMixin in both lazy classes

* review: add tests for fast real aggregation

* docstrings

* review: test clarity tweaks

* review: test alphap, betap passed to scipy

* review: use assertEqual

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>

* Fix nearest_neighbour_index for edge case where requested point is float and bounds are int (#4245)

* add failing test

* pass test

* add whatsnew

* Include note to strip out Artifactory (#4666)

* Include note to strip out Artifactory

* What's new

* update black (#4668)

* Fixed cube arithmetic for cubes with meshes. (#4651)

* Use separate cml files for tests/unit/analysis/maths/test_slice multiple passes.

* Roughly functional, but lots of tests to fix.

* Fix

* Fix mockist test for new-style Resolve.  Mesh support not yet tested.

* Fix _create_prepared_item when points/bounds passed in.

* --no-edit

* Simplify prepared-item usage.

* Added tests

* Added whatsnew.

* Reference Iris issues for outstanding problems.

* Small simplifications.

* Updated statement on cube arithmetic with meshes.

* Fix use of 'MathsAddOperationMixin'; add coord-mismatch tests on meshcubes.

* Review changes.

* Hack test which was failing.

* Updated environment lockfiles (#4655)

Co-authored-by: Lockfile bot <noreply@github.com>

* 'Deep' benchmarks to catch scaling with field count (#4654)

* 'Deep' benchmarks to catch scaling with field count

* Better sized benchmarks

* What's new

* Update benchmarks/benchmarks/load/__init__.py

Co-authored-by: Martin Yeo <40734014+trexfeathers@users.noreply.github.com>

Co-authored-by: Martin Yeo <40734014+trexfeathers@users.noreply.github.com>

* Automatic reversal of DimCoord bounds (#4466)

* first pass at ensuring contiguous bounds

* reduce scope and fix old tests

* bounds setter tests

* test contiguity preserved

* check bounds within reverse tests

* add comment; merge test

* simpler approach

* revert now redundant changes

* update cml

* cell equality to ignore bound order

* update cml

* move merge test to integration

* review: simple actions

* review: split up cube reverse tests

* review: check bounds in all cube reverse tests

* whatsnew

* review: reduce test repetition

* tweak whatsnew

* blank line

* Updated environment lockfiles (#4677)

Co-authored-by: Lockfile bot <noreply@github.com>

* Updated environment lockfiles (#4678)

Co-authored-by: Lockfile bot <noreply@github.com>

* Reorder Constraints sections of user guide, add time bounds example (#4656)

* move constraint detail to subsetting cube, add example of constraining bound

* Add whats new

* Bump peter-evans/create-pull-request from 4.0.0 to 4.0.1 (#4675)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/d6d5519d05f5814158ef015b8448f2f74648c421...f1a7646cead32c950d90344a4fb5d4e926972a8f)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* tweak whatsnew (#4682)

* Implement Pydata theme (#4661)

* new theme baseline

* update

* added discussion directive and whatsnew

* fixed bullet point indentation

* updated NOX_CACHE_BUILD to force update the nox cache

* added pr ref

* Update lock-file generation in response to conda-lock change

* Make lockfile shareable and bump nox cache

* MAin branch updates to latest.rst (from dev.rst)

* change references of dev.rst to latest.rst

* corrected bullet points indentation to render correctly.

* updated

* update whatsnew

* Review action and fixed typo for link

* update lock

* updated from main

Co-authored-by: Will Benfold <will.benfold@metoffice.gov.uk>
Co-authored-by: Bill Little <bill.james.little@gmail.com>

* Update percentile tests (#4684)

* update percentile tests

* review action

* Updated environment lockfiles (#4685)

Co-authored-by: Lockfile bot <noreply@github.com>

* Bump actions/download-artifact from 2 to 3 (#4686)

Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/download-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/upload-artifact from 2 to 3 (#4687)

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump peter-evans/create-pull-request from 4.0.1 to 4.0.2 (#4688)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.0.1 to 4.0.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/f1a7646cead32c950d90344a4fb5d4e926972a8f...bd72e1b7922d417764d27d30768117ad7da78a0e)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add dynamic matrix to refresh lockfile job (#4573)

* Add dynamic matrix to refresh lockfile job

* review actions

* added link check and ensured https use (#4694)

* [pre-commit.ci] pre-commit autoupdate (#4691)

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.1.0 → v4.2.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.1.0...v4.2.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Comment rendering (#4689)

* used numpy docstrings to fix docs rendering

* comment rejig

* apply numpydoc strings

* corrected notes

* updated docstring guidance.

* added explicit mention of numpydoc strings

* added whatsnew

* added whatsnew PR ref

* Update lib/iris/fileformats/nimrod_load_rules.py

Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>

* Update docs/src/whatsnew/latest.rst

Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>

* review actions.

Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>

* Updated environment lockfiles (#4696)

Co-authored-by: Lockfile bot <noreply@github.com>

* Cache regular points generation (#4698)

* Cache regular points generation

* Move func to util

* Just apply lru_cache to the function once

* What's new

* Git ignore the benchmarks data folder (#4699)

* Git ignore the benchmarks data folder

* Update path and description

* Catch `AttributeError` from removing `CFReader._dataset` when it doesn't exist (#4646)

* Implement full Mercator parameter support (#4609)

* Changes to coord_systems

* Finish feature

* Define both standard_parallel and scale_factor... but leave one None

* Initial test changes

* Test updates and additions

* New expected test outputs

* What's new and test data version bump

* Review fix

* Edit the right file

* Updated environment lockfiles (#4706)

Co-authored-by: Lockfile bot <noreply@github.com>

* Add evertrol to whatsnew of #4646 given #4705 (#4709)

* Add `iris.analysis.MAX_RUN` aggregator (#4676)

* Failing tests

* Start with the tests

* Draft implementation

* Test results

* Account for masked data and multiple axes

* Testing, bug fixes

* Reduce duplication

* Update .cml and fix test failures

* Don't use map_complete_blocks

* Use dask versions of functions

* Test and fix masked data handling

* Use da.from_array instead of as_lazy_data

* Test chunking, fix dask.array import

* Restore _lazy_data.py as we're not using those changes

* credit Will's epic review (#4711)

* add bullet (#4715)

* Weighted `aggregated_by` (#4589)

* First suggestion for weighted aggregated_by

* Added lazy weighted aggregation

* Added first tests for weighted aggregated_by

* Added further tests and allowed returned kwarg

* Added unit tests for weighted aggregated_by

* Moved _get_aggregator_fn to iris.analyis

* Reused 'returned' keyword

* Added test for lazy output if 'returned' is used

* Removed checksum=False from tests

* Expanded test for lazy aggregation since PERCENTILE is lazy now

* Added schlunma to common links

* Addaux speedup (#4718)

* Attempt faster add-aux-fact

* Added whatsnew draft.

* Put PR ref into whatsnew.

* Review changes.

* Remove time_return benchmarks. (#4717)

* Credit reviewer in what's new contributions. (#4705)

* Credit reviewer in what's new contributions.

* Correct example link in what's new docs.

Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>

Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>

* Change Cell equality to equate nans (fixes #4681) (#4701)

* nans in cells satisfy equality

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address review comments

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Netcdf saving: workaround known netcdf bug. (#4407)

* Netcdf saving: workaround known netcdf bug.

* Fix netcdf dim renaming for anonymous dims.

* Fixes to test-mesh generation to preserve save-load roundtrips.

* Added tests for all the different places where code was fixed.

* Tidy some formatting and comments.

* Fix rebase error?

* Review changes.

* Updated environment lockfiles (#4724)

Co-authored-by: Lockfile bot <noreply@github.com>

* MAX_RUN into docs (#4726)

* Datum support in Iris (#4704)

* Support in helpers

* coord_systems changes

* More additions

* Revert some FUTURE guards and condense datum application

* Fixing tests and improving

* More additions

* Tests pass

* transform_points test

* Review fixes

* In datums, unknown means None

* Fix test

* Suppress FutureWarning in the tests that were failing due to it

* Review fixes

* What's new

* Extra test result

* Bump peter-evans/create-pull-request from 4.0.2 to 4.0.3 (#4733)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.0.2 to 4.0.3.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/bd72e1b7922d417764d27d30768117ad7da78a0e...f094b77505fb89581e68a1163fbd2fffece39da1)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Updated environment lockfiles (#4732)

Co-authored-by: Lockfile bot <noreply@github.com>

* Replace nose with pytest (#4734)

* Replace nose with pytest

* Reinstate magic processor number

* add psutil, fstrings and rename TestMixin

* Caching during load (#4716)

* Caching

* Move _optimum_chunksize docstring

* Should be copying coords because they're mutable

* Handle failure of datetime comparisons gracefully

* Review fixes and more careful checks on mutable objects

* coord_system doesn't have 'copy' but __repr__ covers the whole thing (I think)

* Review fixes

* Revert coord_system changes

* Still need to handle tuple issue

* What's new

* Fix prompted by test failure

* Limit cache size

* Work around comparable datetime issue

* More sensible climatological time points (#4723)

* Initial moves to introduce result_coord_func

* climatological keyword

* Climatology argument to aggregated_by

* Fix the tests

* Review changes from RC

* Test fixes

* Further test fixes

* Move test code back out after rebase

* Different fix for self.coord_dims failure

* Add ifmain

* Revert inadvertent doctest changes

* Review improvements

* Fix doctest black broke

* Stop new tests being executable, which makes the test runner miss them

* Docstrings to comments so they don't obscure test names in results

* Apply suggestions from code review

Co-authored-by: Ruth Comer <10599679+rcomer@users.noreply.github.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Review fixes

* What's new

Co-authored-by: Ruth Comer <10599679+rcomer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Remove Mule monkey patch. (#4738)

* climatology doc tweak (#4740)

* Mesh docs - include examples of converting different mesh formats. (#4739)

* Mesh docs - include examples of converting different mesh formats.

* SMC mesh example fix gotcha.

* Add sample images to mesh conversion examples.

* Mesh conversion examples - fix bullet points.

* Mesh conversion examples - print meshes.

* Whats New entry.

* Mesh conversion examples - more detailed comments.

* Gallery: update NEMO example (#4741)

* update nemo example

* clearer comment

* fixed numbered list entry (#4743)

* Plotting circular data extends args as well as the data (#4649)

* Extend any kwarg array

* Add test

* Add comments and update imagerepo.json

* Try using the value as an array, in case it's been given as a list etc.

* What's new

* Datum performance improvements (#4731)

* Set stacklevel to get more useful FutureWarning

* Remove spare printing

* Only generate crs and globe on demand, handle attr setting

* Updates docstrings to make it clearer what's going on

* Additional tests (and fixes found from testing)

* Warn that inverse_flattening setting doesn't help

* Updated environment lockfiles (#4742)

Co-authored-by: Lockfile bot <noreply@github.com>

* Extra datum translation (#4753)

* Updated environment lockfiles (#4760)

Co-authored-by: Lockfile bot <noreply@github.com>

* Permit fast percentile with mdtol=0 (#4755)

* Permit fast percentile with mdtol=0

* What's new

* Check arrays like they're masked, and make the expected arrays masked

* Bigger arrays to avoid np quirk, bug fix, check masking in 2d_multi

* Suppress numpy masked percentile warning

* Use a larger test array that numpy handles how it handles most of them (the worse case)

* Broadcast the mask in the non-lazy case with added dim

* Bug fix on ndim test

* Review fixes

* Credit @rcomer on PR #4755 review (#4761)

* CPerf and SPerf Benchmark Fixes (#4758)

* CSPerf fixes.

* Correct argument hashing in file names.

* Warn if BENCHMARK_DATA is not set.

* SPerf combine_regions better file handling.

* SPerf don't store FileMixin files long term - too much space.

* More realistic SPerf file sizes, allow larger-than-memory files.

* Add SPerf comment about total disk space.

* Corrected CSPerf directories printout.

* Remove namespace conflict in SPerf combine_regions.

* don't use np.float or np.int (#4763)

* Test warnings (#4764)

* import from collections.abc

* deprecated csr module

* unittest deprecations

* random_integers -> randint

* collections again

* inspect deprecation

* reduce redunancy

* non test class

* Increase speed of iris.analysis.trajectory.interpolate with linear interpolation (#4366)

* Interpolate all points at once rather than sequentially

* This splits the internals of cube.interpolate in an effort not to make so much superfluous data, but is slow

* Generate the extra values then discard them, but with minimal looping

* Updated docstrings to clarify behaviour

* Benchmarks for trajectory (linear and nearest neighbour)

* Update error thrown for aux coord over interpolated and uninterpolated dimensions, and add unit tests

* Update cml

* What's new has moved

* What's new fix for rebase

* Prevent truncation of cell method descriptions with nested brackets (#4436)

* Add failing test for brackets within comments

* Fix test

* Separate private function now splits the cell method before it's parsed

* Add additional failing test for multiple axis methods

* Handle multiple axis cell methods

* Update docstring to reflect eventual choices

* Add warnings to function, and test them

* Significantly simpler warning checks

* Developer documentation on available testing conveniences (#4600)

* Developer documentation on available testing conveniences

* what's new

* Update to be easily usable without code links working

* Rebase

* Cubelist contain only cubes -- resurrected (#4767)

* ensure cubelists only contain cubes

* address __iadd__ and __setitem__

* __setitem__ tests

* test for setting more than 1 item

* Fix __setitem__ and Py2 tweaks

* implement __setslice__ for Python2.7

* change exceptions to warnings

* stickler

* duck type check; move helpers outside class

* blank lines

* proposed: revert warnings to exceptions

* remove stray extra 'test_fail'

* pass sequences through __init__; _assert_is_cube

* Replace CubeList init with new. Cube testing duck-type.

* Review change.

* Provide new-style whatsnew.

* Fix whatsnew

Co-authored-by: Ruth Comer <ruth.comer@metoffice.gov.uk>

* Updated environment lockfiles (#4775)

Co-authored-by: Lockfile bot <noreply@github.com>

* Redirect whatsnew to latest (#4727)

* Redirect whatsnew

* Get the top bar working too

* Clean build with right links

* Update release docs

* Review fixes

* update section name for clarity

* Bump peter-evans/create-pull-request from 4.0.3 to 4.0.4 (#4783)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.0.3 to 4.0.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/f094b77505fb89581e68a1163fbd2fffece39da1...923ad837f191474af6b1721408744feb989a4c27)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Updated environment lockfiles (#4782)

Co-authored-by: Lockfile bot <noreply@github.com>

* Fix whatsnew (#4784)

* Fix whatsnew

* Instead of changing 3.0.rst, rename 'whatsnew.rst' to 'index.rst'

* Missed one

* `PolarStereographic` implementation (#4773)

* PolarStereographic implementation

* Test data version

* What's new

* Accept suggestion to lib/iris/coord_systems.py

Co-authored-by: Patrick Peglar <patrick.peglar@metoffice.gov.uk>

* Permit Stereographic saving

* Update what's new

* Review fixes

Co-authored-by: Patrick Peglar <patrick.peglar@metoffice.gov.uk>

* Add mesh details to cube printout. (#4778)

* Slightly refactor repr-html code to rationalise section handling.

* Test repr-html of string attributes.

* Add mesh details to cube printout.

* Update meshcube printouts in docs.

* Add Mesh section to repr-html.

* Review change : make test clearer.

* Review change : fix string truncation/quoting.

* Retire use of the New labels. (#4801)

* pinned pydata-sphinx-theme until the docs are dark mode friendly. (#4796)

* adopt ci gha (#4503)

* adopt ci gha

* bump iris-test-data to latest version

* ci noxify

* remove dummy cache key name

* remove precommit nox task

* added whatsnew entry

* remove mpl cfg for linkcheck

* trial composite nox-cache

* composite iris-data-cache action

* composite conda-env-cache action

* composite conda-pkg-cache action

* composite cartopy-cache action

* required lock-file composite action input

* review action

* review action

* nox inputs.lock_file

* env_name composite action input

* replace HOME with tilde usage

* cancel in-progress workflow jobs

* refresh-lockfiles gha note

* Scripted Logo Generation, take two (#3935)

* Added logo generation Python script.

* Replace lambda scale_func.n

* Matched current logo appearance.

* Simplified gradient code.

* Renamed land_mask to land_clip.

* Make new svg instead of faffing with reformatting existing.

* Refactor out 'style' in attributes.

* File header.

* Integrated logo generation script into Sphinx docs generation.

* Environment variable logo names.

* Removed banner width calculation to avoid font dependency.

* Refactored to also generate rotating logos.

* Replace plt.tightlayout with plt.subplots_adjust.

* Replaced OrderedDicts with dicts (not needed >=Py3.6).

* Numpy standard import form.

* Replaced animated SVG with precusor files for animated GIF.

* Corrected logo URL to source from ReadTheDocs site.

* Changes to better match 'hand-created' logo.

* Fixed logo rotate zip title.

* Relocated logo generation script to SciTools/marketing.

* update links (#3942)

* update links

* added s to http

* Added logo generation Python script.

* Replace lambda scale_func.n

* Matched current logo appearance.

* Simplified gradient code.

* Renamed land_mask to land_clip.

* Make new svg instead of faffing with reformatting existing.

* Refactor out 'style' in attributes.

* File header.

* Integrated logo generation script into Sphinx docs generation.

* Environment variable logo names.

* Removed banner width calculation to avoid font dependency.

* Refactored to also generate rotating logos.

* Replace plt.tightlayout with plt.subplots_adjust.

* Replaced OrderedDicts with dicts (not needed >=Py3.6).

* Numpy standard import form.

* Replaced animated SVG with precusor files for animated GIF.

* Corrected logo URL to source from ReadTheDocs site.

* Changes to better match 'hand-created' logo.

* Fixed logo rotate zip title.

* Relocated logo generation script to SciTools/marketing.

* Include version string in logo generation.

* Reference official logo generation code.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix README docs reference.

* Include the logos, don't generate on the fly.

Co-authored-by: Ruth Comer <ruth.comer@metoffice.gov.uk>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* fix composite gha (#4809)

* fix composite gha

* bump gha cache_build to force purge

* Whats new entry for SVG logo. (#4810)

* Updated environment lockfiles (#4815)

Co-authored-by: Lockfile bot <noreply@github.com>

* add gha workflow badges (#4813)

* address npy disable cpu features warning (#4811)

* Ensure tests use ncgen/ncdump from the Python env; rationalise uses of ncgen. (#4794)

* Ensure tests use ncgen/ncdump from the Python env; rationalise uses of ncgen a bit.

* Tidy some imports.

* Rationalise use of ncgen and avoid subprocess 'shell=True'.

* Review changes: revert to importing iris.tests first.

* Tidy routines slightly.

* Fix subprocess call.

* Added whatsnew.

* Limit memory benchmark result to 5Mb minimum (#4818)

* docs: cirrus-ci to gha (#4821)

* docs: cirrus-ci to gha

* review actions

* Stereographic load with scale factor (#4785)

* Allow passing of method keyword to numpy.percentile (#4791)

* allow passing of method keyword to numpy.percentile

* review actions

* pass all kwargs

* fix doc kwargs

* Update image testing (#4759)

* Use images from test data

* Bug fixes and docs

* Include circular plot image

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Oops, missed it. There now

* Update test data version

* Update __all__

* Type hinting

* No underscore

* Join strings

* Other underscores

* Other test data version location

* Doc fixes

* Imagerepo refresh info

* Move the lock

* Fix refs

* Remove image index (as it's now in the test data index)

* Fix the json test

* Revert pillow pin removal to separate concerns

* Update imagerepo for test that passes with penultimate val

* Second imagerepo update

* Third set of imagerepo changes

* Fourth imagerepo changes (gallery test)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* whatnew for #4791 (#4823)

* bump ci to latest iris-test-data version (#4825)

* auto generate gha auth token (#4817)

* auto generate gha auth token

* add docs for github app config

* linkcheck ignore metarelate.net

* remove metarelate URL whatsnew reference

* Fix stock CDL headers for unlimited time dimension. (#4827)

* Fix CDL headers for tests.stock.netcdf.

* Whats new entry.

* Improved unlimited dimension check in stock NetCDF generation.

* Whatsnew minor fix (#4829)

* Update latest.rst

* PR link

* add issue link

* Updated environment lockfiles (#4831)

Co-authored-by: Lockfile bot <noreply@github.com>

* rationalise doctests gha (#4819)

* rationalise doctests gha

* update job name with session

* add nox session as composite input

* minor tweak

* review action - tidy nox composite inputs

* Remove pin, update hashes, improve script (#4826)

* Remove pin, update hashes, improve script

* What's new

* Lockfile and whatsnew

* Shareable urls

* Later lockfile

* Shareable...

* Check for consistency before fiddling

* Updates for new test data (idiff'ed)

* Keep lockstep between imagerepo and data

* idiff always runs after imagerepo.json has been updated anyway

* Gallery test updates (also idiff'd)

* Update test data version

* Implement `iris.plot.fill_between` (#4647)

* Remove pin, update hashes, improve script

* What's new

* Lockfile and whatsnew

* Shareable urls

* Later lockfile

* Shareable...

* Check for consistency before fiddling

* Updates for new test data (idiff'ed)

* Keep lockstep between imagerepo and data

* idiff always runs after imagerepo.json has been updated anyway

* Gallery test updates (also idiff'd)

* Implement fill_between and write tests

* Simplify error check

* Test images

* What's new

* Image test results

* Fix error message

* Pre-emptive test data version update

Co-authored-by: Bill Little <bill.james.little@gmail.com>

* Python 3.9 CI & Combine CI workflows (#4832)

* combine ci workflows

* remove badge

* ci unpin python

* post #4759 review actions (#4833)

* pytest less verbose (#4837)

* Remove redundant testing assert methods (#4838)

* remove assertWarnsRegexp

* remove assertStringEqual

* post rebase tidyings

* whatsnew

* NetCDF Generation Adjustments (#4836)

* remove pillow pin from setup.cfg (#4839)

* Support python 3.10 (#4840)

* extend ci support for py310

* fix Cell.__hash__ py310 nan behaviour

* review actions

* update github app docs (#4844)

* Lockfile update and Gregorian to Standard calendar change (#4847)

* sed gregorian

* sed CALENDAR_GREGORIAN

* rename cml files

* revert whatsnew 2.1 and test_cf.py

* revert spelling_allow

* post rebase sed 'gregorian'

* Updated environment lockfiles

* cf-units minimum pin

* review actions

Co-authored-by: Lockfile bot <noreply@github.com>

* Updated environment lockfiles (#4850)

Co-authored-by: Lockfile bot <noreply@github.com>

* avoid setuptools deprecation (#4848)

* [pre-commit.ci] pre-commit autoupdate (#4851)

updates:
- [github.com/pre-commit/pre-commit-hooks: v4.2.0 → v4.3.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.2.0...v4.3.0)
- [github.com/psf/black: 22.3.0 → 22.6.0](https://github.com/psf/black/compare/22.3.0...22.6.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Alternative Cell.__hash__ implementation for NaN support (#4852)

* Alternative Cell.__hash__ implementation for NaN support

* add whatsnew entry

* whatsnew update (#4853)

* adopt setuptools-scm for auto versioning (#4841)

* adopt setuptools-scm for auto versioning

* update py310.yml requirements

* update locks

* simplify docs version

* update dev release docs

* update .gitignore for vscode

* update locks

* add exclude _version.py to MANIFEST.in

* review actions

* update lock files

* update pyproject.toml

* what a whatsnew entry

* fix rtd version (#4855)

* Updated environment lockfiles (#4858)

Co-authored-by: Lockfile bot <noreply@github.com>

* pytest verbosity (#4861)

* Updated environment lockfiles (#4867)

Co-authored-by: Lockfile bot <noreply@github.com>

* Recommend `pytest` in developer guide (#4866)

* recommend pytest

* pros and cons

* tweaks

* add missing skips (#4865)

* Updated environment lockfiles (#4873)

Co-authored-by: Lockfile bot <noreply@github.com>

* Replace testing helper function (#4878)

* replace unmasked_data_as_1d_array

* whatsnew

* Cube arithmetic array type handling (#3790)

* add new tests; fix existing tests

* force lhs lazy if rhs lazy

* black

* add in-place tests

* add handling for unmasked, masked real array combo

* add tests for false skeleton

* rearrange _math_op_common if-loops

* revert maths.py to upstream/master

* simpler solution

* reduce masking arithmetic tests

* expand and improve tests

* add whatsnew

* fix rebase snafu

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update following numpy#21977

* Revert "update following numpy#21977"

This reverts commit 7d62ad6b8779129b9f4c2f7d26f070f5402f95e9.

* update comment

* a before b

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* automate sdist and wheel build and publish (#4849)

* automate sdist and wheel build and publish

* update names

* tidy package manifest and discovery

* add note to dev docs

* update MANIFEST.in

* Update .github/workflows/ci-wheels.yml

Co-authored-by: Martin Yeo <40734014+trexfeathers@users.noreply.github.com>

* add unshallow comment to rtd

* reinstate docs release

* add pypa build comment

* test wheel

* added whatnew + tweak nox wheel emsgs

* Update docs/src/developers_guide/release.rst

Co-authored-by: Martin Yeo <40734014+trexfeathers@users.noreply.github.com>

* Warn users of increased `recombine_submeshes` memory demand. (#4881)

* Warn users of increased recombine_submeshes memory demand.

* Clearer what's new entry.

* Sphinx reset modules (#4885)

* fix sphinx-gallery

* change gallery index reference

* Updated environment lockfiles (#4884)

Co-authored-by: Lockfile bot <noreply@github.com>

* [pre-commit.ci] pre-commit autoupdate (#4886)

updates:
- [github.com/PyCQA/flake8: 4.0.1 → 5.0.2](https://github.com/PyCQA/flake8/compare/4.0.1...5.0.2)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* fix percentile docstring (#4888)

* [pre-commit.ci] pre-commit autoupdate (#4895)

updates:
- [github.com/PyCQA/flake8: 5.0.2 → 5.0.4](https://github.com/PyCQA/flake8/compare/5.0.2...5.0.4)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Upgrade `iris.util.mask_cube` (#4889)

* rewrite mask_cube

* fix import problems

* simplify _unmask_mask and its tests

* parametrize not in place 1d _mask_array

* pytest in-place _mask_array tests

* pytest _mask_array broadcasting tests

* remove iris.tests from test__mask_array

* attempt to preserve metadata

* pull masked array assertions off test class

* pytestify test__unmask_mask.py

* remove _unmask_mask; tweak test

* numpydoc docstring

* add CML checks

* preserve cube.metadata; add final CML

* path handling

* get hold of _mask_array result

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>

* more thorough in_place check

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* review actions

* whatsnew

Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* set default 'unknown' version (#4892)

* Prevent color bar from being misplaced (#4894)

* Prevent color bar from being misplaced

Currently, if `plt.gca()` gives a different Axes object than `axes` in the function `_label`, the color bar will be malpositioned, ending up either on another subfigure, or in another figure window altogether. Specifying the `ax` argument when calling `plt.colorbar` prevents this.

* add whatsnew and test coverage

* check axes and figure

* tidy and add extra axes2 test

Co-authored-by: Bill Little <bill.james.little@gmail.com>

* Lockfile update and skip questionable test (#4901)

* Updated environment lockfiles

* skip questionable test

Co-authored-by: Lockfile bot <noreply@github.com>

* Adapt to the latest setuptools editable configuration. (#4903)

* Adapt to the latest setuptools editable configuration.

* What's new entry.

* What's new typo.

* Update setuptools in Conda requirements.

* pytest gallery tests (#4792)

* pytest parametrize gallery tests

* two context managers as fixtures

* use pathlib instead of os

* docstrings

* use pytest.mark.filterwarnings

* show_replaced_by_check_graphic as fixture

* Revert previous (more readable without)

This reverts commit bdca27cd301fff9de034f14db9da991dac43facf.

* gallery_dir function to variable

* add setup/teardown fixture

* tweak docstring

* remove gallerytests from Makefile

* update dev guide

* doc tweaks

* use monkeypatch to simplify

* move stuff about

* monkeypatch for sys.path; comments and param ids

* revert assert False

* revert deliberate fails

* import_patches fixture

* post rebase check_graphic fix

* update test references

* whatsnew

* test data version

* Updated environment lockfiles (#4904)

Co-authored-by: Lockfile bot <noreply@github.com>

* Revert Cell.__hash__ fix (#4874)

* Revert Cell.__hash__ fix

* add whatsnew entry

* update comments

* Revert #4701 cell nan equality

Co-authored-by: lbdreyer <laura.dreyer@metoffice.gov.uk>

* Add whats new entry for 4734 (#4908)

* Extend new_axis to reshape _DimensionalMetadata objects (#4896)

* extend iris.util.new_axis

* add tests

* fix tests

* Rewrite new_axis; extra stock cube; cube _dimensional_metadata utility and tests

* Add new_axis tests

* Convert to pytest

* Rename broadcast -> expand; review actions

* Add whatsnew entry

Co-authored-by: stephen.worsley <stephen.worsley@metoffice.gov.uk>

* Standard names table update (v79) (#4910)

* update cf standard name table

* add whatsnew

* fix whatsnew

* Pin shapely (#4911)

* pin shapely

* update whatsnew

* Improvements to Pandas-to-Iris bridge (#4890)

* First pass using merge to parse ndim Pandas DataFrame.

* Second pass actually doing array manipulation this time.

* Tidying for public consumption.

* Improved index handling.

* Pytest conversion.

* Minor re-factoring.

* Support for copy argument.

* Calendar handling.

* Correct coord reshaping.

* Correct array indexing.

* DataFrame Series agnosticism.

* Docstrings and structure.

* Error as_cube if FUTURE behaviour is set.

* Tests WIP.

* More tests.

* Completed tests.

* Suppress deprecation warning in TestSeriesAsCube.

* Docstring corrections.

* Another docstring fix.

* Another docstring fix.

* Another docstring fix.

* Doctests.

* Doctests.

* Strict expectation of one Cube in tests.

* format_dimensional_metadata typo.

* Rename pandas_array to pandas_structure.

* Leaner warnings for Series with column arguments.

* Fix date tests.

* Back out pandas_ndim FUTURE flag.

* Uncomment code, whoops.

* What's new entry.

* Remove erroneous pass.

* Doctest typo.

* Handle an empty Pandas structure.

* Test showing that dimension order is preserved.

* Informative error for ragged indexes.

* Include shapely pin in setup.cfg (#4917)

* Include shapely pin in setup.cfg.

* What's new entry.

* update whatsnew (#4914)

* Avoid using mutable default. (#4918)

* Fix some @trexfeathers What's New entries (#4920)

* Minor What's New fixes.

* Missing space.

* Update whatsnew for v3.3.0rc0 (#4921)

* update whatsnew

* fix index

* remove template

* address review comments

* Updated environment lockfiles (#4923)

Co-authored-by: Lockfile bot <noreply@github.com>

* Update whatsnew (#4924)

* reset whatsnew

* remove bugfix section

* address review comments

* Bump peter-evans/create-pull-request from 4.0.4 to 4.1.1 (#4922)

Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.0.4 to 4.1.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](https://github.com/peter-evans/create-pull-request/compare/923ad837f191474af6b1721408744feb989a4c27...18f90432bedd2afd6a825469ffd38aa24712a91d)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Tilde's and wildcard recognition within iris.save. (#4913)

* Draft: Tilde's and wildcard can now be read by iris.save. Modifications and tests provisionally done, docstrings still need editing, with examples and doctests needing being added/editing.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Docstrings updated for expand_filespecs and save

* docstrings for save and expand_filespecs updated to numpy formatting

* Reviewed: Added expansion and relative path tests to expand_filespecs. Ensured list indexing returns single item lists. Minor comment changes.

* Updated latest What's New, and added ESadek-MO to common links

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fixed isort conflicts

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Unpin shapely (#4925)

* unpin shapely

* update whatsnew

* SUM, COUNT, PROPORTION: don't use dask on numpy arrays (#4905)

* COUNT

* SUM

* tidy up

* increase dask min pin

* fix returned weights when masked

* typo

* docstrings

* whatsnew

* include reviewer in whatsnew

* Remove `setUpClass` from Iris tests. (#4927)

* remove IrisTest.setUpClass

* whatsnew

* Better handle `setuptools_scm` versioning (#4926)

* Use setuptools_scm release-branch-semver version_scheme.

* Properly handle all possible scm-rtd combinations.

* What's new entry.

* Use git stash to protect setuptools_scm from RTD.

* Better version badges.

* Better use of git stash.

* Substitute commit SHAs in place of versions where appropriate.

* Make rtd_version safe for use in shields.io badges.

* Make rtd_version safe for use in shields.io badges.

* Allowed collapsing coordinates with `nbounds != (0, 2)` (#4870)

* Allowed collapsing over coordinates with nbounds!=0,2

* Simplified test

* Added further tests for other warnings

* pre-commit

* Used unittest's assertWarnsRegex

* Added What's new? entry

* fix whatsnew (#4932)

* Updated environment lockfiles (#4936)

Co-authored-by: Lockfile bot <noreply@github.com>

* update whatsnew title for v3.3 release (#4939)

* Added zonal mean gallery example, continued from @TTV-Intrepid (#4935)

* Added zonal mean gallery example, continued from @TTV-Intrepid

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added TTVIntrepid to common links for What's New functionality

* Fixed zonal mean plotting, add axh

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor visual changes, ax3 plotting fix, removed warning suppression.

* updated imagerepo for zonal means

* updated ci-tests yml to latest iris test data version

* updated what's new

* updated announcements to one line

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* [pre-commit.ci] pre-commit autoupdate (#4942)

updates:
- [github.com/psf/black: 22.6.0 → 22.8.0](https://github.com/psf/black/compare/22.6.0...22.8.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Remove deprecated 'python setup.py test' support (#4948)

* Remove deprecated 'python setup.py test' support

* add whatsnew entry

* purge unused setup clean_source custom command (#4949)

* purge unnecessary setup.py copy_copyright function (#4950)

* fix setup.py std_names building (#4952)

* fix setup.py std_names building

* refactor

* docs whatsnew entry updates (#4953)

* Fix cube repr for scalar anc_var and cell measures (#4945)

* Fix cube repr for scalar anc_var and cell measures

* Add whatsnew

* Fix cube summary tests

* Review actions: reword whats new, reorder scalar sections in cube printout

* Fix cube summary test, update whatsnew

* Fix code comment

* update pypi trove classifiers (#4954)

* Add new Q+A issue, update author email

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Bill Little <bill.james.little@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Lockfile bot <noreply@github.com>
Co-authored-by: Martin Yeo <40734014+trexfeathers@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bill Little <bill.little@metoffice.gov.uk>
Co-authored-by: Ruth Comer <10599679+rcomer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Patrick Peglar <patrick.peglar@metoffice.gov.uk>
Co-authored-by: Will Benfold <69585101+wjbenfold@users.noreply.github.com>
Co-authored-by: Denis Sergeev <dennis.sergeev@gmail.com>
Co-authored-by: tkknight <2108488+tkknight@users.noreply.github.com>
Co-authored-by: Will Benfold <will.benfold@metoffice.gov.uk>
Co-authored-by: Manuel Schlund <32543114+schlunma@users.noreply.github.com>
Co-authored-by: stephenworsley <49274989+stephenworsley@users.noreply.github.com>
Co-authored-by: Ruth Comer <ruth.comer@metoffice.gov.uk>
Co-authored-by: iris-actions[bot] <107775138+iris-actions[bot]@users.noreply.github.com>
Co-authored-by: scitools-ci[bot] <107775138+scitools-ci[bot]@users.noreply.github.com>
Co-authored-by: Kristofer Krus <krikru@users.noreply.github.com>
Co-authored-by: stephen.worsley <stephen.worsley@metoffice.gov.uk>
Co-authored-by: Elias <110238618+ESadek-MO@users.noreply.github.com>
@trexfeathers trexfeathers mentioned this pull request Feb 8, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jamesp jamesp jamesp left review comments

@trexfeathers trexfeathers trexfeathers approved these changes

Assignees

@trexfeathers trexfeathers

Labels
Type: Enhancement Type: Infrastructure
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@bjlittle @trexfeathers @jamesp