Hypothesis strategy for generating Variable objects #8404

TomNicholas · 2023-11-02T17:04:03Z

Breaks out just the part of #6908 needed for generating arbitrary xarray.Variable objects. (so ignore the ginormous number of commits)

EDIT: Check out this test which performs a mean on any subset of any Variable object!

In [36]: from xarray.testing.strategies import variables

In [37]: variables().example()
<xarray.Variable (ĭ: 3)>
array([-2.22507386e-313-6.62447795e+016j,
                    nan-6.46207519e+185j,
       -2.22507386e-309+3.33333333e-001j])

@andersy005 @maxrjones @jhamman I thought this might be useful for the NamedArray testing. (xref #8370 and #8244)

@keewis and @Zac-HD sorry for letting that PR languish for literally a year 😅 This PR addresses your feedback about accepting a callable that returns a strategy generating arrays. That suggestion makes some things a bit more complex in user code but actually allows me to simplify the internals of the variables strategy significantly. I'm actually really happy with this PR - I think it solves what we were discussing, and is a sensible checkpoint to merge before going back to making strategies for generating composite objects like DataArrays/Datasets work.

Closes part of Public hypothesis strategies for generating xarray data #6911
Tests added
User visible changes (including notable bug fixes) are documented in whats-new.rst
New functions/methods are listed in api.rst

…ctor

for more information, see https://pre-commit.ci

…atible

…s/xarray into hypothesis-strategies

for more information, see https://pre-commit.ci

keewis

I didn't have time to check the tests yet, but here are a few comments

keewis · 2023-11-13T17:48:29Z

doc/user-guide/testing.rst

+Testing your code
+=================


Not sure. It is true that the page has a different target audience than the other pages in the user guide, but then again applications can also be tested. And, so far the "internals" section describes implementation details or extension mechanisms that affect the internals.

doc/user-guide/testing.rst

doc/whats-new.rst

xarray/testing/strategies.py

keewis · 2023-11-13T18:18:49Z

xarray/testing/strategies.py

+    return (
+        npst.integer_dtypes()
+        | npst.unsigned_integer_dtypes()
+        | npst.floating_dtypes()
+        | npst.complex_number_dtypes()
+    )


we do support string dtypes, but only for a subset of operations. Is this worth mentioning?

This is not meant to be an exhaustive list (yet). It doesn't include datetimes either.

agreed, but most operations don't make sense on string or datetime dtypes so it might be better to make a separate list of dtypes for those?

Sure - I'm just saying let's defer detailed discussions of which types to test until another issue / PR, the point of this PR is to provide a framework flexible enough to easily test xarray functions with any type we want, which this achieves.

xarray/testing/strategies.py

xarray/testing/testing.py

Co-authored-by: Justus Magin <keewis@users.noreply.github.com>

…omNicholas/xarray into hypothesis-strategies-variable

Zac-HD · 2023-11-13T22:44:21Z

The only final thing is that the docs don't build because of one weird warning (our docs are set to fail on any warnings):
xarray/xarray/testing/strategies.py:docstring of xarray.testing.strategies.accept.<locals>.variables:47:
    WARNING: Block quote ends without a blank line; unexpected unindent.
Given that I don't define any local variables called accept, but hypothesis apparently does, I guess this must be hypothesis' fault somehow?

My guess is that this is an existing docstring, the location of which is being misreported due to the various wrappers that Hypothesis inserts. I'd be very surprised if Hypothesis is modifying docstrings somehow, but I guess trimming trailing whitespace is the kind of thing that could happen somewhere in the stack.

No direct insight, but getting the full text of the docstring it's complaining about should help?

…ategy

TomNicholas · 2023-12-05T05:53:27Z

I got the docs build to pass! The warning was due to extra lines in the examples of the variables strategy docstring. I only managed to find it by trial and error 🙄

@keewis do you want to review the tests before I merge it? (The test failures now are something groupby-related, and are also happening in #8521, so definitely not my fault!)

keewis

I didn't spot anything that we wouldn't be able to change after merging / releasing, so I'd say let's merge and see how well it works in practice.

keewis · 2023-12-05T19:52:19Z

xarray/testing/strategies.py

+    )
+
+
+def smallish_arrays(


is the only reason we have this function the default strategy for shape (and maybe some additional typing)? If so, we might be able to use functools.partial on npst.arrays? Unless you meant to expose this as public API (it's not in the API reference)

That is the only reason. I did not think of using functools.partial - that's a good idea, I can try that out before merging.

That actually won't work because we do need to be able to pass shape and dtype to the array_strategy_fn.

But I tried removing smallish_arrays completely and the tests still seem to complete in a reasonable amount of time, so I've actually just taken it out for now.

…ray#8404

* fix import of xarray.testing internals that was changed by pydata/xarray#8404 * bump minimum required version of xarray * linting

* main: (26 commits) Filter null values before plotting (pydata#8535) Update concat.py (pydata#8538) Add getitem to array protocol (pydata#8406) Added option to specify weights in xr.corr() and xr.cov() (pydata#8527) Filter out doctest warning (pydata#8539) Bump actions/setup-python from 4 to 5 (pydata#8540) Point users to where in their code they should make mods for Dataset.dims (pydata#8534) Add Cumulative aggregation (pydata#8512) dev whats-new Whats-new for 2023.12.0 (pydata#8532) explicitly skip using `__array_namespace__` for `numpy.ndarray` (pydata#8526) Add `eval` method to Dataset (pydata#7163) Deprecate ds.dims returning dict (pydata#8500) test and fix empty xindexes repr (pydata#8521) Remove PR labeler bot (pydata#8525) Hypothesis strategy for generating Variable objects (pydata#8404) Use numbagg for `rolling` methods (pydata#8493) Bump pypa/gh-action-pypi-publish from 1.8.10 to 1.8.11 (pydata#8514) fix RTD docs build (pydata#8519) Fix type of `.assign_coords` (pydata#8495) ...

* main: (58 commits) Adapt map_blocks to use new Coordinates API (pydata#8560) add xeofs to ecosystem.rst (pydata#8561) Offer a fixture for unifying DataArray & Dataset tests (pydata#8533) Generalize cumulative reduction (scan) to non-dask types (pydata#8019) Filter null values before plotting (pydata#8535) Update concat.py (pydata#8538) Add getitem to array protocol (pydata#8406) Added option to specify weights in xr.corr() and xr.cov() (pydata#8527) Filter out doctest warning (pydata#8539) Bump actions/setup-python from 4 to 5 (pydata#8540) Point users to where in their code they should make mods for Dataset.dims (pydata#8534) Add Cumulative aggregation (pydata#8512) dev whats-new Whats-new for 2023.12.0 (pydata#8532) explicitly skip using `__array_namespace__` for `numpy.ndarray` (pydata#8526) Add `eval` method to Dataset (pydata#7163) Deprecate ds.dims returning dict (pydata#8500) test and fix empty xindexes repr (pydata#8521) Remove PR labeler bot (pydata#8525) Hypothesis strategy for generating Variable objects (pydata#8404) ...

commit 0a0f800 Merge: 33c8033 41d33f5 Author: Deepak Cherian <dcherian@users.noreply.github.com> Date: Tue Jan 2 20:42:51 2024 -0700 Merge branch 'main' into depr-groupby-squeeze-2 commit 33c8033 Author: Deepak Cherian <deepak@cherian.net> Date: Tue Jan 2 20:40:42 2024 -0700 Don't skip for resampling commit d7be352 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Jan 3 03:24:13 2024 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit d13fa0e Author: Deepak Cherian <dcherian@users.noreply.github.com> Date: Tue Jan 2 20:23:43 2024 -0700 Apply suggestions from code review Co-authored-by: Michael Niklas <mick.niklas@gmail.com> commit dd6ea53 Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 19:29:40 2023 -0700 Silence more warnings commit 44e5a41 Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 19:21:06 2023 -0700 minimize test mods commit 94c1c1f Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:55:46 2023 -0700 Add tests for pydata#8263 commit 0ab4eb6 Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:47:41 2023 -0700 Fix typing commit a064430 Merge: d6a3f2d 03ec3cb Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:47:04 2023 -0700 Merge branch 'main' into depr-groupby-squeeze-2 * main: Fix mypy type ignore (pydata#8564) Support for the new compression arguments. (pydata#7551) FIX: reverse index output of bottleneck move_argmax/move_argmin functions (pydata#8552) Adapt map_blocks to use new Coordinates API (pydata#8560) add xeofs to ecosystem.rst (pydata#8561) Offer a fixture for unifying DataArray & Dataset tests (pydata#8533) Generalize cumulative reduction (scan) to non-dask types (pydata#8019) commit d6a3f2d Author: Deepak Cherian <deepak@cherian.net> Date: Thu Dec 21 18:46:50 2023 -0700 Fix generator for aggregations commit 97f1695 Author: Deepak Cherian <deepak@cherian.net> Date: Tue Dec 19 10:58:11 2023 -0700 Fix docs commit 5b33b98 Author: Deepak Cherian <deepak@cherian.net> Date: Sun Dec 17 20:35:53 2023 -0700 fix whats-new commit 80b2b36 Author: Deepak Cherian <deepak@cherian.net> Date: Sun Dec 17 20:26:17 2023 -0700 Reduce more warnings commit 5f6f4ea Merge: a57d4ae 2971994 Author: Deepak Cherian <deepak@cherian.net> Date: Sat Dec 16 20:33:13 2023 -0700 Merge branch 'main' into depr-groupby-squeeze-2 * main: (26 commits) Filter null values before plotting (pydata#8535) Update concat.py (pydata#8538) Add getitem to array protocol (pydata#8406) Added option to specify weights in xr.corr() and xr.cov() (pydata#8527) Filter out doctest warning (pydata#8539) Bump actions/setup-python from 4 to 5 (pydata#8540) Point users to where in their code they should make mods for Dataset.dims (pydata#8534) Add Cumulative aggregation (pydata#8512) dev whats-new Whats-new for 2023.12.0 (pydata#8532) explicitly skip using `__array_namespace__` for `numpy.ndarray` (pydata#8526) Add `eval` method to Dataset (pydata#7163) Deprecate ds.dims returning dict (pydata#8500) test and fix empty xindexes repr (pydata#8521) Remove PR labeler bot (pydata#8525) Hypothesis strategy for generating Variable objects (pydata#8404) Use numbagg for `rolling` methods (pydata#8493) Bump pypa/gh-action-pypi-publish from 1.8.10 to 1.8.11 (pydata#8514) fix RTD docs build (pydata#8519) Fix type of `.assign_coords` (pydata#8495) ... commit a57d4ae Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 21:36:04 2023 -0700 Test one more warning commit bf8139d Author: Deepak Cherian <dcherian@users.noreply.github.com> Date: Fri Dec 1 21:33:45 2023 -0700 Update xarray/tests/test_groupby.py commit 4e9a063 Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 21:10:14 2023 -0700 Set squeeze=None for Dataset too commit c2e576e Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 20:54:17 2023 -0700 Fix first, last commit 6d8e822 Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 20:46:21 2023 -0700 better warning commit 62c334b Author: Deepak Cherian <deepak@cherian.net> Date: Fri Dec 1 20:45:17 2023 -0700 silence warnings commit b7805a8 Author: dcherian <deepak@cherian.net> Date: Tue Aug 15 10:54:25 2023 -0600 Deprecate `squeeze` in GroupBy. Closes pydata#2157

* fix import of xarray.testing internals that was changed by pydata/xarray#8404 * bump minimum required version of xarray * linting

TomNicholas and others added 30 commits August 11, 2022 03:11

copied files defining strategies over to this branch

587ebb8

placed testing functions in their own directory

acbfa69

moved hypothesis strategies into new testing directory

73d763f

begin type hinting strategies

db2deff

renamed strategies for consistency with hypothesis conventions

746cfc8

added strategies to public API (with experimental warning)

03cd9de

strategies for chunking patterns

2fe3583

rewrote variables strategy to have same signature as Variable constru…

4db3629

…ctor

test variables strategy

14d11aa

fixed most tests

418a359

added helpers so far to API docs

c8a7d0e

add hypothesis to docs CI env

d48aceb

add todo about attrs

a20e341

draft of new user guide page on testing

3a4816f

types for dataarrays strategy

d0406a2

draft for chained chunking example

65a222d

[pre-commit.ci] auto fixes from pre-commit.com hooks

e1d718a

for more information, see https://pre-commit.ci

only accept strategy objects

57d0f5b

fixed failure with passing in two custom strategies that must be comp…

82c734c

…atible

syntax error in example

029f19a

allow sizes dict as argument to variables

46895fe

copied subsequences_of strategy

50c62e9

coordinate_variables generates non-dimensional coords

e21555a

dataarrays strategy given nothing working!

1688779

improved docstrings

0a29d32

datasets strategy works (given nothing)

3259849

Merge branch 'hypothesis-strategies' of https://github.com/TomNichola…

717fabe

…s/xarray into hypothesis-strategies

[pre-commit.ci] auto fixes from pre-commit.com hooks

d76e5b6

for more information, see https://pre-commit.ci

pass dims or data to dataarrays() strategy

c25940c

importorskip hypothesis in tests

cd7b065

keewis reviewed Nov 13, 2023

View reviewed changes

TomNicholas and others added 7 commits November 13, 2023 13:19

Use .copy in convert_to_sparse

6bbd13b

Co-authored-by: Justus Magin <keewis@users.noreply.github.com>

Use st.builds in sparse example

29ecd7d

Co-authored-by: Justus Magin <keewis@users.noreply.github.com>

correct intersphinx link in whatsnew

631e810

Merge branch 'hypothesis-strategies-variable' of https://github.com/T…

c613027

…omNicholas/xarray into hypothesis-strategies-variable

rename module containing assertion functions

4412d98

clarify sentence

1ea0dcf

add general ImportError if hypothesis not installed

cf1a45e

TomNicholas added 3 commits November 14, 2023 10:20

add See Also link to strategies docs page from docstring of every str…

ea738cd

…ategy

typo in ImportError message

79b0094

Merge branch 'main' into hypothesis-strategies-variable

c6d43ca

TomNicholas mentioned this pull request Nov 29, 2023

Repeated coordinates leads to unintuitive (broken?) indexing behaviour #3731

Open

TomNicholas added 2 commits December 4, 2023 16:21

Merge branch 'main' into hypothesis-strategies-variable

00079bd

remove extra blank lines in examples

cbcd486

keewis approved these changes Dec 5, 2023

View reviewed changes

TomNicholas added 2 commits December 5, 2023 17:10

remove smallish_arrays

69ddd08

Merge branch 'main' into hypothesis-strategies-variable

ea90162

TomNicholas merged commit ab6a255 into pydata:main Dec 5, 2023
18 of 28 checks passed

Variable refactor automation moved this from In progress to Done Dec 5, 2023

TomNicholas added a commit to TomNicholas/datatree that referenced this pull request Dec 10, 2023

fix import of xarray.testing internals that was changed by pydata/xar…

eb19bd5

…ray#8404

TomNicholas added a commit to xarray-contrib/datatree that referenced this pull request Dec 10, 2023

Fix for xarray v2023.12.0 (#294)

d3b2a6d

* fix import of xarray.testing internals that was changed by pydata/xarray#8404 * bump minimum required version of xarray * linting

TomNicholas mentioned this pull request Dec 14, 2023

Duckarray tests for constructors and properties #6903

Open

4 tasks

TomNicholas mentioned this pull request Dec 18, 2023

Support non-str Hashables in DataArray #8559

Merged

3 tasks

Zac-HD mentioned this pull request Apr 1, 2024

Hypothesis strategies in xarray.testing.strategies #6908

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hypothesis strategy for generating Variable objects #8404

Hypothesis strategy for generating Variable objects #8404

TomNicholas commented Nov 2, 2023 •

edited

keewis left a comment

keewis Nov 13, 2023

keewis Nov 13, 2023

TomNicholas Nov 13, 2023

keewis Nov 13, 2023

TomNicholas Nov 13, 2023

Zac-HD commented Nov 13, 2023

TomNicholas commented Dec 5, 2023 •

edited

keewis left a comment

keewis Dec 5, 2023

TomNicholas Dec 5, 2023

TomNicholas Dec 5, 2023

		Testing your code
		=================

		)


		def smallish_arrays(

Hypothesis strategy for generating Variable objects #8404

Hypothesis strategy for generating Variable objects #8404

Conversation

TomNicholas commented Nov 2, 2023 • edited

keewis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zac-HD commented Nov 13, 2023

TomNicholas commented Dec 5, 2023 • edited

keewis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomNicholas commented Nov 2, 2023 •

edited

TomNicholas commented Dec 5, 2023 •

edited