test_raster_quadmesh_autorange_reversed[dask.array] fails with latest dask

[jonringer]

ALL software version info

most likely to be relevant:

dask==2021.10.0
xarray=0.20.1
datashaer==0.13.0
pandas==1.3.4

literally ALL python packages present in build closure:

aiodns-3.0.0
aiohttp-3.8.1
aiosignal-1.2.0
alabaster-0.7.12
aniso8601-9.0.1
apipkg-1.5
appdirs-1.4.4
arrow-1.2.1
asgiref-3.4.1
astunparse-1.6.3
async_generator-1.10
async-timeout-4.0.1
atomicwrites-1.4.0
attrs-21.2.0
Automat-20.2.0
Babel-2.9.1
backcall-0.2.0
backports-entry-points-selectable-1.1.0
bcrypt-3.2.0
beautifulsoup4-4.10.0
beniget-0.4.1
bleach-4.1.0
blinker-1.4
bokeh-2.4.2
bootstrapped-pip-21.1.3
brotli-1.0.9
brotlicffi-1.0.9.2
brotlipy-0.7.0
buildcatrust-0.1.3
cchardet-2.1.7
certifi-2021.10.08
cffi-1.15.0
cftime-1.5.1.1
chardet-4.0.0
charset-normalizer-2.0.8
click-8.0.3
cloudpickle-1.6.0
colorama-0.4.4
colorcet-2.0.6
commonmark-0.9.1
constantly-15.1.0
construct-2.10.67
coverage-5.5
cppy-1.1.0
cramjam-2.4.0
cryptography-3.4.8
cryptography_vectors-3.4.8
cycler-0.11.0
Cython-0.29.24
dask-2021.10.0
dask-2021.10.0
datashader-0.13.0
datashape-0.5.4
debugpy-1.5.1
decorator-5.1.0
defusedxml-0.7.1
distlib-0.3.3
distributed-2021.11.2
Django-2.2.24
docopt-0.6.2
docutils-0.17.1
entrypoints-0.3
execnet-1.9.0
fastparquet-0.7.1
fb-re2-1.0.7
filelock-3.3.0
flake8-4.0.1
flaky-3.7.0
Flask-2.0.2
Flask-Limiter-1.4
Flask-RESTful-0.3.9
flit-core-3.2.0
fonttools-4.26.2
freezegun-1.1.0
frozenlist-1.2.0
fsspec-2021.10.1
future-0.18.2
gast-0.5.1
gevent-20.9.0
greenlet-1.1.2
gssapi-1.7.2
gunicorn-20.1.0
HeapDict-1.0.1
hiro-0.5.1
holoviews-1.14.6
html5lib-1.1
httpbin-0.7.0
hyperlink-21.0.0
hypothesis-6.27.1
hypothesis-6.27.1
idna-3.3
imagesize-1.3.0
importlib-metadata-4.8.2
incremental-21.3.0
iniconfig-1.1.1
intreehooks-1.0
ipykernel-6.5.1
ipython-7.29.0
ipython_genutils-0.2.0
iso8601-0.1.16
itsdangerous-2.0.1
jedi-0.18.0
Jinja2-3.0.3
jpylyzer-2.0.0
jsonschema-3.2.0
jupyter_client-7.1.0
jupyter_core-4.9.1
jupyterlab_pygments-0.1.2
kiwisolver-1.3.2
limits-1.5.1
llvmlite-0.37.0
locket-0.2.1
lxml-4.6.4-5
m2r-0.2.1
markdown-3.3.5
markupsafe-2.0.1
matplotlib-3.5.0
matplotlib-inline-0.1.3
mccabe-0.6.1
memory_profiler-0.55.0
mistune-0.8.4
mock-4.0.3
more-itertools-8.12.0
mpi4py-3.0.3
mpmath-1.2.1
msgpack-1.0.3
multidict-5.2.0
multipledispatch-0.6.0
munkres-1.1.4
nbclient-0.5.9
nbconvert-6.3.0
nbformat-5.1.3
nbsmoke-0.5.0
nest_asyncio-1.5.1
netCDF4-1.5.8
networkx-2.6.3
nose-1.3.7
numba-0.54.1
numpy-1.21.4
olefile-0.46
ordereddict-1.1
packaging-20.9
pandas-1.3.4
pandocfilters-1.5.0
panel-0.12.5
param-1.12.0
paramiko-2.8.0
parso-0.8.1
partd-1.2.0
path.py-12.0.1
pathtools-0.1.2
pbr-5.8.0
pep517-0.9.1
pexpect-4.8.0
pickleshare-0.7.5
Pillow-8.4.0
pip-21.1.3
pkgconfig-1.5.5
platformdirs-2.4.0
pluggy-1.0.0
ply-3.11
poetry-core-1.0.7
pretend-1.0.9
prompt-toolkit-3.0.19
psutil-5.8.0
ptyprocess-0.7.0
py-1.11.0
pyasn1-0.4.8
pyasn1-modules-0.2.8
pybind11-2.8.1
pycares-4.1.2
pycodestyle-2.8.0
pycparser-2.21
py-cpuinfo-8.0.0
pyct-0.4.8
pyflakes-2.4.0
Pygments-2.10.0
PyHamcrest-2.0.2
pymemcache-3.5.0
pynacl-1.4.0
pyopenssl-20.0.1
pyparsing-2.4.7
pyperf-2.3.0
pyroma-3.2
pyrsistent-0.18.0
pysocks-1.7.1
pyspnego-0.3.1
PyStemmer-2.0.1
pytest-5.4.3
pytest-6.2.5
pytest-asyncio-0.16.0
pytest-benchmark-3.2.2
pytest-cov-2.12.1
pytest-expect-1.1.0
pytest-flake8-1.0.7
pytest-forked-1.3.0
pytest-freezegun-0.4.2
pytest-httpbin-1.0.0
pytest-mock-3.6.1
pytest-mpl-0.13
pytest-rerunfailures-10.2
pytest-runner-5.3.1
pytest-subtests-0.5.0
pytest-timeout-1.4.2
pytest-vcr-1.0.2
pytest-xdist-2.4.0
pytest-xprocess-0.18.1
python-dateutil-2.8.2
python-dotenv-0.19.2
python-lz4-3.1.10
python-snappy-0.6.0
pythran-0.9.12
pytoml-0.1.20
pytz-2021.3
pyviz_comms-2.1.0
PyYAML-5.4.1.1
pyzmq-22.3.0
raven-6.10.0
re_assert-1.1.0
redis-3.5.3
regex-2021.11.10
reportlab-3.6.2
requests-2.26.0
rich-10.14.0
ruamel-base-1.0.0
ruamel-yaml-0.17.16
ruamel-yaml-clib-0.2.4
scipy-1.7.3
semantic_version-2.8.5
service-identity-21.1.0
setuptools-57.2.0
setuptools-rust-0.12.1
setuptools-scm-6.3.2
setuptools-scm-git-archive-1.1
sh-1.14.2
simplejson-3.17.6
six-1.16.0
smbprotocol-1.8.3
snowballstemmer-2.2.0
sortedcontainers-2.4.0
soupsieve-2.3.1
sphinx-4.3.1
sphinxcontrib-applehelp-1.0.2
sphinxcontrib-devhelp-1.0.2
sphinxcontrib-htmlhelp-2.0.0
sphinxcontrib-jsmath-1.0.1
sphinxcontrib-qthelp-1.0.3
sphinxcontrib-serializinghtml-1.1.5
sphinxcontrib-websupport-1.2.4
sqlparse-0.4.2
sympy-1.9
tblib-1.7.0
testpath-0.5.0
thrift-0.15.0
tkinter-3.9.6
toml-0.10.2
tomli-1.2.2
tomlkit-0.7.2
toolz-0.11.2
tornado-6.1
tqdm-4.62.3
traitlets-5.1.1
trustme-0.9.0
Twisted-21.7.0
typing_extensions-3.10.0.2
tzdata-2021.1
u-msgpack-python-2.7.1
urllib3-1.26.7
vcrpy-4.1.1
vcversioner-2.16.0.0
virtualenv-20.10.0
watchdog-2.1.6
wcwidth-0.2.5
webencodings-0.5.1
werkzeug-2.0.1
wheel-0.36.2
wrapt-1.13.3
xarray-0.20.1
XlsxWriter-1.2.9
yarl-1.7.2
zict-2.0.0
zipp-3.6.0
zope.event-4.5.0
zope.interface-5.4.0
zopfli-0.1.9
zstd-1.5.0.2

Description of expected behavior and the observed behavior

Trying to run test suite with latest versions of each dependency.

Complete, minimal, self-contained example code that reproduces the issue

pytest

Stack traceback and/or browser JavaScript console output

_____________ test_raster_quadmesh_autorange_reversed[dask.array] ______________
[gw0] linux -- Python 3.9.6 /nix/store/18lvrpzq5skcc1gl2jfsz4pynddafl4g-python3-3.9.6/bin/python3.9

array_module = <module 'dask.array' from '/nix/store/wazh9yqw5dg6xlp728hbf4q4rmd3yl1a-python3.9-dask-2021.10.0/lib/python3.9/site-packages/dask/array/__init__.py'>

    @pytest.mark.parametrize('array_module', array_modules)
    def test_raster_quadmesh_autorange_reversed(array_module):
        c = ds.Canvas(plot_width=8, plot_height=4)
        da = xr.DataArray(
            array_module.array(
                [[1, 2, 3, 4],
                 [5, 6, 7, 8]]
            ),
            coords=[('b', [-1, -2]),
                    ('a', [-1, -2, -3, -4])],
            name='Z')

        y_coords = np.linspace(-2.25, -0.75, 4)
        x_coords = np.linspace(-4.25, -0.75, 8)
        out = xr.DataArray(array_module.array(
            [[8., 8., 7., 7., 6., 6., 5., 5.],
             [8., 8., 7., 7., 6., 6., 5., 5.],
             [4., 4., 3., 3., 2., 2., 1., 1.],
             [4., 4., 3., 3., 2., 2., 1., 1.]],
            dtype='f8'),
            coords=[('b', y_coords),
                    ('a', x_coords)]
        )

        res = c.quadmesh(da, x='a', y='b', agg=ds.sum('Z'))
>       assert_eq_xr(res, out)

datashader/tests/test_quadmesh.py:213:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

agg = <xarray.DataArray (b: 4, a: 8)>
array([[2.79797330e-316, 0.00000000e+000, 0.00000000e+000,
        0.00000000e+000, 3....* a        (a) float64 -4.25 -3.75 -3.25 -2.75 -2.25 -1.75 -1.25 -0.75
  * b        (b) float64 -2.25 -1.75 -1.25 -0.75
b = <xarray.DataArray 'array-9c1c69efb2f94e71e88c3025373a3534' (b: 4, a: 8)>
dask.array<array, shape=(4, 8), dtype=float64...* b        (b) float64 -2.25 -1.75 -1.25 -0.75
  * a        (a) float64 -4.25 -3.75 -3.25 -2.75 -2.25 -1.75 -1.25 -0.75
close = False

    def assert_eq_xr(agg, b, close=False):
        """Assert that two xarray DataArrays are equal, handling the possibility
        that the two DataArrays may be backed by ndarrays of different types"""
        if cupy:
            if isinstance(agg.data, cupy.ndarray):
                agg = xr.DataArray(
                    cupy.asnumpy(agg.data), coords=agg.coords, dims=agg.dims
                )
            if isinstance(b.data, cupy.ndarray):
                b = xr.DataArray(
                    cupy.asnumpy(b.data), coords=b.coords, dims=b.dims
                )
        if close:
            xr.testing.assert_allclose(agg, b)
        else:
>           xr.testing.assert_equal(agg, b)
E           AssertionError: Left and right DataArray objects are not equal
E
E           Differing values:
E           L
E               array([[2.797973e-316, 0.000000e+000, 0.000000e+000, 0.000000e+000,
E                       3.622051e-316, 1.976263e-323, 3.622049e-316, 1.581010e-322],
E                      [3.623886e-316, 1.976263e-323, 3.623884e-316, 1.383384e-322,
E                       3.957741e-316, 1.976263e-323, 3.957738e-316, 2.420922e-322],
E                      [3.957792e-316, 1.976263e-323, 3.957786e-316, 5.187689e-322,
E                       3.697745e-316, 1.976263e-323, 3.697742e-316, 2.519735e-322],
E                      [3.697758e-316, 1.976263e-323, 3.697754e-316, 2.914987e-322,
E                       2.945559e-316, 1.976263e-323, 0.000000e+000, 0.000000e+000]])
E           R
E               array([[8., 8., 7., 7., 6., 6., 5., 5.],
E                      [8., 8., 7., 7., 6., 6., 5., 5.],
E                      [4., 4., 3., 3., 2., 2., 1., 1.],
E                      [4., 4., 3., 3., 2., 2., 1., 1.]])

datashader/tests/test_pandas.py:104: AssertionError

Screenshots or screencasts of the bug in action

[ianthomas23]

@jonringer Sorry it has taken so long to respond to this.

Your list of installed packages shows an inconsistency:

dask-2021.10.0
distributed-2021.11.2

If I try to install these two packages in a clean virtual environment:

> pip install dask==2021.10.0 distributed==2021.11.2
ERROR: distributed 2021.11.2 has requirement dask==2021.11.2, but you'll have dask 2021.10.0 which is incompatible.

So I think the action here is to delete your python environment and create a new one with consistent dask and distributed.

[jonringer]

Since I created the original issue, I've created the following explanation of what I'm doing:

Disclaimer:
I'm aware that the version used in this package may be outside of the expect range of package versions. However, python requires a single version of a given package in any runtime environment; so we are forced to try and have as coherent of a package set as possible. Generally we prefer having the latest of each package, which will often be outside of a pinned version range. This issue is meant to be more of a "inform upstream that there is a [potential] issue" rather than "we expect upstream to immediately react to the issue".