PseudoNetCDF tests failing randomly

[keewis]

The py37-windows CI seems to fail for newer PRs:

_______________ TestPseudoNetCDFFormat.test_uamiv_format_write ________________

self = <xarray.tests.test_backends.TestPseudoNetCDFFormat object at 0x000002E11FF2DC08>

    def	test_uamiv_format_write(self):
	fmtkw =	{"format": "uamiv"}

	expected = open_example_dataset(
            "example.uamiv", engine="pseudonetcdf", backend_kwargs=fmtkw
	)
	with self.roundtrip(
	    expected,
            save_kwargs=fmtkw,
	    open_kwargs={"backend_kwargs": fmtkw},
            allow_cleanup_failure=True,
	) as actual:
>           assert_identical(expected, actual)

xarray\tests\test_backends.py:3532:
_ _ _ _	_ _ _ _	_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _
xarray\core\formatting.py:628: in diff_dataset_repr
    summary.append(diff_attrs_repr(a.attrs, b.attrs, compat))
_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _	_ _ _ _

a_mapping = {'CPROJ': 0, 'FILEDESC': 'CAMx            ', 'FTYPE': 1, 'GDNAM': 'CAMx            ', ...}
b_mapping = {'CPROJ': 0, 'FILEDESC': 'CAMx            ', 'FTYPE': 1, 'GDNAM': 'CAMx            ', ...}
compat = 'identical', title = 'Attributes'
summarizer = <function summarize_attr at 0x000002E1156813A8>, col_width = None

    def _diff_mapping_repr(a_mapping, b_mapping, compat, title, summarizer, col_width=None):
        def extra_items_repr(extra_keys, mapping, ab_side):
            extra_repr = [summarizer(k, mapping[k], col_width) for k in extra_keys]
            if extra_repr:
                header = f"{title} only on the {ab_side} object:"
                return [header] + extra_repr
            else:
                return []

	a_keys = set(a_mapping)
        b_keys = set(b_mapping)

        summary = []

	diff_items = []

        for k in a_keys & b_keys:
	    try:
                # compare xarray variable
                compatible = getattr(a_mapping[k], compat)(b_mapping[k])
                is_variable = True
            except AttributeError:
                # compare attribute value
                compatible = a_mapping[k] == b_mapping[k]
                is_variable = False

>           if not compatible:
E           ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

[dcherian]

cc @barronh

[barronh]

I want to make sure I understand the genesis of the error. My guess is that if you added a print(k) statement, you'd see that this is failing on the VGLVLS attribute. Is that right?

If so, I'm guessing the error is that VGLVLS is not a scalar. NetCDF files and uamiv files may have attributes with values. This is the case for VGLVLS in the IOAPI format, which uamiv is made to emulate.

As a result, compatible is an array not a scalar. A simple solution would be to wrap compatible in a call to all.

Can you confirm which attribute this is failing on and what the value of compatible is when it fails?

[keewis]

the dataset is from xarray/tests/data/example.uamiv and when opening it we can see that VGLVLS is indeed not a scalar:

    VGLVLS:    [1. 0.]

While trying to reproduce the error, I noticed that the bit of code above does not get called unless expected.identical(actual) == False. This means we actually have two bugs: one that makes _diff_mapping_repr choke on non-scalar (ndarray?) attributes and should be easy to fix, and another one that to me seems random (look at the commits of #3706: the last two don't fail while 1e07dce fails) and makes expected different from actual.

[barronh]

Let me know if you need my input, but I think the testcase solution is more general than PseudoNetCDF.

[keewis]

I just noticed that this is not restricted to windows, this also randomly fails for other CI suites.

#3728 fixes the formatting issue so now the cause of the errors should not be hidden anymore.

[max-sixty]

Should we disable this one test if there's not an immediate solution to hand?