run all tests #142

ocefpaf · 2022-09-15T16:15:50Z

This is something we should've done a long time ago.

conda-forge-linter · 2022-09-15T16:15:54Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

ocefpaf · 2022-09-28T13:24:35Z

There are 8 tests failing on ppc64le:

ERROR: runTest (tst_multifile.NonuniformTimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile.py", line 126, in runTest
    dates.extend(cftime.num2date(t[:], t.units, calendar))
  File "src/cftime/_cftime.pyx", line 586, in cftime._cftime.num2date
  File "src/cftime/_cftime.pyx", line 385, in cftime._cftime.cast_to_int
OverflowError: time values outside range of 64 bit signed integers

======================================================================
ERROR: runTest (tst_multifile2.NonuniformTimeTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile2.py", line 114, in runTest
    dates.extend(cftime.num2date(t[:], t.units, t.calendar))
  File "src/cftime/_cftime.pyx", line 586, in cftime._cftime.num2date
  File "src/cftime/_cftime.pyx", line 385, in cftime._cftime.cast_to_int
OverflowError: time values outside range of 64 bit signed integers

======================================================================
FAIL: runTest (tst_vlen.Vlen_ScaledInts)
testing packing float vlens as scaled integers (issue #1003).
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_vlen.py", line 220, in runTest
    assert(err.max() < nc['vl'].scale_factor)
AssertionError

======================================================================
FAIL: runTest (tst_masked.PrimitiveTypesTestCase)
testing auto-conversion of masked arrays and packed integers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_masked.py", line 118, in runTest
    assert_array_equal(datapacked[:],packeddata2)
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 934, in assert_array_equal
    assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatched elements: 2 / 10 (20%)
Max absolute difference: 20271
Max relative difference: 1.54375143
 x: array([-32134,  30448,   5655,   3599,      0,      0,      0,      0,
       -27211, -32766], dtype=int16)
 y: array([ 13131,  19120,   5655,   3599,      0,      0,      0,      0,
       -27211, -32766], dtype=int16)

======================================================================
FAIL: runTest (tst_masked2.PrimitiveTypesTestCase)
testing auto-conversion of masked arrays and packed integers
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_masked2.py", line 118, in runTest
    assert_array_almost_equal(data,datacheck4)
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1046, in assert_array_almost_equal
    assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 768, in assert_array_compare
    flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 745, in func_assert_same_pos
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

x and y nan location mismatch:
 x: MaskedArray([nan, nan,  0.,  0.], dtype=float32)
 y: MaskedArray([1.5625,    inf, 3.75  , 4.125 ], dtype=float32)

======================================================================
FAIL: runTest (tst_multifile.VariablesTestCase)
testing multi-file dataset access
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile.py", line 52, in runTest
    assert_array_equal(np.arange(0,nx),f.variables['x'][:])
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 934, in assert_array_equal
    assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatched elements: 79 / 100 (79%)
Max absolute difference: 471670300
Max relative difference: 97.
 x: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,...
 y: array([        0,         0, 202182159, 471670303,         0,         0,
              -1,  16777215,         8,         9,         0,         0,
               3,         5,         0,         0,        -1,  16777215,...

======================================================================
FAIL: runTest (tst_multifile2.VariablesTestCase)
testing multi-file dataset access
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_multifile2.py", line 52, in runTest
    assert_array_equal(np.arange(0,nx),f.variables['x'][:])
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 934, in assert_array_equal
    assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatched elements: 80 / 100 (80%)
Max absolute difference: 812388678
Max relative difference: 97.
 x: array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16,
       17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33,
       34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50,...
 y: array([415223568,  44201712,  67438087, 202182159,         0,         0,
              -1,  16777215,         8,         9, 812388688,  44201712,
               8,         6,         0,         0,        -1,  16777215,...

======================================================================
FAIL: test_unmasked (tst_scaled.SetAutoScaleTrue)
Testing auto-scaling of variables for set_auto_scale(True)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_scaled.py", line 116, in test_unmasked
    assert_array_almost_equal(vv_scaled, np.ones(10))
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1046, in assert_array_almost_equal
    assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

Mismatched elements: 8 / 10 (80%)
Max absolute difference: 32135.
Max relative difference: 32135.
 x: MaskedArray([-3.2134e+04,  3.0448e+04,  5.6550e+03,  3.5990e+03,
              0.0000e+00,  0.0000e+00,  0.0000e+00,  0.0000e+00,
              1.0000e+00,  1.0000e+00], dtype=float32)
 y: array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1.])

======================================================================
FAIL: test_auto_scale_write (tst_scaled.WriteAutoScaleTest)
Testing automatic packing to all kinds of integer types
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_scaled.py", line 190, in test_auto_scale_write
    assert(maxerrnorm < 0.5 + 1e-5)
AssertionError

======================================================================
FAIL: runTest (tst_types.PrimitiveTypesTestCase)
testing primitive data type
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/test_tmp/test/tst_types.py", line 69, in runTest
    assert_array_almost_equal(datarr,ranarr[1:n1dim].astype(data.dtype))
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 1046, in assert_array_almost_equal
    assert_array_compare(compare, x, y, err_msg=err_msg, verbose=verbose,
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 768, in assert_array_compare
    flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
  File "/home/conda/feedstock_root/build_artifacts/netcdf4_1663263851859/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/lib/python3.10/site-packages/numpy/testing/_private/utils.py", line 745, in func_assert_same_pos
    raise AssertionError(msg)
AssertionError: 
Arrays are not almost equal to 6 decimals

x and y nan location mismatch:
 x: array([[ 2.350989e-38,           nan,  3.030902e+24,  7.134550e+31,
                  nan,           nan,  8.923098e-32,  3.500230e-27,
        -2.458268e-38,  1.775738e+36],...
 y: array([[ 2.350989e-38,           nan,  1.485376e-43,           nan,
                  nan,           nan,  8.923098e-32,  3.500230e-27,
        -2.458268e-38,  1.775738e+36],...

----------------------------------------------------------------------
Ran 97 tests in 109.403s

FAILED (failures=8, errors=2, skipped=1)

I don't think that ppc64le is used by many or if it is even support upstream. @conda-forge/netcdf4 team should we try to fix these or just skip?

xylar · 2022-09-29T03:34:32Z

@ocefpaf, I'm afraid I don't know about whether the tests are expected to pass on ppc64le. I do know that we're quite interested in having conda-forge package support on this architecture because some of the newer HPC systems use it and our testing software can't currently be supported.

@dopplershift, any insight into whether we would expect these tests to pass?

dopplershift · 2022-09-29T08:23:44Z

@WardF Any insights on testing on ppc64le?

ocefpaf · 2022-09-29T13:23:09Z

I do know that we're quite interested in having conda-forge package support on this architecture because some of the newer HPC systems

Good to know!

WardF · 2022-09-29T14:23:04Z

@dopplershift I would expect the libnetcdf tests to pass on ppc64le (and indeed they do, over at libnetcdf-feedstock). I would have to dig into the failures observed here to see if they are occurring at the C library level (in which case I can probably help) or somewhere in the Python code (in which case I am less equipped to help). But to answer the specific question, ppc64le is one of the platforms we support, and if something is broken in the libnetcdf library, we would need to fix it.

WardF · 2022-09-29T15:33:13Z

Thinking more about this, it is possible that the python testing is covering code that's not being tested in libnetcdf-feedstock or in the netcdf-c code. If somebody more knowledgeable about Python can make the determination that the issue is the C library, I'm happy to jump in and 1) expand our testing and 2) fix the underlying code so that the tests pass.

xylar · 2022-09-29T21:24:59Z

I went ahead and posted an issue: Unidata/netcdf4-python#1196

@jswhit, as the main developer I'm wondering if you would be willing to comment either there or here.

jswhit · 2022-09-30T20:51:30Z

so this is a new thing (testing on ppc64le) and not something that used to pass and now doesn't?

xylar · 2022-09-30T21:06:24Z

@jswhit, yes, that's right. Some folks ran into trouble with the last release (see #141) and @ocefpaf decided to turn on a lot more testing, presumably to see if there were any obvious problems with the conda-forge build. In that process, everything worked great except ppc64le). So these tests have never been run before by us. If we were to ignore them, that would be "business as usual" for us. But we're trying to be more diligent than that :-)

jswhit · 2022-09-30T21:24:41Z

some of the test failures are for cftime and appear to be related to be overflowing the range of 64 bit integers when casting from a float. I wonder if it could be a problem with the version of numpy being used.

xylar · 2022-09-30T21:28:49Z

I wonder if it could be a problem with the version of numpy being used.

The ppc64le build is using numpy 1.20 (1.21 for python 3.10), the same as the other linux builds that pass. So it doesn't seem obvious to me that the numpy version could explain any differences unless there is a problem with the ppc64le build of numpy 1.20 that isn't in the others.

jswhit · 2022-09-30T23:49:07Z

I wonder if it could be a problem with the version of numpy being used.

The ppc64le build is using numpy 1.20 (1.21 for python 3.10), the same as the other linux builds that.So it doesn't seem obvious to me that the numpy version could explain any differences unless there is a problem with the ppc64le build of numpy 1.20 that isn't in the others.

That's just a wild guess - how hard would it be to try numpy 1.22 or 1.23?

xylar · 2022-10-01T01:33:42Z

That's just a wild guess - how hard would it be to try numpy 1.22 or 1.23?

I'll look into doing that for testing purposes but part of what makes conda-forge work is that we have specific versions of packages like numpy that we use consistently across the ecosystem of packages. So if that's a requirement, we'd need to disable ppc64le support until we're ready to migrate to a newer numpy.

xylar · 2022-10-01T02:50:29Z

@ocefpaf, if this works, maybe it would be possible to pin a higher version of numpy? According to the knowledge base, this is okay but what would be downstream consequences be? Also, could we pin just for ppc64le? I'm almost certainly jumping the gun since we don't know if it will work...

xylar · 2022-10-01T03:25:11Z

No luck with numpy 1.22. let's try 1.23.

xylar · 2022-10-01T04:41:53Z

@jswhit, I appreciate the recommendation but no luck there. Let us know if anything else comes to mind.

ocefpaf · 2022-10-03T11:39:44Z

BTW, we do run all the tests in the cftime feedstock and we do not catch that error. It is possible that the tests do not cover this problem.

xylar · 2022-10-05T02:38:22Z

@ocefpaf, I think it's better at this point to just disable the tests for ppc64le as you suggested earlier. It seems better to have the tests for other architectures than not at all. If we or someone else has a chance to investigate the failures at a later time, that would be great! But we have to be realistic about our limited time right now.

ocefpaf · 2022-10-05T14:45:03Z

@xylar we don't need new artifacts to be built, so we can wait for a new release and just rebase here. I'm OK keeping this as a draft until then.

xylar · 2022-11-17T10:24:03Z

@conda-forge-admin, please rerender

xylar · 2022-11-17T10:24:36Z

@ocefpaf, do we want to explore this again following #146?

github-actions · 2022-11-17T10:26:36Z

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/netcdf4-feedstock/actions/runs/3487394746.

ocefpaf · 2022-11-17T15:45:06Z

@ocefpaf, do we want to explore this again following #146?

Rebased and restarted. Let's see how it goes.

xylar · 2022-11-17T16:32:15Z

Windows tests are passing, so that's the one that I was concerned about. the alternative linux architectures are clearly still having trouble.

It's frustrating that so many folks are having trouble with DLLs in the windows version but I don't have any relevant expertise to help there or to debug with folks if it's just user error.

ocefpaf · 2022-11-17T18:00:31Z

It's frustrating that so many folks are having trouble with DLLs in the windows version but I don't have any relevant expertise to help there or to debug with folks if it's just user error.

Are folks reporting to you directly these failures? Are those with the conda-forge package? Sorry if I missed any important ping, I'm quite slow lately with my notifications.

xylar · 2022-11-17T18:08:03Z

I'm thinking of the comments on #136 that keep coming in and then #145 just posted yesterday or the day before.

xylar · 2022-11-17T18:08:22Z

I don't work directly with anyone using Windows as far as I'm aware.

ocefpaf · 2022-11-17T18:38:08Z

I don't work directly with anyone using Windows as far as I'm aware.

Lucky you!

I'm thinking of the comments on #136 that keep coming in and then #145 just posted yesterday or the day before.

They are unrelated to netcdf4 per-se. I commented on both issues.

xylar · 2022-11-17T21:06:08Z

@ocefpaf, great I'm glad you were able to help!

ocefpaf requested review from dopplershift, pelson and xylar as code owners September 15, 2022 16:15

xylar mentioned this pull request Sep 29, 2022

Seeing test failures on conda-forge building for ppc64le Unidata/netcdf4-python#1196

Open

xylar force-pushed the run_tests branch from 6defa58 to cb69288 Compare November 17, 2022 10:23

ocefpaf added 2 commits November 17, 2022 12:44

run all tests

82bd6c4

bump

a822298

ocefpaf force-pushed the run_tests branch from cb69288 to a822298 Compare November 17, 2022 15:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

run all tests #142

run all tests #142

ocefpaf commented Sep 15, 2022

conda-forge-linter commented Sep 15, 2022

ocefpaf commented Sep 28, 2022

xylar commented Sep 29, 2022

dopplershift commented Sep 29, 2022

ocefpaf commented Sep 29, 2022

WardF commented Sep 29, 2022 •

edited

WardF commented Sep 29, 2022

xylar commented Sep 29, 2022

jswhit commented Sep 30, 2022

xylar commented Sep 30, 2022

jswhit commented Sep 30, 2022

xylar commented Sep 30, 2022

jswhit commented Sep 30, 2022

xylar commented Oct 1, 2022 •

edited

xylar commented Oct 1, 2022

xylar commented Oct 1, 2022

xylar commented Oct 1, 2022

ocefpaf commented Oct 3, 2022

xylar commented Oct 5, 2022

ocefpaf commented Oct 5, 2022

xylar commented Nov 17, 2022

xylar commented Nov 17, 2022

github-actions bot commented Nov 17, 2022

ocefpaf commented Nov 17, 2022

xylar commented Nov 17, 2022

ocefpaf commented Nov 17, 2022

xylar commented Nov 17, 2022

xylar commented Nov 17, 2022

ocefpaf commented Nov 17, 2022

xylar commented Nov 17, 2022

run all tests #142

Are you sure you want to change the base?

run all tests #142

Conversation

ocefpaf commented Sep 15, 2022

conda-forge-linter commented Sep 15, 2022

ocefpaf commented Sep 28, 2022

xylar commented Sep 29, 2022

dopplershift commented Sep 29, 2022

ocefpaf commented Sep 29, 2022

WardF commented Sep 29, 2022 • edited

WardF commented Sep 29, 2022

xylar commented Sep 29, 2022

jswhit commented Sep 30, 2022

xylar commented Sep 30, 2022

jswhit commented Sep 30, 2022

xylar commented Sep 30, 2022

jswhit commented Sep 30, 2022

xylar commented Oct 1, 2022 • edited

xylar commented Oct 1, 2022

xylar commented Oct 1, 2022

xylar commented Oct 1, 2022

ocefpaf commented Oct 3, 2022

xylar commented Oct 5, 2022

ocefpaf commented Oct 5, 2022

xylar commented Nov 17, 2022

xylar commented Nov 17, 2022

github-actions bot commented Nov 17, 2022

ocefpaf commented Nov 17, 2022

xylar commented Nov 17, 2022

ocefpaf commented Nov 17, 2022

xylar commented Nov 17, 2022

xylar commented Nov 17, 2022

ocefpaf commented Nov 17, 2022

xylar commented Nov 17, 2022

WardF commented Sep 29, 2022 •

edited

xylar commented Oct 1, 2022 •

edited