TST: special: bump tolerances for new cdftnc regression tests

[rgommers]

One bump is small and hence business as usual. For the other one something is wrong in the code it looks like, since the test is already xfailed on 32-bit systems. There's a plan to write more cdflib tests, so that should turn up then.

This is a follow-up to PR gh-19902 and issue gh-19896.

Failures observed on macOS 14, arm64, and conda compilers (clang 16.0.6):

__________________________________________________________________________ test_nctdtr_gh19896 ___________________________________________________________________________
scipy/special/tests/test_cdflib.py:473: in test_nctdtr_gh19896
    assert_allclose(actarr, resarr, rtol=1e-12, atol=0.0)
        actarr     = [np.float64(0.9999276519560749), np.float64(0.9999276519560749), np.float64(0.9999908831755224), np.float64(0.9999990265452424), np.float64(0.3524153312279711), np.float64(0.39749697267251405), ...]
        df         = 980
        dfarr      = [0.98, 9.8, 98, 980]
        p          = 38
        pnoncarr   = [-3.8, 0.38, 3.8, 38]
        resarr     = [0.9999276519560749, 0.9999276519560749, 0.9999908831755221, 0.9999990265452424, 0.3524153312279712, 0.39749697267251416, ...]
        t          = 15
        tarr       = [0.0015, 0.15, 1.5, 15]
/Users/rgommers/mambaforge/envs/scipy-dev/lib/python3.10/contextlib.py:79: in inner
    return func(*args, **kwds)
E   AssertionError:
E   Not equal to tolerance rtol=1e-12, atol=0
E
E   Mismatched elements: 2 / 64 (3.12%)
E   Max absolute difference among violations: 3.40505402e-13
E   Max relative difference among violations: 5.18975091e-07
E    ACTUAL: array([9.999277e-01, 9.999277e-01, 9.999909e-01, 9.999990e-01,
E          3.524153e-01, 3.974970e-01, 7.168630e-01, 9.656246e-01,
E          7.234804e-05, 7.234804e-05, 3.538805e-02, 7.954827e-01,...
E    DESIRED: array([9.999277e-01, 9.999277e-01, 9.999909e-01, 9.999990e-01,
E          3.524153e-01, 3.974970e-01, 7.168630e-01, 9.656246e-01,
E          7.234804e-05, 7.234804e-05, 3.538805e-02, 7.954827e-01,...
        args       = (<function assert_allclose.<locals>.compare at 0x128c4f130>, array([9.99927652e-01, 9.99927652e-01, 9.99990883e-01, 9.....23480439e-05, 1.07506892e-02, 1.00000000e+00,
       0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]))
        func       = <function assert_array_compare at 0x103e3c790>
        kwds       = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=1e-12, atol=0', 'strict': False, ...}
        self       = <contextlib._GeneratorContextManager object at 0x103e13700>
_________________________________________________________________________ test_nctdtrinc_gh19896 _________________________________________________________________________
scipy/special/tests/test_cdflib.py:487: in test_nctdtrinc_gh19896
    assert_allclose(actual, desired, rtol=2e-12, atol=0.0)
        actual     = array([  3.09023231,   1.4061413 ,   2.01422518,  13.72706712,
       278.97656839,   3.09023231,   1.43124279,   2.01422518,
         3.71274314,  -3.0869511 ])
        desired    = [3.090232306168629, 1.406141304556198, 2.014225177124157, 13.727067118283456, 278.9765683871208, 3.090232306168629, ...]
        dfarr      = [0.001, 0.98, 9.8, 98, 980, 10000, ...]
        parr       = [0.001, 0.1, 0.3, 0.8, 0.999, 0.001, ...]
        tarr       = [0.0015, 0.15, 1.5, 15, 300, 0.0015, ...]
/Users/rgommers/mambaforge/envs/scipy-dev/lib/python3.10/contextlib.py:79: in inner
    return func(*args, **kwds)
E   AssertionError:
E   Not equal to tolerance rtol=2e-12, atol=0
E
E   Mismatched elements: 1 / 10 (10%)
E   Max absolute difference among violations: 1.19859678e-11
E   Max relative difference among violations: 3.88278512e-12
E    ACTUAL: array([  3.090232,   1.406141,   2.014225,  13.727067, 278.976568,
E            3.090232,   1.431243,   2.014225,   3.712743,  -3.086951])
E    DESIRED: array([  3.090232,   1.406141,   2.014225,  13.727067, 278.976568,
E            3.090232,   1.431243,   2.014225,   3.712743,  -3.086951])
        args       = (<function assert_allclose.<locals>.compare at 0x128c4f7f0>, array([  3.09023231,   1.4061413 ,   2.01422518,  13.7270...2518,  13.72706712,
       278.97656839,   3.09023231,   1.43124279,   2.01422518,
         3.71274314,  -3.0869511 ]))
        func       = <function assert_array_compare at 0x103e3c790>
        kwds       = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=2e-12, atol=0', 'strict': False, ...}
        self       = <contextlib._GeneratorContextManager object at 0x103e13700>
======================================================================== short test summary info =========================================================================
FAILED scipy/special/tests/test_cdflib.py::test_nctdtr_gh19896 - AssertionError:
FAILED scipy/special/tests/test_cdflib.py::test_nctdtrinc_gh19896 - AssertionError:

[skip ci]

[ilayn]

I don't quite understand how it is passing for a while and then starts to fail.

[rgommers]

It's still passing in CI, it's failing for me locally with a compiler that isn't tested in CI.

[ilayn]

I also see that the actarr build up results in numpy floats

actarr     = [np.float64(0.9999276519560749), np.float64(0.9999276519560749), np.float64(0.9999908831755224), np.float64(0.9999990265452424), ...]

Not sure, if that plays a role when comparison is done.

But the absolute difference between the offenders is 1e-13 so I don't think this is a code problem rather my hard-copied values from the old SciPy version (Fortran based) are not that reliable cross-platform.

[rgommers]

rather my hard-copied values from the old SciPy version (Fortran based) are not that reliable cross-platform.

That sounds very plausible. The original issue (gh-19896) was that nan was returned, so using rtol=1e-6 for a regression test should still be fine.

[ilayn]

Thanks @rgommers