[0.8] test_query failure

[olebole]

Hi,

I tried to build the latest sunpy version for upload as Debian package, and got the following error during test:

__________________________________ test_query __________________________________

    @example(a.Time("2006-08-01", "2006-08-01"))
>   @example(a.Time("1983-05-01", "1983-05-02"))
    # This example tests a time range with a satellite jump and no overlap
    @example(a.Time("2009-11-30", "2009-12-3"))
    @given(goes_time())
    def test_query(time):

sunpy/net/dataretriever/tests/test_goes_ud.py:63: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python2.7/dist-packages/hypothesis/executors.py:58: in default_new_style_executor
    return function(data)
/usr/lib/python2.7/dist-packages/hypothesis/core.py:111: in run
    return test(*args, **kwargs)
sunpy/net/dataretriever/tests/test_goes_ud.py:73: in test_query
    qr1 = LCClient.search(time, Instrument('XRS'))
sunpy/net/dataretriever/client.py:314: in search
    self.map_.get('TimeRange'), **kwergs)
sunpy/net/dataretriever/sources/goes.py:87: in _get_url_for_timerange
    satellitenumber = kwargs.get('satellitenumber', self._get_goes_sat_num(date))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <sunpy.net.dataretriever.sources.goes.XRSClient object at 0x7f93b86a6890>
date = datetime.datetime(1983, 5, 1, 0, 0)

    def _get_goes_sat_num(self, date):
        """[...]"""
        goes_operational = {
            2: TimeRange('1981-01-01', '1983-04-30'),
            5: TimeRange('1983-05-02', '1984-07-31'),
            6: TimeRange('1983-06-01', '1994-08-18'),
            7: TimeRange('1994-01-01', '1996-08-13'),
            8: TimeRange('1996-03-21', '2003-06-18'),
            9: TimeRange('1997-01-01', '1998-09-08'),
            10: TimeRange('1998-07-10', '2009-12-01'),
            11: TimeRange('2006-06-20', '2008-02-15'),
            12: TimeRange('2002-12-13', '2007-05-08'),
            13: TimeRange('2006-08-01', '2006-08-01'),
            14: TimeRange('2009-12-02', '2010-10-04'),
            15: TimeRange('2010-09-01', datetime.datetime.utcnow())
        }
    
        results = []
        for sat_num in goes_operational:
            if date in goes_operational[sat_num]:
                # if true then the satellite with sat_num is available
                results.append(sat_num)
    
        if results:
            # Return the newest satellite
            return max(results)
        else:
            # if no satellites were found then raise an exception
            raise ValueError('No operational GOES satellites on {}'.format(
>               date.strftime(TIME_FORMAT)))
E           ValueError: No operational GOES satellites on 1983-05-01 00:00:00

sunpy/net/dataretriever/sources/goes.py:55: ValueError
---------------------------------- Hypothesis ----------------------------------
Falsifying example: test_query(time=<Time(datetime.datetime(1983, 5, 1, 0, 0, 1), datetime.datetime(1983, 5, 1, 0, 0, 1), None)>)

[nabobalis]

Hi,

We discovered this via building our wheels. We got around it by finding out that conda hypothesis is 3.17 while on pip it is 3.18. Right now we have no fix or understanding of what causes this. However this only happens on Py3 versions and not Py2.7, which is the opposite case for your test failure. So this is strange, it could be that the server was having issues during that test.

[olebole]

So this is a test that requires online access?
For Debian builds, online access is disabled during build (and testing), which may explain its occurence.
In this case, just the @pytest.mark.online mark is missing.

[Cadair]

No this test doesn't require an internet connection, I think it is because of a change to the way hypothesis handles datetimes. I will investigate probably next week. It is however not an indication of an actual code failure.

[olebole]

Copied from #2272:

The other issue I can not replicate regardless of python version.

Then, I still think it is connected with network access. You may try to do export http_proxy=127.0.0.1:9 before running the tests; this should disable network access.

[nabobalis]

So regarding the other issue, I enabled testing and updated. Updated pytest mock to the lastest version to fix the other test error and was able to get this test failure with the updated pytest-mock package.

On both Py2 and Py3.

platform linux2 -- Python 2.7.13, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /home/nabil/Downloads/sunpy-master, inifile: setup.cfg
plugins: rerunfailures-3.0, mock-1.6.2, cov-2.5.1, hypothesis-3.12.0

platform linux -- Python 3.5.4, pytest-3.2.1, py-1.4.34, pluggy-0.4.0
rootdir: /home/nabil/Downloads/sunpy-master, inifile: setup.cfg
plugins: rerunfailures-3.0, mock-1.6.2, cov-2.5.1, hypothesis-3.12.0

I am going to see if I can pin down a package difference between our Conda CI and versions debian has.

[nabobalis]

So I updated our wheels to use the latest pip version of all the packages and we get that test failure on Py3 only.

Locally on Debian testing, I only have pytest-mock and reruns installed via pip and I once I remembered that hypothesis uses the local dir to write file and deleted that directory. I only get this test failure on Py2.

This is all very confusing for me.

What I do find troubling is why the test seems to take so long, it looks like it hangs at time. I suspect we might need to rewrite this test.

[olebole]

#2276 claims to fix this; however I still see the problem in version 0.8.1 -- just the first lines are a bit different (and confusing):

__________________________________ test_query __________________________________

    @example(a.Time("2006-08-01", "2006-08-01"))
>   # This example tests a time range with a satellite jump and no overlap
    @example(a.Time("2009-11-30", "2009-12-3"))
    @given(goes_time())
    def test_query(time):

sunpy/net/dataretriever/tests/test_goes_ud.py:63: 
[... remaining lines are identical ...]

However, this issue was also not marked as fixed anyway (on purpose?).

[nabobalis]

It has fixed this test failure that was happening on the wheel builds. The fix is more of a hack and it will be worked on more in master probably for 0.9.

I will give it a go on Debian sometime soon.

[nabobalis]

So a quick test from my end.

Installing 0.8.1 and running it using the system packages did the tests fine for 3.5 but failed for 2.7.
I updated Py2.7 hypothesis to the latest version and that seemed to fix the 2.7 issue.

[olebole]

Since this (and #2272) seems to be problems in the test and not in the code, I am going to disable the tests for the moment and upload the new Debian package.

[nabobalis]

Going to close this for now.

[olebole]

Just FYI, this problem remains on Debians s390x platform (IBM zSystems; 64 bit, bit endian) with sunpy 0.9.2. Other platforms pass.

[Cadair]

I believe this is fixed by #2691 (of all things), we still get it occasionally on our CI, it is a very irritating transient failure.