2 tests failing on MacOS CI build (but the same tests are passing on Linux CI build!)

[JackKelly]

But, strangely, the tests are passing on Linux CI build! So maybe this isn't a priority, given that our ML server is Linux?

Here's the failure... looks like it's a failure to download metadata from ESO?

https://github.com/openclimatefix/nowcasting_dataset/runs/4223005327?check_suite_focus=true

pytest -s -n 2 --dist=loadfile    
Unable to revert mtime: /Library/Fonts
============================= test session starts ==============================
platform darwin -- Python 3.9.8, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/runner/work/nowcasting_dataset/nowcasting_dataset
plugins: xdist-2.4.0, forked-1.3.0, cov-3.0.0
gw0 I / gw1 I
gw0 [123] / gw1 [123]


  0%|          | 0/100 [00:00<?, ?it/s]
  7%|▋         | 7/100 [00:00<00:01, 62.71it/s]
 14%|█▍        | 14/100 [00:00<00:01, 65.21it/s]
 21%|██        | 21/100 [00:00<00:01, 48.94it/s]
100%|██████████| 100/100 [00:00<00:00, 230.09it/s]

  0%|          | 0/100 [00:00<?, ?it/s]
 21%|██        | 21/100 [00:00<00:00, 204.79it/s]
 42%|████▏     | 42/100 [00:00<00:01, 46.33it/s] 
 53%|█████▎    | 53/100 [00:00<00:00, 51.23it/s]
 62%|██████▏   | 62/100 [00:01<00:00, 57.02it/s]
 71%|███████   | 71/100 [00:01<00:00, 61.35it/s]
 80%|████████  | 80/100 [00:01<00:00, 48.38it/s]
 91%|█████████ | 91/100 [00:01<00:00, 56.21it/s]
100%|██████████| 100/100 [00:01<00:00, 59.49it/s]

  0%|          | 0/100 [00:00<?, ?it/s]
  8%|▊         | 8/100 [00:00<00:01, 70.64it/s]
 21%|██        | 21/100 [00:00<00:00, 85.30it/s]
 30%|███       | 30/100 [00:00<00:01, 55.61it/s]
100%|██████████| 100/100 [00:00<00:00, 199.59it/s]

  0%|          | 0/100 [00:00<?, ?it/s]
 25%|██▌       | 25/100 [00:00<00:00, 111.93it/s]
 37%|███▋      | 37/100 [00:00<00:00, 102.45it/s]
 47%|████▋     | 47/100 [00:00<00:00, 73.80it/s] 
 55%|█████▌    | 55/100 [00:00<00:00, 64.91it/s]
 62%|██████▏   | 62/100 [00:01<00:00, 38.09it/s]
 77%|███████▋  | 77/100 [00:01<00:00, 46.34it/s]
 84%|████████▍ | 84/100 [00:01<00:00, 49.11it/s]
100%|██████████| 100/100 [00:01<00:00, 66.20it/s]

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:36<00:00, 36.59s/it]
100%|██████████| 1/1 [00:36<00:00, 36.59s/it]

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:33<00:00, 33.04s/it]
100%|██████████| 1/1 [00:33<00:00, 33.04s/it]

  0%|          | 0/1 [00:00<?, ?it/s]
100%|██████████| 1/1 [00:34<00:00, 34.67s/it]
100%|██████████| 1/1 [00:34<00:00, 34.67s/it]

  0%|          | 0/2 [00:00<?, ?it/s]
 50%|█████     | 1/2 [00:33<00:33, 33.25s/it]
100%|██████████| 2/2 [00:34<00:00, 14.14s/it]
100%|██████████| 2/2 [00:34<00:00, 17.01s/it]

  0%|          | 0/10 [00:00<?, ?it/s]
 10%|█         | 1/10 [00:29<04:29, 29.97s/it]
 50%|█████     | 5/10 [01:04<00:58, 11.72s/it]
 90%|█████████ | 9/10 [01:33<00:09,  9.37s/it]
100%|██████████| 10/10 [01:33<00:00,  9.38s/it]
....................ss...............s.......F..s..............F..........................................ssssss...........
=================================== FAILURES ===================================
___________ test_sample_spatial_and_temporal_locations_for_examples ____________
[gw1] darwin -- Python 3.9.8 /Users/runner/hostedtoolcache/Python/3.9.8/x64/bin/python

    def test_sample_spatial_and_temporal_locations_for_examples():  # noqa: D103
        local_path = Path(nowcasting_dataset.__file__).parent.parent
    
>       gsp = GSPDataSource(
            zarr_path=f"{local_path}/tests/data/gsp/test.zarr",
            start_dt=datetime(2019, 1, 1),
            end_dt=datetime(2019, 1, 2),
            history_minutes=30,
            forecast_minutes=60,
            image_size_pixels=64,
            meters_per_pixel=2000,
        )

tests/test_manager.py:19: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
<string>:12: in __init__
    ???
nowcasting_dataset/data_sources/gsp/gsp_data_source.py:64: in __post_init__
    self.load()
nowcasting_dataset/data_sources/gsp/gsp_data_source.py:84: in load
    self.metadata = get_gsp_metadata_from_eso()
nowcasting_dataset/data_sources/gsp/eso.py:73: in get_gsp_metadata_from_eso
    shape_data = get_gsp_shape_from_eso()
nowcasting_dataset/data_sources/gsp/eso.py:131: in get_gsp_shape_from_eso
    shape_gpd = gpd.read_file(response)
../../../hostedtoolcache/Python/3.9.8/x64/lib/python3.9/site-packages/geopandas/io/file.py:174: in _read_file
    data = filename.read()
../../../hostedtoolcache/Python/3.9.8/x64/lib/python3.9/http/client.py:476: in read
    s = self._safe_read(self.length)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <http.client.HTTPResponse object at 0x12d1d46d0>, amt = 20901

    def _safe_read(self, amt):
        """Read the number of bytes requested, compensating for partial reads.
    
        Normally, we have a blocking socket, but a read() can be interrupted
        by a signal (resulting in a partial read).
    
        Note that we cannot distinguish between EOF and an interrupt when zero
        bytes have been read. IncompleteRead() will be raised in this
        situation.
    
        This function should be used when <amt> bytes "should" be present for
        reading. If the bytes are truly not available (due to EOF), then the
        IncompleteRead exception can be used to detect the problem.
        """
        s = []
        while amt > 0:
            chunk = self.fp.read(min(amt, MAXAMOUNT))
            if not chunk:
>               raise IncompleteRead(b''.join(s), amt)
E               http.client.IncompleteRead: IncompleteRead(3342194 bytes read, 20901 more expected)

../../../hostedtoolcache/Python/3.9.8/x64/lib/python3.9/http/client.py:628: IncompleteRead
________________________ test_get_pv_gsp_shape_from_eso ________________________
[gw1] darwin -- Python 3.9.8 /Users/runner/hostedtoolcache/Python/3.9.8/x64/bin/python

    def test_get_pv_gsp_shape_from_eso():
        """
        Test to get the gsp metadata from eso. This should take ~1 second.
        @return:
        """
    
>       gsp_shapes = get_gsp_shape_from_eso(load_local_file=False)

tests/data_sources/gsp/test_gsp_metadata.py:64: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
nowcasting_dataset/data_sources/gsp/eso.py:131: in get_gsp_shape_from_eso
    shape_gpd = gpd.read_file(response)
../../../hostedtoolcache/Python/3.9.8/x64/lib/python3.9/site-packages/geopandas/io/file.py:174: in _read_file
    data = filename.read()
../../../hostedtoolcache/Python/3.9.8/x64/lib/python3.9/http/client.py:476: in read
    s = self._safe_read(self.length)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <http.client.HTTPResponse object at 0x12d4af1c0>, amt = 47525

    def _safe_read(self, amt):
        """Read the number of bytes requested, compensating for partial reads.
    
        Normally, we have a blocking socket, but a read() can be interrupted
        by a signal (resulting in a partial read).
    
        Note that we cannot distinguish between EOF and an interrupt when zero
        bytes have been read. IncompleteRead() will be raised in this
        situation.
    
        This function should be used when <amt> bytes "should" be present for
        reading. If the bytes are truly not available (due to EOF), then the
        IncompleteRead exception can be used to detect the problem.
        """
        s = []
        while amt > 0:
            chunk = self.fp.read(min(amt, MAXAMOUNT))
            if not chunk:
>               raise IncompleteRead(b''.join(s), amt)
E               http.client.IncompleteRead: IncompleteRead(3315570 bytes read, 47525 more expected)

../../../hostedtoolcache/Python/3.9.8/x64/lib/python3.9/http/client.py:628: IncompleteRead
=============================== warnings summary ===============================
tests/test_geospatial.py::test_osgb_to_lat_lon
  /Users/runner/hostedtoolcache/Python/3.9.8/x64/lib/python3.9/site-packages/pyproj/transformer.py:184: UserWarning: Best transformation is not available due to missing Grid(short_name=uk_os_OSTN15_NTv2_OSGBtoETRS.tif, full_name=, package_name=, url=https://cdn.proj.org/uk_os_OSTN15_NTv2_OSGBtoETRS.tif, direct_download=True, open_license=True, available=False)
    super().__init__(

tests/data_sources/test_pv_data_source.py::test_get_example_and_batch
  /Users/runner/hostedtoolcache/Python/3.9.8/x64/lib/python3.9/site-packages/xarray/core/indexes.py:97: FutureWarning: Indexing a timezone-naive DatetimeIndex with a timezone-aware datetime is deprecated and will raise KeyError in a future version.  Use a timezone-naive object instead.
    indexer = index.slice_indexer(

tests/data_sources/gsp/test_gsp_data_source.py: 16 warnings
tests/test_manager.py: 8 warnings
tests/data_sources/gsp/test_gsp_metadata.py: 8 warnings
  /Users/runner/hostedtoolcache/Python/3.9.8/x64/lib/python3.9/site-packages/pandas/core/indexing.py:1773: SettingWithCopyWarning: 
  A value is trying to be set on a copy of a slice from a DataFrame.
  Try using .loc[row_indexer,col_indexer] = value instead
  
  See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
    self._setitem_single_column(ilocs[0], value, pi)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_manager.py::test_sample_spatial_and_temporal_locations_for_examples
FAILED tests/data_sources/gsp/test_gsp_metadata.py::test_get_pv_gsp_shape_from_eso
====== 2 failed, 111 passed, 10 skipped, 34 warnings in 763.40s (0:12:43) ======
Error: Process completed with exit code 1.

[peterdudfield]

Seems to only fail sometimes

another example - https://github.com/openclimatefix/nowcasting_dataset/runs/4223150100?check_suite_focus=true

[JackKelly]

Seems to only fail sometimes

Maybe the ESO website sometimes goes down briefly?

[peterdudfield]

strange error, and doesnt fail on

Seems to only fail sometimes

Maybe the ESO website sometimes goes down briefly?

maybe, but its happening often enough, that I'm not sure

[peterdudfield]

I might try changing

    fileobj = urllib.request.urlopen(url)
    d = json.loads(fileobj.read())

to

    with urllib.request.urlopen(url) as fileobj:
        d = json.loads(fileobj.read())

just copying - https://docs.python.org/3/howto/urllib2.html

[peterdudfield]

I think there are some issue with - pvlive - I have notified Jamie, and he'll have a look

[peterdudfield]

server was reset on their side, and it seems better now

This python tests worked alot better: https://github.com/openclimatefix/nowcasting_dataset/actions/runs/1466878711

[peterdudfield]

Slowed down again - I'm going to skip tests as they are not critical for nowcasting dataset -they are only used to building the GSP dataset.

[peterdudfield]

Ill keep this open for the moment