IndexError: list index out of range

[cehbrecht]

• daops version: 0.5.0
• Python version:
• Operating System:

Description

Error seen on user request on production service.

See error 41 from 26.03 in this notebook:
https://nbviewer.jupyter.org/github/roocs/rooki/blob/master/notebooks/tests/test-c3s-cmip6-subset-errors-dkrz-2021-03-23.ipynb

What I Did

Run this request:

wf = ops.Subset(
        ops.Input(
            'tas', ['c3s-cmip6.ScenarioMIP.NIMS-KMA.KACE-1-0-G.ssp245.r1i1p1f1.Amon.tas.gr.v20191217']
        ),
        time="2021-01-01/2100-12-31",
        area="-10,30,35,70"
)
resp = wf.orchestrate()
resp.status

Get into this error:

Process error: list index out of range

  File "/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/daops/utils/consolidate.py", line 42, in consolidate
    ds = open_xr_dataset(dset)
  File "/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/roocs_utils/xarray_utils/xarray_utils.py", line 33, in open_xr_dataset
    return xr.open_dataset(dset[0], use_cftime=True)
IndexError: list index out of range

[ellesmith88]

This runs for me without a problem.

I get IndexError: list index out of range when I try to run tests with non-existent data, as the lines
https://github.com/roocs/roocs-utils/blob/e5f220519578b45b882f87c8348ee0bd8d707f3d/roocs_utils/project_utils.py#L146-L147
don't find any files.

So it seems as though the data is missing somewhere.

[agstephens]

@cehbrecht - it seems we have the files on our file system for this:

DSID=c3s-cmip6.ScenarioMIP.NIMS-KMA.KACE-1-0-G.ssp245.r1i1p1f1.Amon.tas.gr.v20191217

$ ls /badc/cmip6/data/CMIP6/ScenarioMIP/NIMS-KMA/KACE-1-0-G/ssp245/r1i1p1f1/Amon/tas/gr/v20191217/*.nc

/badc/cmip6/data/CMIP6/ScenarioMIP/NIMS-KMA/KACE-1-0-G/ssp245/r1i1p1f1/Amon/tas/gr/v20191217/tas_Amon_KACE-1-0-G_ssp245_r1i1p1f1_gr_201501-210012.nc

Please can you check that you have the same file on the DKRZ file system.

[cehbrecht]

Yes, it works also now on DKRZ side. Both in catalog and file-system.

Maybe it was a temporary failure on the file-system? Or permission changes?

I'm closing this.