Improve file reading - planning

[clairemerker]

• retrieve just one variable (cfgrib.open_dataset() just supports all or one)
• keep cfgrib.open_datasets() as our standard for single files
• check difference between psy.open_dataset() and xarray.open_dataset() for netcdf (keyword args), and change to array.open_dataset()
• check difference between cfgrib.open_datasets() and xarray.open_dataset()(keyword args), and maybe add kwargs to iconarry.open_dataset(), otherwise thik about renaming.
• parallelise reading of multiple files with dask as new function e.g. open_multiple_files()
  • the user needs to specify filter_by_keys arguments if the file contains variables with same shortName but different coordinates
  • use xarray.open_mfdataset() / dask

Further ideas

• time cfgrib.open_datasets() against cfgrib.open_dataset()
• look at finch to test and time things

[victoria-cherkas]

ds1 = psy.open_dataset(nc)
ds2 = xr.open_dataset(nc)

ds1.equals(ds2) 
> False 

Psyplot open_dataset brings bounds variables of the dataset to coordinates, eg clon_bnds, clat_bnds, height_2_bnds, etc.. with xr.open_dataset they remain as data variables.
https://github.com/psyplot/psyplot/blob/131e7dc45a6cfd2a963080ec83fa343da362b89d/psyplot/data.py#L678

Can achieve this bounds in coordinates wth decode_coords='all' eg:

ds1= xr.open_dataset(nc, decode_cf=True,
                         decode_coords='all', 
                         decode_times=True)

Other decoding from psyplot, but more related to plotting.

[clairemerker]

Hm... I think, strategically I would like to avoid psyplot as a dependency from iconarray (because it is quite a large package)... What do you think?

[victoria-cherkas]

Agree, it would be good to separate, and feasible (I believe!)

Just wanted to document here in case I forget about this.

[victoria-cherkas]

Merged #55 :)