Switch simulationStartTime to xarray

[xylar]

This should hopefully improve problems with string decoding

closes #704

[xylar]

@darincomeau, I haven't had a chance to fully test this but I think it will fix #704

[xylar]

@darincomeau, could you re-test with this branch? I think it's finally fixed. I'm running the analysis you pointed me to last week right now but it's at least getting through the initialization phase where it was going wrong before.

[darincomeau]

@xylar I'm now getting errors in all of the SOSE/WOCE transect tasks, e.g.:

analysis task soseTransects_plotJFM_lon_75.0_salinity failed during run
Traceback (most recent call last):
  File "/global/homes/d/dcomeau/miniconda3/envs/mpas-analysis/lib/python3.8/site-packages/xarray/backends/file_manager.py", line 198, in _acquire_with_cache_info
    file = self._cache[self._key]
  File "/global/homes/d/dcomeau/miniconda3/envs/mpas-analysis/lib/python3.8/site-packages/xarray/backends/lru_cache.py", line 53, in __getitem__
    value = self._cache[key]
KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('/global/cscratch1/sd/dcomeau/e3sm_scratch/cori-knl/mpas-analysis-output/20200602.A_WCYCL1850-DIB-ISMF.ne30_SOwISC10to60E2r2.cori\
-knl/yrs1-10/clim/obs/remapped/SOSE_transects_5km_lon_75.0_uniform_10_to_1500m_at_10m.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', Fal\
se))]

[xylar]

@darincomeau, I didn't have any trouble with those. I'm running with a pretty new conda environment. Could you rebuild your conda environment (or create a new one) and see if that fixes this issue? I wonder if xarray has maybe been updated to fix this issue?

conda config --add channels conda-forge
conda config --set channel_priority strict
conda create -n mpas-analysis python=3.8 numpy scipy "matplotlib-base>=3.0.2" \
    netCDF4 "xarray>=0.14.1" dask bottleneck lxml "nco>=4.8.1" pyproj \
    pillow cmocean progressbar2 requests setuptools shapely "cartopy>=0.18.0" \
    cartopy_offlinedata geometric_features gsw "pyremap<0.1.0" \
    "mpas_tools>=0.0.8"
conda activate mpas-analysis
python -m mpas_analysis configs/polarRegions.conf config.20200602.A_WCYCL1850-DIB-ISMF.ne30_SOwISC10to60E2r2.cori-knl --generate=soseTransects

[xylar]

@darincomeau, I'm seeing a similar error when I run code that used to work fine (ISMIP6 stuff) in an environment that used to work fine on files that have nothing to do with PIO2 and aren't new. I'm not sure if this could be something specific to Cori, maybe a recent update or something? I'm kind of baffled at the moment.

[xylar]

I see, in my case, it's just a very confusing "file not found" error. Maybe the same for you and it will work if you purge and re-run?

[darincomeau]

Thanks @xylar - I had created a new environment last week before submitting this issue. I also tried clearing out the clim/obs directory to remove the troublesome files and running again (although not --purge to save time), and still got those errors. I'll try purging and running again.

[xylar]

@darincomeau, short of that, you are going to have to debug which file is missing and why. The error message you posted doesn't give the offending file name so I would need that info to help you debug.

[darincomeau]

The tasks that were failing for which I originally opened #704 now complete:

https://portal.nersc.gov/project/m3412/mpas_analysis_output/20200602.A_WCYCL1850-DIB-ISMF.ne30_SOwISC10to60E2r2.cori-knl/yrs1-10/

It seems the transect (and regionalTSDiagrams) errors related to remapped obs files are a separate issue, so it shouldn't hold this up.

Thanks @xylar !