Check for missing variables in MPAS time series

[xylar]

If variables are missing in an existing time series data set, delete the data set and start over rather than trying to append.

[xylar]

@milenaveneziani, I discovered this bug in the process of testing #294.

Here's a simple test where you should see that things don't work with develop but are fixed here:

./run_mpas_analysis --purge config.run_name --generate=timeSeriesSST
./run_mpas_analysis config.run_name

With this branch, you should see an mpasTimeSeriesOcean.log that looks something like this:

Computing MPAS time series from first year plus files:
    mpaso.hist.am.timeSeriesStatsMonthly.0001-01-01.nc through
    mpaso.hist.am.timeSeriesStatsMonthly.0005-12-01.nc
running: ncrcat --record_append --no_tmp_fl -v timeMonthly_avg_avgValueWithinOceanRegion_avgSurfaceTemperature,xtime_startMonthly,xtime_endMonthly /media/xylar/extra_data/analysis/edison/G-QU240-master-intel/run/./mpaso.hist.am.timeSeriesStatsMonthly.0001-01-01.nc ... /media/xylar/extra_data/analysis/edison/G-QU240-master-intel/run/./mpaso.hist.am.timeSeriesStatsMonthly.0005-12-01.nc /media/xylar/extra_data/analysis/output/GMPAS-QU240/check_time_series_vars/timeseries/mpasTimeSeriesOcean.nc
Execution time: 0:00:00.10

Computing MPAS time series from first year plus files:
    mpaso.hist.am.timeSeriesStatsMonthly.0001-01-01.nc through
    mpaso.hist.am.timeSeriesStatsMonthly.0005-12-01.nc
Warning: deleting file /media/xylar/extra_data/analysis/output/GMPAS-QU240/check_time_series_vars/timeseries/mpasTimeSeriesOcean.nc because some variables were missing
running: ncrcat --record_append --no_tmp_fl -v timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerSalinity,timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerArea,timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerTemperature,timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerThickness,timeMonthly_avg_avgValueWithinOceanLayerRegion_sumLayerMaskValue,timeMonthly_avg_avgValueWithinOceanRegion_avgSurfaceTemperature,xtime_startMonthly,xtime_endMonthly /media/xylar/extra_data/analysis/edison/G-QU240-master-intel/run/./mpaso.hist.am.timeSeriesStatsMonthly.0001-01-01.nc ... /media/xylar/extra_data/analysis/edison/G-QU240-master-intel/run/./mpaso.hist.am.timeSeriesStatsMonthly.0005-12-01.nc /media/xylar/extra_data/analysis/output/GMPAS-QU240/check_time_series_vars/timeseries/mpasTimeSeriesOcean.nc
Execution time: 0:00:00.18

Note the warning about deleting the output file.

With develop, I see timeSeriesOHC.log as follows:

Plotting OHC time series and T, S, and OHC vertical trends...
  Read in depth and compute specific depth indexes...
  Load ocean data...
analysis task timeSeriesOHC failed during run 
Traceback (most recent call last):
  File "/home/xylar/code/mpas-work/analysis/mpas_analysis_repo/mpas_analysis/shared/analysis_task.py", line 325, in run
    self.run_task()
  File "/home/xylar/code/mpas-work/analysis/mpas_analysis_repo/mpas_analysis/ocean/time_series_ohc.py", line 221, in run_task
    endDate=self.endDate)
  File "/home/xylar/code/mpas-work/analysis/mpas_analysis_repo/mpas_analysis/shared/io/mpas_reader.py", line 85, in open_mpas_dataset
    ds = subset_variables(ds, variableList)
  File "/home/xylar/code/mpas-work/analysis/mpas_analysis_repo/mpas_analysis/shared/io/mpas_reader.py", line 141, in subset_variables
    'variables: {}.'.format(variableList, allvars))
ValueError: Empty dataset is returned.
Variables [u'timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerSalinity', u'timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerArea', u'timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerTemperature', u'timeMonthly_avg_avgValueWithinOceanLayerRegion_avgLayerThickness', u'timeMonthly_avg_avgValueWithinOceanLayerRegion_sumLayerMaskValue']
are not found within the dataset variables: [u'xtime_startMonthly', u'xtime_endMonthly', u'timeMonthly_avg_avgValueWithinOceanRegion_avgSurfaceTemperature'].

Execution time: 0:00:00.05

[milenaveneziani]

@xylar, I can confirm that this branch is behaving as expected. All tests passed as indicated.

[xylar]

Thanks, @milenaveneziani