PMP mean climate regional application

[lee1043]

No description provided.

[lee1043]

@acordonez when you have time, could you check if this has any impact on the mean climate demo?

[acordonez]

@lee1043 Yes I can run the demo this week.

[acordonez]

@lee1043 I've gotten this error running the basic case from the Jupyter notebook demo:

(Edit: I did not rerun the climatologies demo first)

case_id:  basicTest 
 test_data_set: ['ACCESS1-0', 'CanCM4'] 
 realization:  
 vars: ['rlut'] 
 varname_in_test_data: None 
 reference_data_set: ['all'] 
 target_grid: 2.5x2.5 
 regrid_tool: regrid2 
 regrid_tool_ocn: esmf 
 save_test_clims: False 
 test_clims_interpolated_output: None 
 filename_template: cmip5.historical.%(model_version).r1i1p1.mon.%(variable).198101-200512.AC.v20200426.nc 
 sftlf_filename_template: sftlf_%(model_version).nc 
 generate_sftlf: True 
 regions_specs: {'global': {}, 'NHEX': {'domain': {'latitude': (30.0, 90)}}, 'SHEX': {'domain': {'latitude': (-90.0, -30)}}, 'TROPICS': {'domain': {'latitude': (-30.0, 30)}}, '90S50S': {'domain': {'latitude': (-90.0, -50)}}, '50S20S': {'domain': {'latitude': (-50.0, -20)}}, '20S20N': {'domain': {'latitude': (-20.0, 20)}}, '20N50N': {'domain': {'latitude': (20.0, 50)}}, '50N90N': {'domain': {'latitude': (50.0, 90)}}, 'CONUS': {'domain': {'latitude': (24.7, 49.4), 'longitude': (-124.78, -66.92)}}, 'land': {'value': 100}, 'land_NHEX': {'value': 100, 'domain': {'latitude': (30.0, 90)}}, 'land_SHEX': {'value': 100, 'domain': {'latitude': (-90.0, -30)}}, 'land_TROPICS': {'value': 100, 'domain': {'latitude': (-30.0, 30)}}, 'land_CONUS': {'value': 100, 'domain': {'latitude': (24.7, 49.4), 'longitude': (-124.78, -66.92)}}, 'ocean': {'value': 0}, 'ocean_NHEX': {'value': 0, 'domain': {'latitude': (30.0, 90)}}, 'ocean_SHEX': {'value': 0, 'domain': {'latitude': (-90.0, -30)}}, 'ocean_TROPICS': {'value': 0, 'domain': {'latitude': (30.0, 30)}}, 'ocean_50S50N': {'value': 0.0, 'domain': {'latitude': (-50.0, 50)}}, 'ocean_50S20S': {'value': 0.0, 'domain': {'latitude': (-50.0, -20)}}, 'ocean_20S20N': {'value': 0.0, 'domain': {'latitude': (-20.0, 20)}}, 'ocean_20N50N': {'value': 0.0, 'domain': {'latitude': (20.0, 50)}}, 'NAM': {'domain': {'latitude': (20.0, 90), 'longitude': (-180, 180)}}, 'NAO': {'domain': {'latitude': (20.0, 80), 'longitude': (-90, 40)}}, 'SAM': {'domain': {'latitude': (-20.0, -90), 'longitude': (0, 360)}}, 'PNA': {'domain': {'latitude': (20.0, 85), 'longitude': (120, 240)}}, 'PDO': {'domain': {'latitude': (20.0, 70), 'longitude': (110, 260)}}, 'AllMW': {'domain': {'latitude': (-40.0, 45.0), 'longitude': (0.0, 360.0)}}, 'AllM': {'domain': {'latitude': (-45.0, 45.0), 'longitude': (0.0, 360.0)}}, 'NAMM': {'domain': {'latitude': (0.0, 45.0), 'longitude': (210.0, 310.0)}}, 'SAMM': {'domain': {'latitude': (-45.0, 0.0), 'longitude': (240.0, 330.0)}}, 'NAFM': {'domain': {'latitude': (0.0, 45.0), 'longitude': (310.0, 60.0)}}, 'SAFM': {'domain': {'latitude': (-45.0, 0.0), 'longitude': (0.0, 90.0)}}, 'ASM': {'domain': {'latitude': (0.0, 45.0), 'longitude': (60.0, 180.0)}}, 'AUSM': {'domain': {'latitude': (-45.0, 0.0), 'longitude': (90.0, 160.0)}}, 'AIR': {'domain': {'latitude': (7.0, 25.0), 'longitude': (65.0, 85.0)}}, 'AUS': {'domain': {'latitude': (-20.0, -10.0), 'longitude': (120.0, 150.0)}}, 'Sahel': {'domain': {'latitude': (13.0, 18.0), 'longitude': (-10.0, 10.0)}}, 'GoG': {'domain': {'latitude': (0.0, 5.0), 'longitude': (-10.0, 10.0)}}, 'NAmo': {'domain': {'latitude': (20.0, 37.0), 'longitude': (-112.0, -103.0)}}, 'SAmo': {'domain': {'latitude': (-20.0, 2.5), 'longitude': (-65.0, -40.0)}}} 
 regions: {'rlut': ['Global']} 
 test_data_path: demo_data/CMIP5_demo_clims/ 
 reference_data_path: demo_data/obs4MIPs_PCMDI_clims 
 metrics_output_path: demo_output/basicTest 
 diagnostics_output_path: demo_output/basicTest 
 debug: False 

--- prepare mean climate metrics calculation ---
--- start mean climate metrics calculation ---
varname: rlut
level: None
reference_data_set (all):  ['alternate1', 'default']
ref: alternate1
ref_data_full_path: demo_data/obs4MIPs_PCMDI_clims/rlut/CERES-EBAF-4-0/v20210804/rlut_mon_CERES-EBAF-4-0_PCMDI_gn.200301-201812.AC.v20210804.nc

Traceback (most recent call last):
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/bin/mean_climate_driver.py", line 253, in <module>
    ds_ref = load_and_regrid(
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/pcmdi_metrics/mean_climate/lib/load_and_regrid.py", line 36, in load_and_regrid
    ds = xcdat_open(
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/pcmdi_metrics/io/xcdat_openxml.py", line 30, in xcdat_open
    ds = xcdat.open_mfdataset(
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xcdat/dataset.py", line 216, in open_mfdataset
    ds = xr.open_mfdataset(
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/backends/api.py", line 1041, in open_mfdataset
    datasets = [preprocess(ds) for ds in datasets]
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/backends/api.py", line 1041, in <listcomp>
    datasets = [preprocess(ds) for ds in datasets]
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xcdat/dataset.py", line 501, in _preprocess
    ds_new = decode_time(ds_new)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xcdat/dataset.py", line 347, in decode_time
    ds = ds.assign_coords({coords.name: decoded_time})
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/common.py", line 615, in assign_coords
    data.coords.update(results)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/coordinates.py", line 169, in update
    coords, indexes = merge_coords(
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/merge.py", line 553, in merge_coords
    collected = collect_variables_and_indexes(aligned, indexes=indexes)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/merge.py", line 354, in collect_variables_and_indexes
    variable = as_variable(variable, name=name)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/variable.py", line 169, in as_variable
    obj = obj.to_index_variable()
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/variable.py", line 630, in to_index_variable
    return IndexVariable(
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/variable.py", line 2910, in __init__
    self._data = PandasIndexingAdapter(self._data)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/indexing.py", line 1478, in __init__
    self.array = safe_cast_to_index(array)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/indexes.py", line 190, in safe_cast_to_index
    index = pd.Index(np.asarray(array), **kwargs)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/core/indexing.py", line 471, in __array__
    return np.asarray(self.get_duck_array(), dtype=dtype)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/coding/variables.py", line 74, in get_duck_array
    return self.func(self.array.get_duck_array())
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xcdat/dataset.py", line 686, in _get_cftime_coords
    return decode_cf_datetime(offsets, units, calendar=calendar, use_cftime=True)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/coding/times.py", line 312, in decode_cf_datetime
    dates = _decode_datetime_with_cftime(flat_num_dates, units, calendar)
  File "/Users/ordonez4/miniconda3/envs/pmp_test_311/lib/python3.9/site-packages/xarray/coding/times.py", line 210, in _decode_datetime_with_cftime
    cftime.num2date(num_dates, units, calendar, only_use_cftime_datetimes=True)
  File "src/cftime/_cftime.pyx", line 580, in cftime._cftime.num2date
  File "src/cftime/_cftime.pyx", line 110, in cftime._cftime._dateparse
  File "src/cftime/_cftime.pyx", line 781, in cftime._cftime._parse_date
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

[lee1043]

@acordonez thank you for the checking. I revised code to prevent this error, could you please re-try? It is fine to not rerun the climatologies demo first.

[acordonez]

@lee1043 I'm still getting the same error. My process was pulling your changes, doing a "pip install ." in my environment, and launching a new Jupyter notebook session using the kernel for the same environment. Is it possible there's something wrong with my input data?

[lee1043]

@acordonez thank you for re-checking it. Thanks to that I was able to narrow down where the error was coming from, and fixed it. I also updated demo 1b notebook slightly because now case_id is included as a part of json file name. When you get a chance, could you please run the demo? I see the demo runs well on my end.

[acordonez]

@lee1043 Both mean climate demos are working for me now.

[lee1043]

@acordonez thank you for checking. I merged the PR.