Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed wenz14jgr #1562

Merged
merged 62 commits into from Apr 17, 2020
Merged

Fixed wenz14jgr #1562

merged 62 commits into from Apr 17, 2020

Conversation

zechlau
Copy link
Contributor

@zechlau zechlau commented Mar 3, 2020

This is now the fixed version of the former branch version2_wenz14jgr. It is up to date with the master and all diagnostics work.

@zechlau zechlau self-assigned this Mar 3, 2020
@zechlau zechlau requested a review from bettina-gier Mar 3, 2020
@bettina-gier
Copy link
Contributor

bettina-gier commented Mar 4, 2020

Are you using a specific Core? Using the newest Core master I tried running the namelist in full, but it hangs on the preprocessor step for fgco2:

INFO    [36323] Attempting to load areacello from file: /work/bd0854/DATA/ESMValTool2/CMIP5_ETHZ/esmHistorical/fx/areacello/HadGEM2-ES/r0i0p0/areacello_fx_HadGEM2-ES_esmHistorical_r0i0p0.nc
ERROR   [36323] Failed to run area_statistics(surface_downward_mass_flux_of_carbon_dioxide_expressed_as_carbon / (kg m-2 s-1) (time: 551; latitude: 216; longitude: 360)
     Dimension coordinates:
          time                                                                       x              -               -
          latitude                                                                   -              x               -
          longitude                                                                  -              -               x
     Attributes:
          Conventions: CF-1.4
          associated_files: baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_ocnBgchem_fx_HadGEM2-ES_esmHistorical_r0i0p0.nc...
          branch_time: 0.0
          cmor_version: 2.7.1
          comment: Gas exchange flux of CO2 (positive into ocean)
          contact: chris.d.jones@metoffice.gov.uk, spencer.liddicoat@metoffice.gov.uk
          experiment: ESM historical
          experiment_id: esmHistorical
          forcing: GHG, Oz, SA, LU, Sl, Vl, BC, OC, (GHG = CO2, N2O, CH4, CFCs)
          frequency: mon
          initialization_method: 1
          institute_id: MOHC
          institution: Met Office Hadley Centre, Fitzroy Road, Exeter, Devon, EX1 3PB, UK, (h...
          mo_runid: akylb
          model_id: HadGEM2-ES
          modeling_realm: ocnBgchem
          original_name: mo: m02s30i249 * 3.86124e-10
          parent_experiment: ESM pre-industrial control
          parent_experiment_id: esmControl
          parent_experiment_rip: r1i1p1
          physics_version: 1
          product: output
          project_id: CMIP5
          realization: 1
          references: Collins, W. J., Bellouin, N., Doutriaux-Boucher, M., Gedney, N., Halloran,...
          source: HadGEM2-ES (2009) atmosphere: HadGAM2 (N96L38); ocean: HadGOM2 (lat: 1.0-0.3...
          table_id: Table Omon (26 July 2011) 25bb94a0408beca44c0f5b601258a94e
          title: HadGEM2-ES model output prepared for CMIP5 ESM historical
     Cell methods:
          mean: time
          where sea: area, {'operator': 'sum', 'fx_files': {'areacello': '/work/bd0854/DATA/ESMValTool2/CMIP5_ETHZ/esmHistorical/fx/areacello/HadGEM2-ES/r0i0p0/areacello_fx_HadGEM2-ES_esmHistorical_r0i0p0.nc'}})
ERROR   [36323] Program terminated abnormally, see stack trace below for more information
Traceback (most recent call last):
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_main.py", line 229, in run
    conf = main(args)
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_main.py", line 157, in main
    process_recipe(recipe_file=recipe, config_user=cfg)
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_main.py", line 207, in process_recipe
    recipe.run()
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_recipe.py", line 1312, in run
    max_parallel_tasks=self._cfg['max_parallel_tasks'])
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_task.py", line 606, in run_tasks
    _run_tasks_sequential(tasks)
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_task.py", line 618, in _run_tasks_sequential
    task.run()
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_task.py", line 234, in run
    input_files.extend(task.run())
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/_task.py", line 238, in run
    self.output_files = self._run(input_files)
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/preprocessor/__init__.py", line 404, in _run
    product.apply(step, self.debug)
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/preprocessor/__init__.py", line 272, in apply
    self.cubes = preprocess(self.cubes, step, **self.settings[step])
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/preprocessor/__init__.py", line 217, in preprocess
    result.append(_run_preproc_function(function, item, settings))
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/preprocessor/__init__.py", line 200, in _run_preproc_function
    return function(items, **kwargs)
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/preprocessor/_area.py", line 213, in area_statistics
    grid_areas = tile_grid_areas(cube, fx_files)
  File "/mnt/lustre02/work/bd0854/b309137/v2_okt2019/ESMValCore/esmvalcore/preprocessor/_area.py", line 153, in tile_grid_areas
    grid_areas = da.tile(grid_areas, [cube.shape[0], 1, 1])
  File "/pf/b/b309137/conda-envs/esmvaltool_okt2019/lib/python3.7/site-packages/dask/array/creation.py", line 800, in tile
    raise NotImplementedError("Only integer valued `reps` supported.")
NotImplementedError: Only integer valued `reps` supported.
INFO    [36323] If you suspect this is a bug or need help, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues and attach the run/recipe_*.yml and run/main_log_debug.txt files from the output directory.

There are two things it does produce before that error:

  • Fig 1c (attached): First of all comparing with the paper - Figure 1 there didn't have anomalies, why did you make figure 1 with anomalies in this recipe? Then some smaller things: You have to equalize the diff model start times in the 1pctrun so you don't get the 1-140 and 1850-1999 datasets distinct from each other. Also the y-axis string contains missing values - I assume that's the start and end year judging from the diag log:
warning:Attempt to reference attribute (ref_start) which is undefined

warning:Attempt to reference attribute (ref_end) which is undefined

warning:["CoordArrays.c":591]:No Valid values in Array, unable to compute Min or Max

warning:TransInitialize: Zero Y coordinate span: defaulting:[errno=1104]

Error: gsn_add_text: no valid strings to add, input data is all missing

tas_Global_CMIP5_1pctCO2_anom__1-1999

  • Fig 2f: Same problem with the y-axis, also e and f should be reversed when comparing to paper, there e was esmHistorical, f 1pctCO2, your recipe has them the other way around.

Let me know if there is any core branch to switch to for running it!

zechlau and others added 22 commits Apr 16, 2020
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
Co-Authored-By: Bettina Gier <gier@uni-bremen.de>
@bettina-gier
Copy link
Contributor

bettina-gier commented Apr 16, 2020

Plot constr_tas-nbp_30-1960.000001.png still needs updating as the current one has the gamma_lt vs gamma_lt in the axis description that got fixed!

@axel-lauer
Copy link
Contributor

axel-lauer commented Apr 16, 2020

Figure has been updated, thanks @bettina-gier for spotting this. @zechlau are you finished fixing the typos? Is this now ready for merging?

Copy link
Contributor

@bettina-gier bettina-gier left a comment

Think everything is good now, good job!

@zechlau
Copy link
Contributor Author

zechlau commented Apr 17, 2020

From my side, everything is finished. Thank you very much for reviewing @axel-lauer and @bettina-gier!

@axel-lauer
Copy link
Contributor

axel-lauer commented Apr 17, 2020

Just tested the recipe one more time after all the recent changes. Works fine. This is ready to be merged.

@axel-lauer axel-lauer merged commit ec26b28 into master Apr 17, 2020
2 of 3 checks passed
@axel-lauer axel-lauer deleted the fixed_wenz14jgr branch Apr 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants