cmorizing lpjg output

[klauswyser]

Where do we stand with respect to LPJG output? There is a flag --land in ece2cmor.py, does it work and process the LPJG output? Has anybody a worksheet (.xls file) or varlist with land variables for a historical or piControl experiment?

[goord]

Hi Klaus, certainly @nierad and others have managed to cmorize lpjg output with this. I believe the flag you are looking for is --lpjg.

[treerink]

Hi the flag for only LPJ-GUESS cmorization is --lpj. You can just use the data request xlsx file for the MIP experiment you are interested in. Many of the identified LPJ-GUESS variables will be cmorized then.

However, some dropped out, this is related to the fact that at the LPJ-GUESS ece2cmor component merge the generate-lpjguess.json.sh script has been changed with a slightly different approach resulting in a different lpjgpar.json where now several of the in fact available LPJ-GUESS variables are missing. I still have to add an issue on this.

[klauswyser]

Thanks for the tips. I tried with a data request created for CMIP/piControl, seems to work. But then ece2cmor.py stops with

INFO:ece2cmor3.ece2cmorlib:Selected 48 LPJG tasks from 48 input tasks
INFO:ece2cmor3.lpjg2cmor:Executing 48 lpjg tasks...
INFO:ece2cmor3.lpjg2cmor:Cmorizing lpjg tasks...
INFO:ece2cmor3.lpjg2cmor:Creating lpjg netcdf file for variable cSoil for year 1980
Error in calling operator remapycon with:
>>> cdo -O remapycon,n128 -setgrid,ece2cmor3/resources/ingrid_T255_unstructured.txt /scratch/local/LPJGtemp.nc /scratch/local/intermediate.nc<<<
STDOUT:

STDERR:cdo remapycon: Started child process "setgrid,ece2cmor3/resources/ingrid_T255_unstructured.txt /scratch/local/LPJGtemp.nc (pipe1.1)".
cdo(2) setgrid (Abort): Open failed on ece2cmor3/resources/ingrid_T255_unstructured.txt!

The resources/ingrid_T255_unstructured.txt file is there, but apparently I am 1 directory too low, The hard-wired path only works if I start ece2cmor.py from directory ~/cmorize/ece2cmor3. Not nice, can't the resources directory be set when launching ece2cmor.py?

[klauswyser]

Next problem:

INFO:ece2cmor3.lpjg2cmor:Creating lpjg netcdf file for variable mrsol for year 1980
Traceback (most recent call last):
  File "./ece2cmor3/ece2cmor.py", line 114, in <module>
    main()
  File "./ece2cmor3/ece2cmor.py", line 106, in main
    ece2cmorlib.perform_lpjg_tasks(args.datadir, args.tmpdir, args.exp, startdate, length)
  File "/home/sm_wyser/.conda/envs/ece2cmor3/lib/python2.7/site-packages/ece2cmor3-1.0.0-py2.7.egg/ece2cmor3/ece2cmorlib.py", line 187, in perform_lpjg_tasks
    lpjg2cmor.execute(lpjg_tasks)
  File "/home/sm_wyser/.conda/envs/ece2cmor3/lib/python2.7/site-packages/ece2cmor3-1.0.0-py2.7.egg/ece2cmor3/lpjg2cmor.py", line 212, in execute
    create_sdepth_axis(task, lpjgfile, freq)
  File "/home/sm_wyser/.conda/envs/ece2cmor3/lib/python2.7/site-packages/ece2cmor3-1.0.0-py2.7.egg/ece2cmor3/lpjg2cmor.py", line 578, in create_sdepth_axis
    sdepthvals = np.array([float(d) for d in depths])
ValueError: could not convert string to float: layer_1

mrsol is soil moisture, right? There were some discussions whether or not it makes sense that soil moisture is saved from LPJG and not from IFS. Does this error has something to do with it?

[goord]

I think we should be able to cmorize soil moisture from lpjg output, so I believe the problem above is caused by something else. Will have a look at it.

[goord]

Apparently lpjg2cmor attempts to read the soil depths from an output file header, but in the header it finds layer_1, ... instead of numbers. I fixed the hard path issue and added logging regarding this soil depth reading, can you pull, re-install and rerun Klaus?

[klauswyser]

Pulled, re-installed and re-run, same result...

[goord]

Yes I expected that but now it should report which file it tries to read to determine the soil layer depths no?

[etiennesky]

Hi, probably @nierad could comment better than me, but soil moisture in IFS and LPJG can be different, since LPJG computes soil moisture internally, even though IFS sends it soil moisture to LPJG. Not sure which one to use for cmorization in this case?

[goord]

The general philosophy of ece2cmor3 is to compute whatever it can, so in this case it will extract mrsol from the lpjg output. This may not be the optimal strategy for all use cases, which I address in issue #224

[klauswyser]

Yes I expected that but now it should report which file it tries to read to determine the soil layer depths no?

Indeed, it does:

INFO:ece2cmor3.lpjg2cmor:Creating depth axis using file /nobackup/rossby20/rossby/joint_exp/ecearth/sm_wyser/ece-run/tbj6/output/lpjg/001/CMIP6/mrsol.out...

[goord]

Can you see in the header of that output file whether the soil depths are listed?

[klauswyser]

The file starts with

          Lon          Lat  Year  Mth  layer_1  layer_2
   -40.000000    83.156990  1980    1  87.8268 185.0000
   -40.000000    83.156990  1980    2  87.7577 185.0000

So yes, it lists 2 layers.

[goord]

...but no soil depths. I am not at all familiar with the lpjguess output files so I don't know what is supposed to happen there...

@nierad or @janitm do you have any ideas?

[janitm]

Ece2cmor3 does indeed try to read the soil layer depths from the column headers of the LPJ-Guess .out-files. However, LPJ-Guess has not traditionally written the soil depths to the output, instead using the layer_1-style header names.

The cmor soil depth axis ticks are "depth in meters", not e.g. a string stating "layer 1" or similar, so the design decision was made a while back to change the LPJ-Guess output so that header names = soil depths. This is so that ece2cmor3 will have all the necessary information already from the output file, which will avoid problems if/when the depths corresponding to the LPJ-Guess soil layers change in the future. The guys at Lund University, mainly David Wårlind and @nierad, will know if the change has already been made in the most recent versions of LPJ-Guess.

This will of course not help with cmorizing any existing LPJ-Guess output files. I suppose the easiest way to cmorize those is to simply edit the header and replace layer_1 and layer_2 with their corresponding values for soil depth.

[nierad]

Hi all,
sorry for the delay I was off-grid for a couple of days. We have discovered a few bugs so far and we are still looking into it. We're currently at the CRESCENDO meeting but will try to solve it as quick as possible. We'll have "layer_1, ..." replaced by the actual max_depth of a layer in the lpjg output as discussed earlier.

[nierad]

Just an update:
We have fixed the outputs and the missing/wrong variables in the lpjgpar.json-file. Currently, we're running a short run, will then test the cmorization again and will then update the code (hopefully later this week).

[zklaus]

@nierad, any news on this?

[nierad]

It is resolved within the new lpjg-branch.