Run create_forcing.exe an error comes out.

[Shuaichen-He]

Hello, I followed /NCAR/hrldas/hrldas/README.ERA5 to run Noah-mp and this error occurred while making the forcing data.

[cenlinhe]

Hi, you need to download ERA-5 model terrain data, because the error message said "Source model terrain data not found".

[Shuaichen-He]

• Hello, thank you for your reply～ I have checked the path to my geo data and it is now fixed.

• But I am having another problem when running it, a problem I have had before when using non-global data, but I am using global data this time and it still has this error, if possible I hope you could tell me something about why this error is reported as happening.

[cenlinhe]

@CharlesZheZhang Could you please provide some insights on this user's issue? It seems to be related to the GRIB data I/O handling in the user's computer environment.

[tslin2]

Hi,

Are you using the ifort compiler?!
If so you can try to add the flag in the user_build_options file to debug the code.
What I did was to add -traceback in F90FLAGS

It should generate some extra information to provide source file traceback information when error occurs. Hope this helps!

[Shuaichen-He]

Hi～
Yes I am using the ifort compiler. (If I switch to a GNU compiler will that solve this problem?)
I have added -tracebace to F90FLAGS according to you and now some extra information appears. I am learning to check the create_forcing.f90 file and in the meantime I hope I could ask you for some advice.

[tslin2]

Hi,
I didn't use the GNU compiler, but you can try, I think compiler is not a problem here.
Looking at the traceback information, you may take a look at the line 1806 in the create_forcing.f90 to see if there is any issue there.

[cenlinhe]

The code itself should be OK, since we tested it using intel, gnu, and pgi compilers in NCAR supercomputer. It is probably related to the F90FLAGS setup which may need some minor changes to work with your computer environment and setup. You can try gnu compiler first.

[tslin2]

Hi, I am also trying to generate the ERA5 forcing.

One point I find based on @CharlesZheZhang namelist is that he changed the name "z" in the Vtable of namelist.input file to 'TERRAIN'.

The create_forcing.exe looks to me it requires to read the name "TERRAIN" from the Vtable in the namelist.input and then it reads the geopotential z from the Zfile_template file in the namelist. The create_forcing.exe then convert the geopotential to the height.

Also for both Zfile_template and LANDSfile_template, they require NetCDF files rather than grib files for generating ERA5 forcing.

@CharlesZheZhang Do you agree with me?!

[CharlesZheZhang]

Hi, I am also trying to generate the ERA5 forcing.

One point I find based on @CharlesZheZhang namelist is that he changed the name "z" in the Vtable of namelist.input file to 'TERRAIN'.

The create_forcing.exe looks to me it requires to read the name "TERRAIN" from the Vtable in the namelist.input and then it reads the geopotential z from the Zfile_template file in the namelist. The create_forcing.exe then convert the geopotential to the height.

Also for both Zfile_template and LANDSfile_template, they require NetCDF files rather than grib files for generating ERA5 forcing.

@CharlesZheZhang Do you agree with me?!

Hello Tzu-Shun,

Yes, you are right. The Vtable in the namelist is used for reading grib files. For ERA5 data, it is a bit different that the forcing data are in grib format (Temp, Wind, Rain, Radiaiton, ...), and the Zfile (Z here for geopotential) and Landmask are in netCDF.

This is occurring only in ERA5, but I agree we need to think of a more general way to better accommodate different forcing data from different formats. Let's do this when we are creating the tutorial note for ERA5 forcing soon.

Zhe

[tslin2]

Thank you @CharlesZheZhang

[Shuaichen-He]

• Thanks for all the advice.

• I changed the compiler to gnu and the problem was solved, but I'm back to the same problem I had at the beginning: Undetermined problem opening file: /geo_127914_0.1x0.1.grb. source model terrain data not found. I also tried nc file, but it did not match the Vtable. I finally changed the Vtable as suggested by tslin2 and it works, thank you very much!

• Unfortunately, I ran into a new problem as shown in the picture. The forcing data I used was global (I couldn't make it using GLDAS local area data before, so I purposely downloaded global data as the source for forcing data making), also I tried increasing the expand loop=50, 100, 200 in namelist as prompted, but the result didn't change, I also tried changing the RAINFALL_INTERP=1, it shows: forrtl: error (65): floating invalid.

• I hope you could give me some advice.

[tslin2]

Hi,
It seems like the forcing data from ERA5 does not cover your domain, which region you are running? Are you using MASK_WATER = .FALSE. or .TRUE.? You may try to expand the domain.
You can also share your namelist file for create_forcing and namelist file for the geo_em_file here, so we can take a look.
Thanks

[Shuaichen-He]

• Hi,Thank you for your reply~
• I downloaded global data as the source for forcing data making.
• MASK_WATER = .FALSE. I did try switching it to .TRUE. ,but the result did not change.
• The following forcing.zip content my namelist files for create_forcing and the geo_em_d01.
• Thank you for your help ~
  forcing.zip

[tslin2]

Hi, @Shuaichen-He
Can you try to remove +360. after the geo_em%lon(i,j) in the below line of create_forcing.F and recompile and run it. The longitude of climate data of ERA5 is already between 0 to 360, so there is no need to shift longitude.

hrldas/hrldas/HRLDAS_forcing/create_forcing.F

Line 1262 in 20be331

call latlon_to_ij(datastruct%proj, geo_em%lat(i,j), geo_em%lon(i,j)+360., etax(i,j), etay(i,j))

[Shuaichen-He]

Hi, @tslin2

• According to your suggestion, I modified line 1262 of the create_forcing.F file, and the above problem was solved, but another problem occurred when generating the LDASIN data files.
• My forcing source data is three-hour data (I have tried hour-by-hour data and this problem also exists): For example, my forcing data is from 2020-01-01_00 to 2020-01-09_23, but the LDASIN file produced is from 0:00 to 09:00 on the day of 2020-01-01.
• I think there must be some problem here 🤔️ and would like to seek your advice. Thank you~🌸
  The following file include the create_forcing.exe’s error log and the incomplete LDASIN file screenshot it produces.

LDASIN.zip

[Shuaichen-He]

Hello~

• I'm not sure if it's due to a problem with the data I extracted. the problem always occurs when I run to the fourth input. (I mean: it stops at 9:00 when I use three hours of forcing data, and at three when I use hour-by-hour forcing data).
• The following link includes three files: the extract files for three days from 202001-20200103, namelist.wps, namelist.input. Could you please help to see if there is any problem.

git clone https://github.com/Shuaichen-He/Noah_mp_test.git

Thanks a lot~

[Shuaichen-He]

Hi，
There is an another attempt.

• The three-hour data run reported an error at the fourth file (i.e., at 09), so I regenerated INIT file at the third file(i.e., at 06) to produce the forcing data. Again, it stopped at the fourth file generated. I think this rules out the possibility of extracting the data incorrectly.
• Thank you ~

[cenlinhe]

Do you mean running ./hrldas.exe instead of running ./create_forcing.exe? If so, what is your namelist.hrldas option?

[tslin2]

Hi, @Shuaichen-He
Can you try increasing EXPAND_LOOP in the namelist, e.g., 11 or 15
I think this should work!

[Shuaichen-He]

Hi, @tslin2 @cenlinhe ,
Thank you very much for your continuous help, now I can run the create_forcing.exe output LDSIN file properly.🌸

• However, just when I further try to run hrldas.exe, the program shows: forrtl: error (75): floating point exception.

• I am not familiar with the SurfaceEnergyFluxVegetatedMod.f90 module and I would like to seek your help.

• Here is my namelist file and the error file.run_hrldas.exe.zip

• Thank you ~

[cenlinhe]

1. What compiler did you use?
2. It seems that most of your points are sea ice points?
3. Could you please try re-downloading the GitHub code because we recently fixed a few bugs which might be the cause of your problem here?
4. The error seems to be that one of the terms becomes NaN in the canopy temperature calculations.

[tslin2]

You may also like to double check the values of climate forcing that generated.
Even though I increase EXPAND_LOOP for ./create_forcing.exe, the climate I generate from your ERA5 files did not seem right, e.g., 2020010101.LDASIN_DOMAIN1 has negative longwave radiation and after that it is too low for every timestep.
If you use 3 hourly average data from ERA5 that may not work, suggesting download hourly ERA5 data. @CharlesZheZhang may have more insights on this.

[Shuaichen-He]

Hi,

• Thanks for your reply~

• @cenlinhe I am using GNU-gfortran compiler. There is indeed a small ocean in the simulation area I selected, is this the reason for the error reported ?(I have tried other driver data before that also had this small ocean and was able to run it) I have tried re-downloading the Github code, and the seem error coming out when I am running hrlds.exe. (In the following zip named hrlds.out)

• @tslin2 One more thing is that this attempt is different from the previous :
  Based on tslin2 findings, I checked and modified my extracted code. Because in the ERA5 data, some variables are calculated cumulatively, such as radiation and precipitation. For these variables, I previously thought that the data of 2020010100 was the first data accumulated from 20200101, but by double checking, I found that it was the accumulated data from the previous full day. So I re-produced the extract data and made the LDASIN file, which also contains the 3 days of data in the zip file below. This time it started from 20200101, but the similar problem still exists when running hrldas.exe.

• I hope you could give me some advice.🥺

• Thank you ~🌸
  run_hrldas.exe_2.zip

[tslin2]

Hi, @Shuaichen-He
you're welcome!
please try changing the soil layer thickness in the namelist, as ERA5 has different soil layer depth
soil_thick_input(1) = 0.07
soil_thick_input(2) = 0.21
soil_thick_input(3) = 0.72
soil_thick_input(4) = 1.55

[CharlesZheZhang]

Hi @Shuaichen-He.
For the first question related to hourly or 3-hourly data, did you use the python code under ERA5 folder to generate hourly forcing data?

1. First run extract_era5.py, you will see three accumulated term, ACSSRD, ACSTRD, ACTP.
2. Next, for these three terms, you will need to run deaccumulate_era5.py. In this code, I keep the 00 hour the same, and use the next hour minus the previous hour to "deaccumulate" the radiation and precipitation term. You will then see, SSTD, STRD, TP three terms.
3. Run create_forcing.exe, this program will generate hourly forcing file LDASIN and convert unit from J/m2 to W/m2.

I tried running the date 2020-01-01 with your namelist on Cheyenne computer for hourly forcing and the results are good. For reference, please see the upload files.
LDASIN2020.zip

The second question, as in @tslin2 reply, the error forrtl: error (75): floating point exception. is related to reading the soil temperature and soil moisture from HRLDAS setup file, which ERA5 has different soil layer than default Noah-MP. By changing the soil layer depth as suggested, it should be able to run successfully. Thanks Tzu-Shun for helping with the solution.

[Shuaichen-He]

It is work！ thanks a lot ~ @tslin2 🌸

• @CharlesZheZhang
• I made some changes by referring to the code in the ERA5 folder. Because in the beginning, I had comprehensive nc data and wanted to convert them to grib format as input via CDO. But then I found that it can not produce the forcing data properly, so I re-downloaded the grib data from ERA5-land for processing.
• I carefully compared my generated LDAIN file with yours, and quantitatively speaking, your file is correct. I checked the data I used to extract the variables (the data downloaded from the ERA5-land website was a file that I sliced into hour-by-hour data before using it for extraction, which is probably the key to the problem), and as shown below, 00 differs from 01 in magnitude, and I naturally assumed that 00 was the cumulative value of the previous full day, which is contrary to the previous assumption that 00 was the beginning, so I combined tslin2's findings modified the deccumulate code.

- By comparing the LDSIN file you provided I think the problem is at the beginning of the cut, I'm still checking what exactly is going on, thank you very much for your help.

• In the meantime, here is the python code I used to extract the variables. I used CDO throughout because I didn't quite understand the decomond part of wgrib's syntax, but I don't think that's the issue at the moment.

• ERA5.zip

• If you have any suggestions I am more than happy to accept them.

• Again, sincere thanks to all ~ 🌸

[cenlinhe]

This bug in ERA5 preprocessing part (HRLDAS_forcing) has been fixed in this PR: #81

[Shuaichen-He]

Hi,@CharlesZheZhang @tslin2

• I have extracted 30 days of data using the code under ERA5 folder and made LDASIN file, but such an error occurs when I run to 22:00 on the first day.I have tried to change another dynamic vegetation schemes but no change. Could you please help me ？😢

• The exact information is placed in noah.out in the file below, along with my LDASIN file and namelist.
  noah_mp.zip

[cenlinhe]

This error is caused by the negative downward longwave radiation. Please double check your forcing file for the LWDOWN variable to make sure it is positive.

[Shuaichen-He]

Hello, @cenlinhe ,Thank you for your reply~

• I also noticed that the LADSIN file has a negative value of 23:00.

• This time I used deaccumulate.py from hrldas/run/example/ERA5 file to make the data needed to drive the holds-forcing, and I found that this only happens when calculating 22:00 to 23:00. After some trial and error, I think it is the ERA5-land data structure that causes some problems with CDO, and I am trying to use ERA5-single levels data to make forcing data.

• When I tried with ERA5-single-level data, the Geopotential and landsemask files did not match, and I had a similar problem to the one at the beginning of this issue after downloading the two single-level files, as shown in the picture. I have tried changing VarName to TERRAIN or z and this is also happens, I don't know where I can find the real name of this Geopotential name. Hello,@tslin2 ~ Would you be willing to help me with this problem?

• Would you consider using single- level data as an example in the future if it is possible, I think it is very smooth and fast in extracting the process of extracting forcing data than ERA5-land, or it is possible to make more general way to better accommodate different forcing data from different formats.
• Related Documents：
  namelists.zip
  extract.zip

[tslin2]

Hi, @Shuaichen-He
Thanks for the feedback. We should emphasize in the doc for ERA5 is based on ERA5-LAND data

What do you mean using ERA5-single-level data?
Do you mean the https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview

The python script was used for the ERA5-LAND, so you had to change it for different version of ERA5 data.
https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-land?tab=form

Are you using the same Geopotential and landsemask files for ERA5-single-level data?

Can you also try to use wgrib instead of cdo for ERA5-LAND in extract_era5.py, we have used wgrib and worked fine.
you can download from here
https://www.cpc.ncep.noaa.gov/products/wesley/wgrib.html

Thanks

[tslin2]

Hi, @Shuaichen-He

You use the grib file, you should try to use the nc files for Geopotential and landseamask files. Right now, the code only read nc files for these two variables. Also, resolution of ERA5-LAND and single-level ERA5 is different

[Shuaichen-He]

Hi, @cenlinhe ,thank you for your reply ~

• I am using Wgrib to extract the variables as in the example you gave in hrldas/HRLDASFORCING/run/example/ERA5. Everything works fine except for SSRD, STRD, and TP which have problems extracting the data CDO at 23:00.

• There was no 23:00 document in this previous response:

• Run create_forcing.exe an error comes out. #56 (comment)

• I used the code you provided for extracting the forcing data and did not change anything, but I still got negative values for 23:00, which has not happened with other datasets, so I think there is something wrong with ERA5_land that I do not understand.

• Thanks～

[tslin2]

Hi, @Shuaichen-He
I created the same forcing at your domain the same time period, 2020-01-01, and everything looked fine, there is no negative values for longwave radiation.
Maybe you can share the python scripts so I can take a look.

[Shuaichen-He]

Hi, @Shuaichen-He
I created the same forcing at your domain the same time period, 2020-01-01, and everything looked fine, there is no negative values for longwave radiation.
Maybe you can share the python scripts so I can take a look.

OK! Thanks a lot! 🌸

• The following are my python scripts I used.

python_scripts.zip

• The CDO version I used is 2.2.0 , and this is two files I extracted, you could also try to use 'cod sub' code to subtract it.

cdo.zip

[tslin2]

I think there are some minor changes you made may cause some issues. e.g., inday
I have attached the files that worked successfully, so you can compare and use it. Please change the path and filename in the python script
era_script.zip

[Shuaichen-He]

• I may have made changes when trying to understand wgrib usage before, thanks for telling me about the error.
  （Also the data download of ERA5_land will put down an extra day of null values, which will indeed be problematic if extracted using CDO.）

• 🙏I have learned a lot during this time of trying and under your guidance. @tslin2 🌸

• Thanks for all the help you have given me.

[tslin2]

You're welcome. Thanks for your feedback. I have also learned a lot.