8 day forecast starting 2019-08-29 #86
Comments
|
First congratulations in getting that done. If rocky can help move to Hera we will see how best to visualize it then share that script. Hopefully that helps us set the baseline. How long did it take you to run it ?
…Sent from my iPhone
On Feb 12, 2020, at 7:06 PM, jedwards4b ***@***.***> wrote:
Here is the complete output: /glade/scratch/jedwards/ufstest/run I don't have enough experience with this model to know if it's any good or not - @arunchawla-NOAA who should look at it?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
It only took about 30 minutes. @rsdunlapiv can you copy this directory to hera? |
|
@jedwards4b given the NSST setting issue, maybe we need to rerun this case. But first lets ensure that the regression tests pass |
|
I started the copy to Hera: |
|
I'm trying to run the dorian test case on cheyenne with the gnu compliler. and I'm running into issues. When I changed the path in src/model/FV3/cime/cime_config/buildnml I now get the following error when running case.submit (or ./check_input_data): Loading input file list: 'Buildconf/ufsatm.input_data_list' |
|
@arunchawla-NOAA the transfer has completed: |
|
@pjpegion this is already fixed. there was typo problem in the file. Could you update the CIME to the latest of remotes/origin/ufs_release_v1.0 branch. This will fix the problem. |
|
@pjpegion we are in the process of debugging this case now, we are not ready for you to run it unless you want to help figure out the problem. Thanks, |
|
Continued from email discussion. And the model is blowing up |
|
I tried nstf_name = 0,1,1,0,5 - this also crashes in the same way. |
|
@arunchawla-NOAA Can we get the post chgres_cube files for this case and try running from that so that we can determine if the problem is there? |
|
@BinLiu-NOAA What do you suggest next. |
|
@jedwards4b I am not sure why in your test, nstf_name = 0,1,1,0,5 or nstf_name = 0,0,0,0,0 could not work. But, if nstf = 2,1,1,0,5 worked, you probably want to double check if the NSST related output fields are correct. I would suggest you checking with @GeorgeGayno-NOAA and xu.li@noaa.gov at EMC to see if chgres_cube and the NSST component of ufs-weather-model are working properly. Also, to be related, did you also ran a test by using NEMSIO format GFS file? If so, that simulation can serve as a kind of control experiment for the one initialized by using grib2 format GFS files. Bin |
|
@BinLiu-NOAA We used grib2 input in this case. We don't have NEMSIO files for same date (dorian case) because the NOMADS server has only last 10 days. If you know the public place that we could get NEMSIO files and use them to run the model, we could try. If you or @GeorgeGayno-NOAA have recent run (dorian case) that uses chores, grib2 combination and works with this options, please share the namelist files and also input files, then we could make some comparison. Otherwise, it could be hard the find the source of the problem for us without deep experience with NSST, model and chgres. |
|
@jedwards4b For your reference, I posted below for the namelist files for my chgres_cube and forecast job for my C96 grib2 test. They are on Hera, together with the chgres_cube and forecast results, if you happen to have access there. Meanwhile, just for clarification, I was using early versions of UFS_UTILS's chgres and the ufs-weather-model (more specifically the HAFS application). But, I don't think it makes much difference here. Bin
&atmos_model_nml &diag_manager_nml &fms_io_nml &fms_nml &fv_grid_nml &fv_core_nml &surf_map_nml &external_ic_nml &gfs_physics_nml &gfdl_cloud_microphysics_nml &interpolator_nml &namsfc |
|
@BinLiu-NOAA Thanks for the namelist files. I'll check them. In the meantime, i think it is better to test with the version that we are using because there could be an issue related with CHGRES or the model stability.
It could be easy test to use CHGRES from 1.0.0alpha01 and try to generate input files and run the model. |
|
BTW, your CHGRES namelist has lots of options and i think most of them are in default value. Right? Anyway, we could try to use your namelist to see what happens. |
|
Is this process grib2? How we could access the input data? Is it in a public place? You are also setting tracers and tracers_input and I think those are not required for grib2 input. |
|
@rsdunlapiv Thanks, it works now |
|
@jedwards4b my run was with the gfs analysis files not the new "canned winds" file that Kate Friedman supplied. Do you still have the results of your run that started this thread? |
@uturuncoglu, please use your own versions of namelist files. I posted mine just for your reference (because you asked). Again, as I mentioned, my tests were based on early versions of UFS_UTILS's chgres_cube and ufs-weather-model. |
|
@rsdunlapiv Can you tar the directory /scratch1/NCEPDEV/hwrf/scrub/Bin.Liu/ufs_utils_grib2/HAFS_uniform_forecast_g2new_C96_2019082900_05L/ on hera and transfer to cheyenne: To confirm please make sure the file gfs.t00z.pgrb2.1p00.f000 is there. We also need Thanks |
|
@BinLiu-NOAA JFYI, i compared your input.nml with CIME generated one and I found following differences, In this case the values indicated by () are the CIME ones. So, we have lots of differences. We were using following documents to set defaults for mr-wearther model v16beta - https://docs.google.com/document/d/1bLbVdWgEIknDQZgTuOZ6IPVEGv5jUgOrCm4GrR96oBU/edit The above list of options even do not exist in the google docs and we are using defaults from the source code. This indicates that there is no common namelist file in EMC side. This makes hard to find the source of the problem. I think we need to use same
Besides these, we also tested those options and the model still failing in the same place. We are still investigating the source of the problem. |
|
@BinLiu-NOAA is this a configuration that is only expected to work for the HAFS (hurricane) configuration? The focus of the release is global medium-range weather, so if HAFS has a lot of application specific changes, it may not be suitable for the release at this time. |
|
@junwang-noaa and @KateFriedman-NOAA can you help here. The namelists that @jedwards4b has based on the documentation provided is different from the one that @BinLiu-NOAA is running. Can we identifty what those differences signify? We want to understand why the runs are blowing up |
|
The problem seems to be strongly linked to turning OFF NSST. |
|
@jedwards4b just for clarification, do the runs work when you use the namelists from @BinLiu-NOAA ? |
|
The namelists from @BinLiu-NOAA are from a different version of both the model and the chgres there appear to be namelist variables defined that we don't have and so we can't just copy the namelist. We tried to pick out all the differences that we could find and run with that - the model died in exactly the same way. |
|
OK we will get together tomorrow at EMC and get back to you |
|
This file gfs.t00z.pgrb2.1p00.f000 is the input used by the @BinLiu-NOAA case. It doesn't have the same name as the file on the ftp site gfs_4_20190829_0000_000.grb2. We haven't been able to find the file on hera to confirm whether it is or is not the same file. |
|
So here is the summary of a long discussion between EMC, NCAR and DTC
Bottom line is CIME has the right flags we just need to decide using 2,0,0,0,0 for NEMSIO (Jim will do a test to see how this works) All other differences in namelists were a red herring as we were testing with different codes and should be ignored We still have a decision to make for NSST with grib2 data and there is a related ticket for that |
|
I have created PRs NOAA-EMC/fv3atm#67 |
|
@jedwards4b Why do we need PRs NOAA-EMC/fv3atm#67? I thought we are resolving the issue using method 1) in Arun's email. |
|
I offer PR #67 as an alternative or perhaps additional choice. I think
that we'll make a decision based on the output of the two runs.
…On Sat, Feb 15, 2020 at 8:59 PM junwang-noaa ***@***.***> wrote:
@jedwards4b <https://github.com/jedwards4b> Why do we need PRs
NOAA-EMC/fv3atm#67 <NOAA-EMC/fv3atm#67>? I
thought we are resolving the issue using method 1) in Arun's email.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=ABOXUGGYXC3IYS4GMRLUEKDRDC23PA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL35BZI#issuecomment-586666213>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOXUGGUY73G5QZGKWGR72LRDC23PANCNFSM4KUG5EQQ>
.
--
Jim Edwards
CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO
|
|
@arunchawla-NOAA You were going to provide an ncl script to verify the dorian test case. |
|
we have sample ncl scripts for review. We want to check in other systems |
|
@jedwards4b Which directories/files do you need to get from https://ftp.emc.ncep.noaa.gov/EIB/UFS/ |
|
@ceceliadid just this one: https://ftp.emc.ncep.noaa.gov/EIB/UFS/inputdata/201908/20190829/gfs_4_20190829_0000_000.grb2 If you want the complete list of fix files that's a lot more complicated to provide. |
|
@jedwards4b Great, thanks. Yup, I need the fix files too to set up the dorian example on Stampede. It would be really nice for the non-pre-configured platforms to have another tarball in the ftp directory like the simple-test-case. Are you using the same fix files that are in that simple-test-case directory? Or pulling them out of global/fix? |
|
I'm not sure that I understand - if a user defines the required environment
variables and sets up a case the required files will be downloaded to the
platform- why would you want a tarball?
…On Sun, Mar 1, 2020 at 9:36 AM ceceliadid ***@***.***> wrote:
@jedwards4b <https://github.com/jedwards4b> Great, thanks. Yup, I need
the fix files too to set up the dorian example on Stampede. It would be
really nice for the non-pre-configured platforms to have another tarball in
the ftp directory like the simple-test-case. Are you using the same fix
files that are in that simple-test-case directory? Or pulling them out of
global/fix?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=ABOXUGBLM4EE6JBPPVBC2QLRFKFHJA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENND4KQ#issuecomment-593116714>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOXUGBMUPAOPNAE6FXO7YDRFKFHJANCNFSM4KUG5EQQ>
.
--
Jim Edwards
CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO
|
|
Okay, so if I'm on Stampede, CIME will go and get the fix files it needs from the EMC ftp site. I didn't understand that. During what CIME step (e.g. setup, build) are the .input_data_list files supposed to be created, and when is check_input_data called (assuming that's what gets the files)? Going into ./case.build the Buildconf directory has nothing in it for me - does that suggest an error? |
|
So good point - the files you need are listed in
Buildconf/ufsatm.input_data_list
check_input_data is called by case.submit
you can also call it by hand if you wish: ./check_input_data --download
…On Sun, Mar 1, 2020 at 1:05 PM ceceliadid ***@***.***> wrote:
Okay, so if I'm on Stampede, CIME will go and get the fix files it needs
from the EMC ftp site. I didn't understand that. During what CIME step
(e.g. setup, build) are the .input_data_list files supposed to be created,
and when is check_input_data called (assuming that's what gets the files)?
Going into ./case.build the Buildconf directory has nothing in it for me -
does that suggest an error?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=ABOXUGAQV5EFLRILUJCK7GDRFK5ZHA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENNJKFY#issuecomment-593138967>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOXUGAL226CIHTJVH7IM63RFK5ZHANCNFSM4KUG5EQQ>
.
--
Jim Edwards
CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO
|
|
On Stampede should I be setting the --machine argument for create_newcase to linux? It looks like that's what the instructions say to do. But I get ERROR: No machine linux found |
|
Nevermind, I realized I probably have to go to the conf_machines.xml file and deal with porting CIME to a new machine to get the example to work on Stampede. The documentation in 5.2 should probably say set machine to "centos7-linux" instead of linux if that's going to be the name of the more generic linux platform. I see that there is a stampede2-skx entry in conf_machines.xml, with lots of hardcoded Jim stuff in it. Would it make sense to make that more general and offer a Stampede template that people could start with? |
|
NO you don't need the --machine argument on stampede - the code is already
ported.
All you should need is to set UFS_INPUT and UFS_SCRATCH and to source the
script in NCEPlibs
…On Sun, Mar 1, 2020 at 3:23 PM ceceliadid ***@***.***> wrote:
Nevermind, I realized I probably have to go to the conf_machines.xml file
and deal with porting CIME to a new machine to get the example to work on
Stampede.
The documentation in 5.2 should probably say set machine to
"centos7-linux" instead of linux if that's going to be the name of the more
generic linux platform.
I see that there is a stampede2-skx entry in conf_machines.xml, with lots
of hardcoded Jim stuff in it. Would it make sense to make that more general
and offer a Stampede template that people could start with?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=ABOXUGHGJI5ZY4NAAERL4WLRFLN5VA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENNMYWY#issuecomment-593153115>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOXUGHFH3D34KFTKF6LCW3RFLN5VANCNFSM4KUG5EQQ>
.
--
Jim Edwards
CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO
|
|
@jedwards4b @climbfuji That doesn't work, I think because of the same permissions issue we ran into before. It looks like the case.setup goes to the config_machines.xml file with all the hardcoded paths in it. It is setting locations like DIN_LOC_ROOT following the values in the Stampede section, so DIN_LOC_ROOT is set to: /work/01118/tg803972/stampede2/UFS/ufs_inputdata That's the same UFS directory that Dom and I couldn't read before and I get a read error now as well. Does the Stampede2 app build depend on reading into that UFS directory? Or maybe I'm doing something wrong? |
|
Do you have the latest ufs-mrweather-app ? How are you setting
UFS_SCRATCH and UFS_INPUT?
cime should be in hash 035f22da1, as you can see here
<https://github.com/ESMCI/cime/blob/ufs_release_v1.0/config/ufs/machines/config_machines.xml#L340>,
cime should look wherever you've set UFS_INPUT for input data.
…On Sun, Mar 1, 2020 at 11:27 PM ceceliadid ***@***.***> wrote:
@jedwards4b <https://github.com/jedwards4b> @climbfuji
<https://github.com/climbfuji> That doesn't work, I think because of the
same permissions issue we ran into before. It looks like the case.setup
goes to the config_machines.xml file with all the hardcoded paths in it. It
is setting locations like DIN_LOC_ROOT following the values in the Stampede
section, so DIN_LOC_ROOT is set to:
/work/01118/tg803972/stampede2/UFS/ufs_inputdata
That's the same UFS directory that Dom and I couldn't read before and I
get a read error now as well. Does the Stampede2 app build depend on
reading into that UFS directory?
Or maybe I'm doing something wrong?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=ABOXUGDPIMSP5EQ5OD2NYHDRFNGVLA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENOCM7A#issuecomment-593241724>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOXUGD3AGUF3OHKV3CUZALRFNGVLANCNFSM4KUG5EQQ>
.
--
Jim Edwards
CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO
|
|
Thank you for answering so many questions over the weekend! I guess I should be leaving off the tag? For UFS_SCRATCH and UFS_INPUT I exported them as in the quickstart. |
|
Yes leave off the tag.
…On Mon, Mar 2, 2020 at 7:43 AM ceceliadid ***@***.***> wrote:
Thank you for answering so many questions over the weekend!
I'm using the command in the quick start:
git clone -b ufs-v1.0.0.alpha02
https://github.com/ufs-community/ufs-mrweather-app.git my_ufs_sandbox
I guess I should be leaving off the tag?
git clone https://github.com/ufs-community/ufs-mrweather-app.git
my_ufs_sandbox
For UFS_SCRATCH and UFS_INPUT I exported them as in the quickstart.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=ABOXUGD7QRY3TXSUEXXI47TRFPAXNA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPRYWQ#issuecomment-593435738>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABOXUGBAWV7SU4ACANGPUZLRFPAXNANCNFSM4KUG5EQQ>
.
--
Jim Edwards
CESM Software Engineer
National Center for Atmospheric Research
Boulder, CO
|
|
On cheyenne - I ran test SMS_Ld2.C96.GFSv15p2.cheyenne_intel --workflow ufs-mrweather I think that the ncl script needs more documentation - edit the script and set MaxLen=17 |
|
I can build now on Stampede. I was just about to put in the 48 hour run. |
|
We are working on the documentation here at DTC. Coming soon.
…On Mon, Mar 2, 2020 at 9:27 AM jedwards4b ***@***.***> wrote:
On cheyenne - I ran test SMS_Ld2.C96.GFSv15p2.cheyenne_intel --workflow
ufs-mrweather
and plotted the 48 hour results for comparison against
ftp.emc.ncep.noaa.gov/EIB/UFS/visualization_example/Dorian_C96_noNSST
I think that the ncl script needs more documentation -
I needed to make a subdirectory plots and run with
edit the script and set MaxLen=17
ncl 'Model="UFS"' 'initDate=2019082900' 'Dir="/glade/scratch/jedwards/SMS_Ld2.C96.GFSv15p2.cheyenne_intel.20200302_080140_u0rgyu/run/"' t2_wind10_GFS_grb.ncl
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=AE7WQAQUG3XRCI3OLATACG3RFPM6JA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENP62IQ#issuecomment-593489186>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE7WQAUW33TAA6MY2EBZ4ZDRFPM6JANCNFSM4KUG5EQQ>
.
|
|
The script is available in the ftp site and I am sure you can successfully
run it. But we will add a bit more documentation to it to reduce struggles.
…On Mon, Mar 2, 2020 at 9:29 AM ceceliadid ***@***.***> wrote:
I can build now on Stampede. I was just about to put in the 48 hour run.
I would like to test the plotting script once you have it.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#86?email_source=notifications&email_token=AE7WQAXLRFTKNR4NKEPBSIDRFPNHBA5CNFSM4KUG5EQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENP7DKA#issuecomment-593490344>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AE7WQAT6QNNYEETQWXWJTIDRFPNHBANCNFSM4KUG5EQQ>
.
|
|
@jedwards4b this issue seems to have morphed into a testing status update mechanism. Since the original issue has been resolved suggest closing this issue |
|
Yes I ran on cheyenne and was able to reproduce the plots from the ftp site |
Here is the complete output: /glade/scratch/jedwards/ufstest/run I don't have enough experience with this model to know if it's any good or not - @arunchawla-NOAA who should look at it?
The text was updated successfully, but these errors were encountered: