Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Re-enable C48mx500_3DVarAOWCDA test #2438

Closed
3 tasks
WalterKolczynski-NOAA opened this issue Mar 28, 2024 · 9 comments · Fixed by NOAA-EMC/GDASApp#1001 or #2543
Closed
3 tasks

Re-enable C48mx500_3DVarAOWCDA test #2438

WalterKolczynski-NOAA opened this issue Mar 28, 2024 · 9 comments · Fixed by NOAA-EMC/GDASApp#1001 or #2543
Labels
maintenance Regular updates and maintenance work

Comments

@WalterKolczynski-NOAA
Copy link
Contributor

What new functionality do you need?

The C48mx500_3DVarAOWCDA had to be disabled as the ocean prep obs job was failing when some categories of obs were not present in dump. Once GDASApp has been updated, the submodule in workflow should be updated and the test turned back.

What are the requirements for the new functionality?

C48mx500_3DVarAOWCDA completes successfully.

Acceptance Criteria

  • GDASApp updated to handle missing obs types
  • GDASApp hash updated in workflow
  • C48mx500_3DVarAOWCDA test reenabled on Hera

Suggest a solution (optional)

No response
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the maintenance Regular updates and maintenance work label Mar 28, 2024
Merged
7 tasks
@JessicaMeixner-NOAA
Copy link
Contributor

FYI @guillaumevernieres @CatherineThomas-NOAA

@guillaumevernieres guillaumevernieres mentioned this issue Mar 28, 2024
Merged
guillaumevernieres added a commit to NOAA-EMC/GDASApp that referenced this issue Mar 28, 2024
@guillaumevernieres
Remove sst's from obs proc (#1001)
30d803c 
Should fix
[global-workflow/issues/2438](NOAA-EMC/global-workflow#2438 (comment))
@WalterKolczynski-NOAA
Copy link
Contributor Author

This issue isn't resolved until the GDASapp hash has been updated in workflow

@guillaumevernieres
Copy link
Contributor

I didn't mean to close it @WalterKolczynski-NOAA , sorry! We just merged a fix in the GDASApp.

@JessicaMeixner-NOAA
Copy link
Contributor

This should be able to be re-enabled after #2514 (updating GSI hash) and #2420 (updating gdas app hash) are merged.

@JessicaMeixner-NOAA
Copy link
Contributor

The GSI hash has been updated. We are now waiting on #2420 to update the GDAS app hash. Hopefully this will be done soon so we can get this test turned back on.

@JessicaMeixner-NOAA
Copy link
Contributor

#2420 is now merged, however there is a new issue #2528 preventing this test from being turned back on.

@JessicaMeixner-NOAA
Copy link
Contributor

JessicaMeixner-NOAA commented Apr 24, 2024 •
edited
Loading

The issue #2528 has been resolved enough to get past the first error. Now we have a new error that:
FileNotFoundError: [Errno 2] No such file or directory: '/scratch1/NCEPDEV/climate/Jessica.Meixner/cycling/iau_02/C48mx500_3DVarAOWCDA_t03/COMROOT/c48t03/gdas.20210324/12//model_data/ice/restart/20210324.180000.cice_model.res.nc'
in gdasocnanalprep

@JessicaMeixner-NOAA
Copy link
Contributor

Currently in COM for ice restarts I have:

$ls /scratch1/NCEPDEV/climate/Jessica.Meixner/cycling/iau_02/C48mx500_3DVarAOWCDA_t03/COMROOT/c48t03/gdas.20210324/12/model_data/ice/restart/
20210324.150000.cice_model.res.nc

whereas before we had:

$ ls /scratch1/NCEPDEV/climate/Jessica.Meixner/cycling/turnonmarine/c48ci01/COMROOT/c48t01/gdas.20210324/12/model_data/ice/restart
20210324.150000.cice_model.res.nc  20210324.180000.cice_model.res.nc  20210324.210000.cice_model.res.nc

I believe this is a change from #2510 @aerorahul should additional ice restart files needed for DA be copied to COM or should the DA jobs be looking for this data in another directory?

@guillaumevernieres
Copy link
Contributor

guillaumevernieres commented Apr 24, 2024 •
edited
Loading

If this was tested without the "IAU" flag on, then there's a bug in the g-w. The restarts should be for 18z, not 15z. Also, it looks like the atmos restarts are available at 15-18 and 21, but this isn't the case for mom6 and cice6.

It's a bug in the marine DA @JessicaMeixner-NOAA . We're inserting the analysis in the wrong restart when DO_IAU is switched on ... Oops.

@guillaumevernieres guillaumevernieres mentioned this issue Apr 24, 2024
Closed
@JessicaMeixner-NOAA JessicaMeixner-NOAA mentioned this issue Apr 25, 2024
Merged
7 tasks
WalterKolczynski-NOAA pushed a commit that referenced this issue Apr 27, 2024
@JessicaMeixner-NOAA
Turn C48mx500_3DVarAOWCDA back on (#2543)
4b96c12 
This PR turns the C48mx500_3DVarAOWCDA test back on. This required a few
bugfixes in GDAS app which have now been merged.

Resolves #2438
Resolves #2528

co-authors: @guillaumevernieres and @aerorahul who provided the bug
fixes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
maintenance Regular updates and maintenance work
Projects
None yet
Milestone
No milestone
3 participants
@guillaumevernieres @JessicaMeixner-NOAA @WalterKolczynski-NOAA