make_bcs output directory reorg #729

biljanaorescanin · 2023-03-20T12:39:45Z

Work towards new output structure for boundary conditions.
Removing "SMAP" from EASE grid cases. Changes were minimal since Weiyuan already prepared all the codes.

Requires first merging of #718.

cc: @gmao-rreichle @weiyuan-jiang

biljanaorescanin · 2023-03-20T12:44:02Z

I've tested few cases and all were zero diff. ( NLv3 C720, NLv5 M09, ICA C180 and one LatLon case for ICA. )
@gmao-rreichle let me know if you think I should run for more cases.

github-actions · 2023-03-20T21:11:55Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

gmao-rreichle · 2023-03-20T21:19:38Z

Thanks, @biljanaorescanin. The changes look ok to me, but I think it would be better to put this branch on top of the branch associated with #718, and point the PR to that branch until #718 has been merged. Currently, there's some overlap between the PRs.
The present PR is meant to have make_bcs create the bcs files in the new dir tree structure, right?
If that's the case, then I think we should change the title of the PR to something like "make_bcs output directory reorg", and maybe change the title of #718 to something like "update make_bcs legacy input path.

github-actions · 2023-03-20T23:20:35Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

…orescan_restructure_bcs

github-actions · 2023-03-20T23:26:44Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

…orescan_restructure_bcs

github-actions · 2023-03-20T23:29:09Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

biljanaorescanin · 2023-03-23T12:51:51Z

@weiyuan-jiang we still have conflicts.

We should probably also change this in make_bcs and python codes, after email we all got:
#SBATCH --ntasks=28
#SBATCH --constraint=sky
to
#SBATCH --constraint="[cas|sky]" and use $SLURM_CPUS_ON_NODE

weiyuan-jiang · 2023-03-23T13:28:53Z

@weiyuan-jiang we still have conflicts.

We should probably also change this in make_bcs and python codes, after email we all got: #SBATCH --ntasks=28 #SBATCH --constraint=sky to #SBATCH --constraint="[cas|sky]" and use $SLURM_CPUS_ON_NODE

I have not finished my change on python program. You can go ahead change your make_bcs script, I will follow

change constraint option

weiyuan-jiang · 2023-03-23T18:24:38Z

@biljanaorescanin , Have you tried to submitted many jobs? I have concern over this line

GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Utils/Raster/makebcs/make_bcs

Line 945 in 4b33979

/bin/rm -r $OUTDIR

It appears at lat-lon, cubed-sphere and EASE cases. If some jobs are submitted simultaneously, this directory may be deleted by the other job that finishes earlier.

weiyuan-jiang · 2023-03-23T18:26:48Z

Now make_bcs.py should be in sync with make_bcs. The three program make_latlon_bcs.py, make_cube_bcs.py and make_ease_bcs.py can be tested independently

biljanaorescanin · 2023-03-23T18:41:42Z

@biljanaorescanin , Have you tried to submitted many jobs? I have concern over this line

GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Utils/Raster/makebcs/make_bcs

Line 945 in 4b33979

/bin/rm -r $OUTDIR

It appears at lat-lon, cubed-sphere and EASE cases. If some jobs are submitted simultaneously, this directory may be deleted by the other job that finishes earlier.

I would have to try again. I did run before and there was no issues. But it has been a while so I don't know for sure.

biljanaorescanin · 2023-03-23T19:28:18Z

Looks like running CF and LatLon cases is OK in all combinations, for many resolutions at the time, but any experiment with EASE has to be run separately just EASE.
This part we added for version:
Options for EASE grid version (land-only):
EASEv1
EASEv2 (default)

Gets appended anytime we have EASE in combination... so I get for example this mess if I do:
Land BCs version: ICA
Atmospheric resolution: b c12 M36
EASE grid version: EASEv2
Ocean resolution: O1
SMAP_EASEv2_M36.j
SMAP_EASEv2_c12.j
SMAP_EASEv2_b.j

I see during testing I didn't run many combinations at the time but individual and if I did test many cases they were in same case type CF, EASE or LatLon. So that OUTDIR you asked about is not the problem but EASE grid version being appended.

gmao-rreichle · 2023-03-23T20:50:03Z

Looks like running CF and LatLon cases is OK in all combinations, for many resolutions at the time, but any experiment with EASE has to be run separately just EASE.

@biljanaorescanin, @weiyuan-jiang: For make_bcs only, maybe we can just add augment the following comment:

GEOSgcm_GridComp/GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Utils/Raster/makebcs/make_bcs

Line 182 in 4b33979

    
           echo "Enter alphanumeric code(s) for atmospheric horizontal resolution from the above list"

with something like "Do NOT specify a mix EASE grid and cs/LatLon grid resolutions."

This should be enough of a fix for make_bcs until it is deprecated. I assume the same problem does not occur in the python version of make_bcs.

Does this make sense?

weiyuan-jiang · 2023-03-24T17:25:20Z

Looks like running CF and LatLon cases is OK in all combinations, for many resolutions at the time, but any experiment with EASE has to be run separately just EASE.

I still think we should remove that line. The OUTDIR is shared across all the jobs. You got lucky if the jobs are executed sequentially.

biljanaorescanin · 2023-03-24T17:28:13Z

Looks like running CF and LatLon cases is OK in all combinations, for many resolutions at the time, but any experiment with EASE has to be run separately just EASE.

I still think we should remove that line. The OUTDIR is shared across all the jobs. You got lucky if the jobs are executed sequentially.

We should then remove it for python. Let me know when it is ready so I can start testing.

gmao-rreichle

approving for surface-preproc-team

tclune

CMake changes ok.

Glob be dangerous ...

GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Utils/Raster/makebcs/CMakeLists.txt

…/Utils/Raster/makebcs/CMakeLists.txt Co-authored-by: Matthew Thompson <matthew.thompson@nasa.gov>

github-actions · 2023-05-08T17:40:20Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

gmao-rreichle

I accepted & merged Matt's suggestion and re-approved the PR

The base branch was changed.

github-actions · 2023-05-10T14:21:30Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

github-actions · 2023-05-10T14:23:42Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

github-actions · 2023-05-10T14:24:19Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

github-actions · 2023-05-15T20:28:37Z

Label error. Requires exactly 0 of: Contingent - DNA, Needs Lead Approval, Contingent -- Do Not Approve. Found: 0 diff, Contingent - DNA

biljanaorescanin added 3 commits March 15, 2023 13:31

rearrange bcs directories + some cleanup

2e28335

text edit

67d45cd

change time

d9a2555

biljanaorescanin added the 0 diff The changes in this pull request have verified to be zero-diff with the target branch. label Mar 20, 2023

additional cleanup

b94854e

gmao-rreichle added the Contingent - DNA These changes are contingent on other PRs (DNA=do not approve) label Mar 20, 2023

biljanaorescanin changed the title ~~makebcs directory reorg~~ make_bcs output directory reorg Mar 20, 2023

biljanaorescanin changed the base branch from develop to feature/borescan_update_legacy_bcs_path March 20, 2023 23:25

Merge branch 'feature/borescan_update_legacy_bcs_path' into feature/b…

924ef20

…orescan_restructure_bcs

Merge branch 'feature/borescan_update_legacy_bcs_path' into feature/b…

efa5036

…orescan_restructure_bcs

initial merger from feature/wjiang/make_bcs_python

b4141a8

weiyuan-jiang mentioned this pull request Mar 22, 2023

Feature/wjiang/make bcs python #730

Closed

biljanaorescanin and others added 2 commits March 23, 2023 10:29

Update make_bcs

b3b18cb

change constraint option

sync with make_bcs job script

4b33979

add comment for ease grid

1cc21ba

gmao-rreichle approved these changes May 8, 2023

View reviewed changes

gmao-rreichle marked this pull request as ready for review May 8, 2023 14:19

gmao-rreichle requested review from a team as code owners May 8, 2023 14:19

tclune previously approved these changes May 8, 2023

View reviewed changes

gmao-rreichle approved these changes May 8, 2023

View reviewed changes

mathomp4 reviewed May 8, 2023

View reviewed changes

GEOSagcm_GridComp/GEOSphysics_GridComp/GEOSsurface_GridComp/Utils/Raster/makebcs/CMakeLists.txt Outdated Show resolved Hide resolved

Update GEOSagAdd CONFIGURE_DEPENDS for python to GEOSsurface_GridComp…

4f85f89

…/Utils/Raster/makebcs/CMakeLists.txt Co-authored-by: Matthew Thompson <matthew.thompson@nasa.gov>

gmao-rreichle previously approved these changes May 8, 2023

View reviewed changes

gmao-rreichle changed the base branch from feature/borescan_update_legacy_bcs_path to develop May 10, 2023 14:20

gmao-rreichle requested a review from a team as a code owner May 10, 2023 14:20

Merge branch 'develop' into feature/borescan_restructure_bcs

62141a6

gmao-rreichle approved these changes May 10, 2023

View reviewed changes

Merge branch 'develop' into feature/borescan_restructure_bcs

60a905e

sdrabenh approved these changes May 16, 2023

View reviewed changes

sdrabenh removed the Contingent - DNA These changes are contingent on other PRs (DNA=do not approve) label May 17, 2023

Merge branch 'develop' into feature/borescan_restructure_bcs

2caed56

mathomp4 self-requested a review May 17, 2023 14:31

mathomp4 approved these changes May 17, 2023

View reviewed changes

sdrabenh merged commit 1b4a86d into develop May 17, 2023
4 checks passed

sdrabenh deleted the feature/borescan_restructure_bcs branch May 17, 2023 14:32

biljanaorescanin mentioned this pull request May 24, 2023

EASE grid naming convention GEOS-ESM/GEOSldas#605

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make_bcs output directory reorg #729

make_bcs output directory reorg #729

biljanaorescanin commented Mar 20, 2023 •

edited by gmao-rreichle

Loading

biljanaorescanin commented Mar 20, 2023 •

edited

Loading

github-actions bot commented Mar 20, 2023

gmao-rreichle commented Mar 20, 2023

github-actions bot commented Mar 20, 2023

github-actions bot commented Mar 20, 2023

github-actions bot commented Mar 20, 2023

biljanaorescanin commented Mar 23, 2023

weiyuan-jiang commented Mar 23, 2023

weiyuan-jiang commented Mar 23, 2023

weiyuan-jiang commented Mar 23, 2023

biljanaorescanin commented Mar 23, 2023

biljanaorescanin commented Mar 23, 2023

gmao-rreichle commented Mar 23, 2023

weiyuan-jiang commented Mar 24, 2023

biljanaorescanin commented Mar 24, 2023

gmao-rreichle left a comment

tclune left a comment

github-actions bot commented May 8, 2023

gmao-rreichle left a comment

github-actions bot commented May 10, 2023

github-actions bot commented May 10, 2023

github-actions bot commented May 10, 2023

github-actions bot commented May 15, 2023

make_bcs output directory reorg #729

make_bcs output directory reorg #729

Conversation

biljanaorescanin commented Mar 20, 2023 • edited by gmao-rreichle Loading

biljanaorescanin commented Mar 20, 2023 • edited Loading

github-actions bot commented Mar 20, 2023

gmao-rreichle commented Mar 20, 2023

github-actions bot commented Mar 20, 2023

github-actions bot commented Mar 20, 2023

github-actions bot commented Mar 20, 2023

biljanaorescanin commented Mar 23, 2023

weiyuan-jiang commented Mar 23, 2023

weiyuan-jiang commented Mar 23, 2023

weiyuan-jiang commented Mar 23, 2023

biljanaorescanin commented Mar 23, 2023

biljanaorescanin commented Mar 23, 2023

gmao-rreichle commented Mar 23, 2023

weiyuan-jiang commented Mar 24, 2023

biljanaorescanin commented Mar 24, 2023

gmao-rreichle left a comment

Choose a reason for hiding this comment

tclune left a comment

Choose a reason for hiding this comment

github-actions bot commented May 8, 2023

gmao-rreichle left a comment

Choose a reason for hiding this comment

github-actions bot commented May 10, 2023

github-actions bot commented May 10, 2023

github-actions bot commented May 10, 2023

github-actions bot commented May 15, 2023

biljanaorescanin commented Mar 20, 2023 •

edited by gmao-rreichle

Loading

biljanaorescanin commented Mar 20, 2023 •

edited

Loading