Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Long-form symlinks for final atm restart file should be relative, not absolute #1446

Closed
WalterKolczynski-NOAA opened this issue Apr 5, 2023 · 0 comments · Fixed by #1816
Closed

Long-form symlinks for final atm restart file should be relative, not absolute #1446

WalterKolczynski-NOAA opened this issue Apr 5, 2023 · 0 comments · Fixed by #1816
Assignees
@WalterKolczynski-NOAA
Labels
bug Something isn't working

Comments

@WalterKolczynski-NOAA
Copy link
Contributor

Expected behavior
Long-form symlinks for the final restart time should be relative (pointing to the files in the same directory), not contain the full path.

Current behavior
The long-form symlinks for the final restart time use absolute paths to point to files in the same directory. This will cause issues if the directory is moved or archived in a tarball.

lrwxrwxrwx 1 wkolczyn global  128 Apr  4 06:05 20211227.000000.ca_data.tile1.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/ca_data.tile1.nc
lrwxrwxrwx 1 wkolczyn global  128 Apr  4 06:05 20211227.000000.ca_data.tile2.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/ca_data.tile2.nc
lrwxrwxrwx 1 wkolczyn global  128 Apr  4 06:05 20211227.000000.ca_data.tile3.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/ca_data.tile3.nc
lrwxrwxrwx 1 wkolczyn global  128 Apr  4 06:05 20211227.000000.ca_data.tile4.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/ca_data.tile4.nc
lrwxrwxrwx 1 wkolczyn global  128 Apr  4 06:05 20211227.000000.ca_data.tile5.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/ca_data.tile5.nc
lrwxrwxrwx 1 wkolczyn global  128 Apr  4 06:05 20211227.000000.ca_data.tile6.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/ca_data.tile6.nc
lrwxrwxrwx 1 wkolczyn global  123 Apr  4 06:05 20211227.000000.coupler.res -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/coupler.res
lrwxrwxrwx 1 wkolczyn global  126 Apr  4 06:05 20211227.000000.fv_core.res.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_core.res.nc
lrwxrwxrwx 1 wkolczyn global  132 Apr  4 06:05 20211227.000000.fv_core.res.tile1.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_core.res.tile1.nc
lrwxrwxrwx 1 wkolczyn global  132 Apr  4 06:05 20211227.000000.fv_core.res.tile2.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_core.res.tile2.nc
lrwxrwxrwx 1 wkolczyn global  132 Apr  4 06:05 20211227.000000.fv_core.res.tile3.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_core.res.tile3.nc
lrwxrwxrwx 1 wkolczyn global  132 Apr  4 06:05 20211227.000000.fv_core.res.tile4.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_core.res.tile4.nc
lrwxrwxrwx 1 wkolczyn global  132 Apr  4 06:05 20211227.000000.fv_core.res.tile5.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_core.res.tile5.nc
lrwxrwxrwx 1 wkolczyn global  132 Apr  4 06:05 20211227.000000.fv_core.res.tile6.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_core.res.tile6.nc
lrwxrwxrwx 1 wkolczyn global  135 Apr  4 06:05 20211227.000000.fv_srf_wnd.res.tile1.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_srf_wnd.res.tile1.nc
lrwxrwxrwx 1 wkolczyn global  135 Apr  4 06:05 20211227.000000.fv_srf_wnd.res.tile2.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_srf_wnd.res.tile2.nc
lrwxrwxrwx 1 wkolczyn global  135 Apr  4 06:05 20211227.000000.fv_srf_wnd.res.tile3.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_srf_wnd.res.tile3.nc
lrwxrwxrwx 1 wkolczyn global  135 Apr  4 06:05 20211227.000000.fv_srf_wnd.res.tile4.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_srf_wnd.res.tile4.nc
lrwxrwxrwx 1 wkolczyn global  135 Apr  4 06:05 20211227.000000.fv_srf_wnd.res.tile5.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_srf_wnd.res.tile5.nc
lrwxrwxrwx 1 wkolczyn global  135 Apr  4 06:05 20211227.000000.fv_srf_wnd.res.tile6.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_srf_wnd.res.tile6.nc
lrwxrwxrwx 1 wkolczyn global  134 Apr  4 06:05 20211227.000000.fv_tracer.res.tile1.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_tracer.res.tile1.nc
lrwxrwxrwx 1 wkolczyn global  134 Apr  4 06:05 20211227.000000.fv_tracer.res.tile2.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_tracer.res.tile2.nc
lrwxrwxrwx 1 wkolczyn global  134 Apr  4 06:05 20211227.000000.fv_tracer.res.tile3.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_tracer.res.tile3.nc
lrwxrwxrwx 1 wkolczyn global  134 Apr  4 06:05 20211227.000000.fv_tracer.res.tile4.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_tracer.res.tile4.nc
lrwxrwxrwx 1 wkolczyn global  134 Apr  4 06:05 20211227.000000.fv_tracer.res.tile5.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_tracer.res.tile5.nc
lrwxrwxrwx 1 wkolczyn global  134 Apr  4 06:05 20211227.000000.fv_tracer.res.tile6.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/fv_tracer.res.tile6.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.phy_data.tile1.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/phy_data.tile1.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.phy_data.tile2.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/phy_data.tile2.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.phy_data.tile3.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/phy_data.tile3.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.phy_data.tile4.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/phy_data.tile4.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.phy_data.tile5.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/phy_data.tile5.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.phy_data.tile6.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/phy_data.tile6.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.sfc_data.tile1.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/sfc_data.tile1.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.sfc_data.tile2.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/sfc_data.tile2.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.sfc_data.tile3.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/sfc_data.tile3.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.sfc_data.tile4.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/sfc_data.tile4.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.sfc_data.tile5.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/sfc_data.tile5.nc
lrwxrwxrwx 1 wkolczyn global  129 Apr  4 06:05 20211227.000000.sfc_data.tile6.nc -> /work2/noaa/global/wkolczyn/noscrub/global-workflow/com_refactor_full/gfs.20211222/00//model_data/atmos/restart/sfc_data.tile6.nc

Machines affected
All

To Reproduce
Run global workflow for any experiment with a restart interval and observe the atm restart directory,

Context
Restart files generally contain the valid time of the restart (YYYYMMDD.HHmmss), but for the restart files written at the end of the forecast that verification time is omitted. For uniformity, symlinks were added for these final restart files that contain the verification time.

Additional Information
This issue may be mooted by #1277 (see also ufs-community/ufs-weather-model#1577)

Possible Implementation
Make the links relative or just wait for #1277
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the bug Something isn't working label Apr 5, 2023
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue May 1, 2023
@WalterKolczynski-NOAA
Change write tasks back to per thread
94442ed 
After additional investigation, it was found that `WRTTASK_PER_GROUP`
really has changed to a per-thread value.

Refs: NOAA-EMC#1446, NOAA-EMC#1499
@WalterKolczynski-NOAA WalterKolczynski-NOAA self-assigned this Jul 17, 2023
WalterKolczynski-NOAA added a commit to WalterKolczynski-NOAA/global-workflow that referenced this issue Aug 22, 2023
@WalterKolczynski-NOAA
Replace full-path restart links with relative links
ade24c2 
Symlinks for the final restart time used the full path name, even
though the targets are in the same directory. This means the links
would break if the directory were moved or (more importantly) put
in a tarball. The links have now been replaced with relative links.

Resolves NOAA-EMC#1446
Merged
7 tasks
WalterKolczynski-NOAA added a commit that referenced this issue Aug 24, 2023
@WalterKolczynski-NOAA
Replace full-path restart links with relative links (#1816)
cc21f2b 
Symlinks for the final restart time used the full path name, even
though the targets are in the same directory. This means the links
would break if the directory were moved or (more importantly) put
in a tarball. The links have now been replaced with relative links.

Resolves #1446
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@WalterKolczynski-NOAA WalterKolczynski-NOAA

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Replace full-path restart links with relative links WalterKolczynski-NOAA/global-workflow
1 participant
@WalterKolczynski-NOAA