[develop] Add new RRFS variables such as NWGES and workflow control variables #647

danielabdi-noaa · 2023-03-07T13:53:26Z

DESCRIPTION OF CHANGES:

Addresses issue #654.

General (non-task specific) variables are added to config_defaults.yaml - New directory names NWGES, RRFSE_NWGES, ENSCTRL_COMIN etc,
new variables such as TAG.
Workflow control flags such as DO_SPINUP, DO_DACYCLE, DO_SOIL_ADJUST
Also RRFS_dev1 defines a bunch of new cycles besides forecast, initial etc, so those are added too.
Modifcations in generate_FV3LAM_wflow.py needed for creating input namelist files for DA cycles
Modifications to machine files, mostly worfklow related, but on jet/hera paths to data are also added

I think this PR is priority number 1 because of this since all tasks will depend on it, and can serve as a central place to coordinate workflow variables, common task variables etc.

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

TESTS CONDUCTED:

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

Define shared variables and predefined grids for all tasks #654

CHECKLIST

My code follows the style guidelines in the Contributor's Guide
I have performed a self-review of my own code using the Code Reviewer's Guide
I have commented my code, particularly in hard-to-understand areas
My changes need updates to the documentation. I have made corresponding changes to the documentation
My changes do not require updates to the documentation (explain).
My changes generate no new warnings
New and existing tests pass with my changes
Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

CONTRIBUTORS (optional):

MichaelLueken

@danielabdi-noaa Overall, these changes look good to me. I agree that I'm not entirely sure what the purpose of the SERVICE_ACCOUNT is in the ush/config_defaults.yaml file. I have reached out to @hu5970 to see if he can shine some light on it (or if it even needs to be included). Also, the newly added RRFS_CONUS_3km_HRRRIC grid appears to be identical to the RRFS_CONUScompact_3km grid. The only difference is that the DT_ATMOS value was decreased recently to use less wasteful values. So, I would suggest using RRFS_CONUScompact_3km rather than RRFS_CONUS_3km_HRRRIC.

MichaelLueken · 2023-04-03T14:52:28Z

ush/config_defaults.yaml

  #-----------------------------------------------------------------------
  MACHINE: "BIG_COMPUTER"
  ACCOUNT: ""
+  SERVICE_ACCOUNT: '{{ user.ACCOUNT }}'


It looks like the NOAA-GSL ush/config_defaults.yaml file has both a SERVICE_ACCOUNT and HPSS_ACCOUNT, where HPSS_ACCOUNT defaults to SERVICE_ACCOUNT if not set. @hu5970, are these entries required? I'm seeing that they appeared to be added for Jet. Any additional information would be greatly appreciated. Thanks!

MichaelLueken · 2023-04-03T16:20:55Z

ush/predef_grid_params.yaml

+    WRTCMP_lon_lwr_left: -122.17364391
+    WRTCMP_lat_lwr_left: 21.88588562
+    WRTCMP_dx: 3000.0
+    WRTCMP_dy: 3000.0


It looks like the RRFS_CONUS_3km_HRRRIC grid is identical to the RRFS_CONUScompact_3km grid, with the exception of DT_ATMOS being set to 36. This modification was merged at fd3c778 and was to use less wasteful values of DT_ATMOS. It would probably be best to have the scripts use the RRFS_CONUScompact_3km grid, rather than add the new RRFS_CONUS_3km_HRRRIC grid with a different DT_ATMOS value.

danielabdi-noaa · 2023-04-03T17:50:49Z

@MichaelLueken Thanks for capturing the predefined grid duplicates. It looks like there are no new predef grids to add, so I have removed all changes to predef_grid file.

MichaelLueken · 2023-04-05T18:06:43Z

@danielabdi-noaa Going through the latest changes in the NOAA-GSL/regional_workflow/RRFS_dev1 branch associated with ush/config_defaults.sh, I'm noting the following missing or changed options:

OUTPUT_FH was introduced in Add option to setup sub-hour forecast output frequency NOAA-GSL/regional_workflow#502
DO_ENS_RADDA was introduced in Adding a preliminary capability of radiance DA in EnKF NOAA-GSL/regional_workflow#496
DO_ENVAR_RADAR_REF_ONCE was introduced in Add options for Mixed Ensemble and Scale/Variable-Dependent Localization NOAA-GSL/regional_workflow#451
SAVEINPUT_CYCLEDEF was changed to SAVEDA_CYCLEDEF in Replace save_input with save_da_output and use save_input with save_r… NOAA-GSL/regional_workflow#480
DO_SAVE_INPUT was changed to DO_SAVE_DA_OUTPUT in Replace save_input with save_da_output and use save_input with save_r… NOAA-GSL/regional_workflow#480
The defaults for DA_CYCLE_INTERV, RESTART_INTERVAL, and RESTART_INTERVAL_LONG were changed to 1, 1,2, and 1,2, respectively, in updates for RRFS_A real-time NOAA-GSL/regional_workflow#477
FIRE_RRFS_ROOT, OBSPATH_PM, and DO_SDDACYCLE were introduced in Add PM2.5 DA for RRFS-SD NOAA-GSL/regional_workflow#448
RTMA_OBS_FEED was introduced in RTMA updates NOAA-GSL/regional_workflow#473
PREP_FVCOM was introduced in Add function to generate the FVCOM data process for RRFS NOAA-GSL/regional_workflow#472

Until this PR has been merged, I will keep an eye out on further changes to the ush/config_defaults.sh file.

danielabdi-noaa · 2023-04-05T23:18:27Z

@MichaelLueken Thanks a lot for going through the commits and summarizing the changes! I have updated this PR with most of the new variable changes, except for two, OUTPUT_FH goes under task_run_fcst and PREP_FVCOM goes under task_make_ics so those are put in future PRs for the respective tasks.

christinaholtNOAA

@danielabdi-noaa I appreciate your contribution here.

I am very concerned about this approach to adding a bunch of variables that aren't used yet. We have no way to test them, and no way to know whether they will be relevant or needed when the tasks that currently use them in RRFS_dev1 are contributed.

I am also concerned that it will be difficult to communicate what does and does not work to the rest of the community. What I mean is, if someone sees a flag that says "do_something" they expect "something" to work. Given that is not true, I do not think it's fair for us to include those sorts of flags until the feature is enabled.

christinaholtNOAA · 2023-04-06T05:31:22Z

ush/config.rrfs.yaml

This file should not be added yet. We don't have the capabilities in place to run or test any of it, and many of the settings here will not ever be supported in SRW.

Actually you can run it even now. It will generate a workflow file that doesn't have any of the rrfs tasks.

christinaholtNOAA · 2023-04-06T05:33:06Z

ush/config_defaults.yaml

@@ -5,7 +5,7 @@ metadata:
  description: >-
    Default configuration for an experiment. The valid values for most of the
    parameters are specified in valid_param_vals.yaml
-  version: !!str '1.0'
+  version: !!str '2.0'


It's not clear to me how we will want to bump versions, but it definitely doesn't seem like the time to do that now.

Why is this not the right time? Infact, everytime a variable is added is time for a minor version update.

It is generally unclear to me whether this should be the SRW version, the release version, etc.

christinaholtNOAA · 2023-04-06T05:36:06Z

ush/config_defaults.yaml

+  #
+  # RESERVATION_POST:
+  # The reservation for post tasks.  
+  #


Partitions and queues are now all set in the parm/wflow files. None of these are needed here. Reservations, when they are added, should also live in the parm/wflow files.

But the parm/wflow/default_worklfow.yaml files get the there settings from config files, infact, we aready have partitions/queues in them already, so I am just extending that for jobs RRFS adds

PARTITION_DEFAULT: '{{ platform.get("PARTITION_DEFAULT") }}' PARTITION_FCST: '{{ platform.get("PARTITION_FCST") }}' PARTITION_HPSS: '{{ platform.get("PARTITION_HPSS") }}' QUEUE_DEFAULT: '{{ platform.get("QUEUE_DEFAULT") }}' QUEUE_FCST: '{{ platform.get("QUEUE_FCST") }}' QUEUE_HPSS: '{{ platform.get("QUEUE_HPSS") }}'

I'm suggesting that we don't necessarily have to set them explicitly here in config defaults. Here, the default queue is the main queue on a platform. FCST is special because it might need bigger nodes with more memory, and HPSS has internet access -- essentially it's the front-end node on platforms where that makes sense. Because each of these three function in separate ways for the workflow, we can choose from those 3 which task runs under which queue in the task definition.

There is no need for a one-to-one correspondence between available config defaults variables and what values are set as entities.

So, in the definition of the post jobs, we can modify the entry to specifically be debug if we want to change it for a given user/platform.

christinaholtNOAA · 2023-04-06T05:36:43Z

ush/config_defaults.yaml

+  PARTITION_POST: ""
+  QUEUE_POST: ""
+  RESERVATION: ""
+  RESERVATION_POST: ""


None of these will be needed here as they can be set explicitly in the parm/wflow files for the tasks when they are added to the workflow.

See above! We are not setting them directly, but getting them from platform section of the config file.

As I argue above, we do not need them as they generally don't serve any additional function on our platforms than the current functions allow in the XML generation.

ush/config_defaults.yaml

ush/fill_jinja_template.py

christinaholtNOAA · 2023-04-06T06:01:53Z

ush/generate_FV3LAM_wflow.py

@@ -360,6 +359,8 @@ def generate_FV3LAM_wflow(ushdir, logfile: str = "log.generate_FV3LAM_wflow", de
        "target_lon": LON_CTR,
        "target_lat": LAT_CTR,
        "nrows_blend": HALO_BLEND,
+        "regional_bcs_from_gsi": False,
+        "write_restart_with_bcs": False,


Will this impact AQM?

No idea @chan-hoo probably knows.

christinaholtNOAA · 2023-04-06T06:07:40Z

ush/machine/hera.yaml

+      RAPHRR_SOIL_ROOT: ""
+      FIRE_RAVE_DIR: ""
+      AIRCRAFT_REJECT: "/scratch2/BMC/zrtrr/rli/data/amdar_reject_lists"
+      SFCOBS_USELIST: "/scratch2/BMC/zrtrr/rli/data/mesonet_uselists"


I don't think this is the appropriate structure for retrieve_data.py to gather these. I believe there are other efforts to get the data paths incorporated into the workflow.

Aren't these "fixed files" that do not change from test case to test case? retrieve_data.py has so far been used to get ICS/LBCS that change from one test case to test case. @EdwardSnyder-NOAA has put some of these files under epic role account and these paths can be changed as new files are added to locations that SRW can use.

These paths are a combination of fixed and observation files. We were trying to replicate the functionality from the set_rrfs_config.sh found in the dev1 branch. Essentially this script sets paths based on machine and model run (retro/current) type. It made sense to add the logic here even though the retrieve_data.py script isn't used to grab these files. We are definitely open for suggestions on placing this logic elsewhere.

In addition, how often should we utilize the retrieve_data.py script? As the process lightning task grabs lightning data in the jobs script without the retrieve_data.py script. Would we need to integrate all the rrfs obs data into the retrieve_data.py script?

I believe we should be using the retrieve_data script any time we need to pull data from disk and/or external resources.

One assumption made in RRFS_dev was that we'd always have data staged on disk for real-time or retro runs. That is not an appropriate assumption for the community who likely have different goals. SRW users working outside the real-time paradigm, or on platforms where real-time data streams aren't available, or on retrospective case studies, will need a way to pull data from archives like HPSS or AWS.

The retrieve_data tool is meant to allow a user to define whether they have staged data on disk via their own doing, or a supported data stream, or whether the workflow should go get it for them from known external locations.

christinaholtNOAA · 2023-04-06T06:10:13Z

ush/set_FV3nml_ens_stoch_seeds.py

@@ -58,7 +58,7 @@ def set_FV3nml_ens_stoch_seeds(cdate):
    #
    # -----------------------------------------------------------------------
    #
-    fv3_nml_ensmem_fp = f"{os.getcwd()}{os.sep}{FV3_NML_FN}"
+    fv3_nml_ensmem_fp = f"{os.getcwd()}{os.sep}{FV3_NML_FN}_base"


If the name of the file changes, it should change in config_defaults so that it's clear what it is from the beginning. One of the features of having configuration settings is that there is rarely a need to change these types of things in the scripts.

christinaholtNOAA · 2023-04-06T06:11:40Z

ush/set_FV3nml_ens_stoch_seeds.py

@@ -174,7 +174,7 @@ def setUp(self):
        mkdir_vrfy("-p", EXPTDIR)
        cp_vrfy(
            os.path.join(PARMdir, "input.nml.FV3"),
-            os.path.join(EXPTDIR, "input.nml"),
+            os.path.join(EXPTDIR, "input.nml_base"),


Here, it seems like we should be referencing the configuration setting instead of repeating this information, especially if we'd like to change it.

I am not sure what your concern is here, but the way RRFS handles input namelist files is different from how SRW does it. RRFS generates four different input namelist files and symlinks it to input.nml depending on the configuration, as opposed to in-place modification of one namelist file that SRW uses. This also applies for the stochastic physics options modification and I tested that this modification works for the WE2E test community_ensemble_2mems_stoch.

christinaholtNOAA · 2023-04-10T15:16:01Z

ush/config_defaults.yaml

@@ -5,7 +5,7 @@ metadata:
  description: >-
    Default configuration for an experiment. The valid values for most of the
    parameters are specified in valid_param_vals.yaml
-  version: !!str '1.0'
+  version: !!str '2.0'


It is generally unclear to me whether this should be the SRW version, the release version, etc.

christinaholtNOAA · 2023-04-10T15:23:41Z

ush/config_defaults.yaml

+  #
+  # RESERVATION_POST:
+  # The reservation for post tasks.  
+  #


I'm suggesting that we don't necessarily have to set them explicitly here in config defaults. Here, the default queue is the main queue on a platform. FCST is special because it might need bigger nodes with more memory, and HPSS has internet access -- essentially it's the front-end node on platforms where that makes sense. Because each of these three function in separate ways for the workflow, we can choose from those 3 which task runs under which queue in the task definition.

There is no need for a one-to-one correspondence between available config defaults variables and what values are set as entities.

So, in the definition of the post jobs, we can modify the entry to specifically be debug if we want to change it for a given user/platform.

christinaholtNOAA · 2023-04-10T15:24:43Z

ush/config_defaults.yaml

+  PARTITION_POST: ""
+  QUEUE_POST: ""
+  RESERVATION: ""
+  RESERVATION_POST: ""


As I argue above, we do not need them as they generally don't serve any additional function on our platforms than the current functions allow in the XML generation.

christinaholtNOAA · 2023-04-10T15:26:46Z

ush/config_defaults.yaml

+  # Setup default locations for FIRE_RRFS files and update time
+  #  FIRE_RAVE_DIR
+  #  FIRE_RRFS_ROOT
+  #  FIRE_RRFS_update_hour


The data: section of the machine files in SRW currently is meant to work with retrieve_data.py script to set where to get the datas on disk and is a structured section with the structure strictly defined by that tool. The section should remain consistent with the tool's existing behavior.

christinaholtNOAA · 2023-04-10T15:29:49Z

ush/config_defaults.yaml

+  CYCL_HRS: []
+  CYCL_HRS_SPINSTART: []
+  CYCL_HRS_PRODSTART: []
+  CYCL_HRS_PRODSTART_ENS: []
+  CYCL_HRS_RECENTER: []
+  CYCL_HRS_STOCH: []


This is why I'm suggesting that the necessary variables be added with the tool that requires them. They should be added when they are needed, not separately where they cannot be tested on their own, or recognized as to why they are useful.

They are included in PR #638 , and I suggest that be the only place they are added.

christinaholtNOAA · 2023-04-10T15:34:41Z

ush/config_defaults.yaml

+  POSTPROC_LEN_HRS: 1
+  POSTPROC_LONG_LEN_HRS: 1
+  FCST_LEN_HRS_SPINUP: 1
+  FCST_LEN_HRS_CYCLES: []


As you suggest, this one should not be included. A proposed change for the current solution for AQM is being discussed in Issue #709

christinaholtNOAA · 2023-04-10T15:38:06Z

ush/config_defaults.yaml

+  RRFSE_NWGES: '{{ OPSROOT }}/nwges'
+  RRFSE_NWGES_BASEDIR: '{{ RRFSE_NWGES }}'
+  ENSCTRL_NWGES: '{{ OPSROOT }}/nwges'
+  ENSCTRL_NWGES_BASEDIR: '{{ ENSCTRL_NWGES}}'


Putting in a clean implementation now is preferred. A main goal of the RRFS_dev1 merge is to take care of the refactoring needed to address some of the accidental complexities that have snuck in over time.

christinaholtNOAA · 2023-04-10T15:40:07Z

ush/config_defaults.yaml

+  START_TIME_CONVENTIONAL: 00:40:00
+  START_TIME_NSSLMOSIAC: 00:45:00
+  START_TIME_LIGHTNINGNC: 00:45:00
+  START_TIME_PROCSMOKE: 00:45:00


The rocoto section of the config files are 100% configurable. I am definitely not suggesting that the users edit any of the parm/wflow files. The sections defined in those parm/wflow files are completely configurable by the user in their own user config. There is no need to add the sections to the config_defaults.yaml file.

christinaholtNOAA · 2023-04-10T15:41:28Z

ush/config_defaults.yaml

+  #-----------------------------------------------------------------------
+  #
+  LBCS_SEARCH_HRS: 6
+  EXTRN_MDL_LBCS_SEARCH_OFFSET_HRS: 0


Please see my YAGNI argument above. It is a standard, industry wide best practice, and does apply in this case.

christinaholtNOAA · 2023-04-10T15:48:56Z

ush/machine/hera.yaml

+      RAPHRR_SOIL_ROOT: ""
+      FIRE_RAVE_DIR: ""
+      AIRCRAFT_REJECT: "/scratch2/BMC/zrtrr/rli/data/amdar_reject_lists"
+      SFCOBS_USELIST: "/scratch2/BMC/zrtrr/rli/data/mesonet_uselists"


I believe we should be using the retrieve_data script any time we need to pull data from disk and/or external resources.

One assumption made in RRFS_dev was that we'd always have data staged on disk for real-time or retro runs. That is not an appropriate assumption for the community who likely have different goals. SRW users working outside the real-time paradigm, or on platforms where real-time data streams aren't available, or on retrospective case studies, will need a way to pull data from archives like HPSS or AWS.

The retrieve_data tool is meant to allow a user to define whether they have staged data on disk via their own doing, or a supported data stream, or whether the workflow should go get it for them from known external locations.

danielabdi-noaa changed the title ~~Feature/rrfs params~~ [develop] Add new RRFS variables such as NWGES and controls Mar 7, 2023

danielabdi-noaa force-pushed the feature/rrfs_params branch from 013a28f to 0b8ebc4 Compare March 7, 2023 14:01

danielabdi-noaa marked this pull request as ready for review March 7, 2023 14:08

danielabdi-noaa mentioned this pull request Mar 7, 2023

[develop] Add process bufr task. #636

Merged

37 tasks

danielabdi-noaa linked an issue Mar 8, 2023 that may be closed by this pull request

Define shared variables and predefined grids for all tasks #654

Closed

danielabdi-noaa mentioned this pull request Mar 8, 2023

[develop] Update forecast task for RRFS_dev1 run. #658

Closed

37 tasks

danielabdi-noaa added the Priority: HIGH label Mar 9, 2023

danielabdi-noaa added a commit to danielabdi-noaa/ufs-srweather-app that referenced this pull request Mar 9, 2023

Remove FIXgsi as it will be part of PR ufs-community#647

76f89fc

danielabdi-noaa force-pushed the feature/rrfs_params branch 2 times, most recently from e9f8a61 to 7881c7f Compare March 10, 2023 02:38

This was referenced Mar 10, 2023

[develop] Modify get_lbcs and make_lbcs. #666

Merged

[develop] Update post-processing task for RRFS_dev1. #660

Closed

danielabdi-noaa added 2 commits March 25, 2023 19:01

Revert back to old way of creating FIXam

4523e8c

Other fixes.

289a318

danielabdi-noaa force-pushed the feature/rrfs_params branch from 01a0649 to 289a318 Compare March 25, 2023 19:13

danielabdi-noaa added 3 commits March 27, 2023 00:41

Remove unused variables.

f32b6af

Revert accidently removed conditions.

adb2eee

Update realtime lightning_root on jet.

dd4749a

danielabdi-noaa force-pushed the feature/rrfs_params branch from a6bde34 to dd4749a Compare March 30, 2023 02:39

danielabdi-noaa added 2 commits March 31, 2023 16:20

Update lightning_root paths for retro runs.

be2db74

Merge branch 'develop' into feature/rrfs_params

93d5374

danielabdi-noaa added the Priority: HIGH label Apr 1, 2023

Update test config file to conform to new format.

d33219b

MichaelLueken reviewed Apr 3, 2023

View reviewed changes

Remove 3km HRRRIC predef grid.

c94ef05

danielabdi-noaa changed the title ~~[develop] Add new RRFS variables such as NWGES and controls~~ [develop] Add new RRFS variables such as NWGES and workflow control variables Apr 3, 2023

danielabdi-noaa added 2 commits April 3, 2023 17:43

Add HPSS_ACCOUNT.

96fef95

Keep predef grids we have in SRW.

992becc

danielabdi-noaa added 2 commits April 5, 2023 16:12

Add missing config options.

5598722

Merge branch 'develop' into feature/rrfs_params

c05132e

Add some variable updates from RRFS_dev1

dc5340f

christinaholtNOAA requested changes Apr 6, 2023

View reviewed changes

danielabdi-noaa added 2 commits April 7, 2023 03:21

Remove ARCHIVEDIR and NCL vars.

08737e0

Bug fix FCST_LEN_CYCL

0db4606

christinaholtNOAA requested changes Apr 10, 2023

View reviewed changes

danielabdi-noaa mentioned this pull request Apr 10, 2023

[develop] Integrate jinja-templated workflow and add some rrfs workflow files. #721

Closed

37 tasks

danielabdi-noaa closed this Apr 10, 2023

EdwardSnyder-NOAA mentioned this pull request Apr 12, 2023

[develop] Add process lightning task #644

Merged

37 tasks

christinaholtNOAA removed the Priority: HIGH label Apr 25, 2023

[develop] Add new RRFS variables such as NWGES and workflow control variables #647

[develop] Add new RRFS variables such as NWGES and workflow control variables #647

Conversation

danielabdi-noaa commented Mar 7, 2023 • edited Loading

DESCRIPTION OF CHANGES:

Type of change

TESTS CONDUCTED:

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

CHECKLIST

LABELS (optional):

CONTRIBUTORS (optional):

MichaelLueken left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielabdi-noaa commented Apr 3, 2023

MichaelLueken commented Apr 5, 2023

danielabdi-noaa commented Apr 5, 2023

christinaholtNOAA left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielabdi-noaa Apr 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielabdi-noaa Apr 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielabdi-noaa Apr 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielabdi-noaa Apr 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielabdi-noaa commented Mar 7, 2023 •

edited

Loading

danielabdi-noaa Apr 7, 2023 •

edited

Loading

danielabdi-noaa Apr 7, 2023 •

edited

Loading

danielabdi-noaa Apr 7, 2023 •

edited

Loading

danielabdi-noaa Apr 7, 2023 •

edited

Loading