Accessing GSI observing system records for active and available channels

[asewnath]

Description

Initial PR making changes required to update yaml use flags during cylc workflow run. PR does the following:

• Change amsua_n19.yaml to include active channel jinja templating in use_flag field
• Adding two new tasks: CloneGeosAna and GenerateObservingSystemRecords
• Created new object ObservingSystemRecords to handle parsing active and available channel tables from GSI
• Modified render_interface_observation function to call new get_active_channels function.
• Modifying relevant tasks to get observing_system_record_path for render_interface_observation or jedi_dictionary_iterator function

Changes that still need to be made: (can be in this PR or another one)

• Update all yaml files with use_flag field to have active channel jinja templating
• Available channels jinja templating in all yaml files
• Add logger into functions for proper aborts

[danholdaway]

Thank you very much for adding this very important functionality @asewnath. I just have a few minor comments about code readability. The overall functionality looks great.

The only other comment I would make is in having these three files: sat_db_utils, instr_state_machine and observing_system_records. Do we still need all three? What is the difference between what's in the files? It's not very easy to tell by the names of the files.

[asewnath]

Thank you very much for adding this very important functionality @asewnath. I just have a few minor comments about code readability. The overall functionality looks great.

The only other comment I would make is in having these three files: sat_db_utils, instr_state_machine and observing_system_records. Do we still need all three? What is the difference between what's in the files? It's not very easy to tell by the names of the files.

The observing_system_records file contains the ObservingSystemRecords object, which has methods to parse the GSI records and save the yaml files.instr_state_machine is an old name that can be changed to something like gsi_tbl_parser. It contains the code that actually parses the GSI records. Since it's a lot of code, it seems cleaner to have it in a separate file. The sat_db_utils file contains different utilities for the channel update process. read_sat_db is used only by ObservingSystemRecords and can be moved to observing_system_records. run_sat_db_process is legacy code that can be deleted. The only place it is called in is utilities/scripts/swell_sat_db_processing.py, which can also be deleted. Then there are functions that are used for the channel update that are not associated with the ObservingSystemRecords object. This includes get_active_channels and other functions necessary for the active channel retrieval. A separate file seems appropriate for these functions, and the file itself can be changed to get_active_channels or something along those lines. I can also add more documentation to all these files.

[danholdaway]

Thank you very much for adding this very important functionality @asewnath. I just have a few minor comments about code readability. The overall functionality looks great.
The only other comment I would make is in having these three files: sat_db_utils, instr_state_machine and observing_system_records. Do we still need all three? What is the difference between what's in the files? It's not very easy to tell by the names of the files.

The observing_system_records file contains the ObservingSystemRecords object, which has methods to parse the GSI records and save the yaml files.instr_state_machine is an old name that can be changed to something like gsi_tbl_parser. It contains the code that actually parses the GSI records. Since it's a lot of code, it seems cleaner to have it in a separate file. The sat_db_utils file contains different utilities for the channel update process. read_sat_db is used only by ObservingSystemRecords and can be moved to observing_system_records. run_sat_db_process is legacy code that can be deleted. The only place it is called in is utilities/scripts/swell_sat_db_processing.py, which can also be deleted. Then there are functions that are used for the channel update that are not associated with the ObservingSystemRecords object. This includes get_active_channels and other functions necessary for the active channel retrieval. A separate file seems appropriate for these functions, and the file itself can be changed to get_active_channels or something along those lines. I can also add more documentation to all these files.

Thanks for the explanation. I think some renaming and clean up of any unused code should be sufficient.