AATAMS SATTAG - schemas - to keep or to delete ?

[lbesnard]

The following legacy schemas

• aatams_sattag_dm
• aatams_sattag_nrt

have new respective schemas:

• aatams_sattag_qc_dm
• aatams_sattag_qc_nrt

1. Should both legacy ones be kept? If yes why?
2. Should the SQL reporting queries (https://github.com/aodn/reporting/blob/master/report/SQL_reporting_queries/aatams_sattag.sql) be modified accordingly?

[evacougnon]

Unfortunately, the 2 new schemas do not replace each of the old schemas.

At this stage we have to keep aatams_sattag_dm as we still have a dataset collection available on the Portal. This collection is different from aatams_sattag_qc_dm as it includes historical data.

aatams_sattag_nrt can probably be decommissioned, to be discussed with @ggalibert and @atkinsn . Earlier this year we decommissioned the aatams_sattag_nrt datatset collection from the portal as well as the pipeline. @atkinsn kept (and updated accordingly) the metadata records. I don't know what's the procedure about decommissioning a schema but I guess we do not use the data anymore. Same about reporting, I don't think we have anything to report about aatams_sattag_nrt.

[lbesnard]

happy to investigate if this aatams_sattag_nrt can be decomissioned

[lbesnard]

So unfortunately, the aatams_sattag_dm schema relies on the aatams_sattag_dm schema. There is an interdependence which would break the the aatams_sattag_dm harvester.
see https://github.com/aodn/chef-private/blob/master/data_bags/talend/aatams_sattag_dm.json#L5

The aatams_sattag_dm schema cannot be removed unless both collections (NRT and DM) can be decommissioned