[TTAHUB-925]remove-erroneous-records

[GarrettEHill]

Description of change

A bas assumption was made during the migration that caused addition erroneous records to be created. This will remove these records.

How to test

Issue(s)

• https://ocio-jira.acf.hhs.gov/browse/TTAHUB-925

Checklists

Every PR

• Meets issue criteria
• JIRA ticket status updated
• Code is meaningfully tested
• Meets accessibility standards (WCAG 2.1 Levels A, AA)
• API Documentation updated
• Boundary diagram updated
• Logical Data Model updated
• Architectural Decision Records written for major infrastructure decisions

Production Deploy

• Staging smoke test completed

After merge/deploy

• Update JIRA ticket status

[hardwarehuman]

This looks right to me, though I'm wondering if we might want to leave the erroneous records as full permanent tables for now just in case we make a mistake here and want to restore certain records. We can drop the tables later.

[hardwarehuman]

A more standard and scalable way to do something like this is:

CREATE TEMP TABLE err_obj AS
WITH correct_aros AS (
SELECT DISTINCT
  o.id "objectiveId",
  aro.id "aroId"
FROM "Objectives" o
JOIN "ActivityReportObjectives" aro
  ON o.id = aro."objectiveId"
EXCEPT
SELECT DISTINCT
  "objectiveId",
  "aroId"
FROM "erroneousAROs"
),
correct_objectives AS (SELECT DISTINCT "objectiveId" FROM correct_aros)
SELECT
  o.id "objectiveId",
  o."goalId"
FROM "Objectives" o
LEFT JOIN correct_aros c
  ON c."objectiveId" = o.id
WHERE c."objectiveId" IS NULL

added dedupe of objectives on csv file

[thewatermethod]

Works great! Should we add a test case for the objectives roll up?

[GarrettEHill]

Works great! Should we add a test case for the objectives roll up?

I already modified the current tests to include a roll up objective.

[AdamAdHocTeam]

This looks right to me, though I'm wondering if we might want to leave the erroneous records as full permanent tables for now just in case we make a mistake here and want to restore certain records. We can drop the tables later.

+1