13505: Replace e2e up test with more performant version

[JFisk42]

This PR removes the old Universal Pipeline end2end test and replaces it with a more performant test. The goal of this new test is not to verify each step in the report lifecycle or lineage, but to verify that it is sent to the correct receivers and that the contents match what we expect them to be.

Note: Still new here so please feel free to correct any wording issues in documentation/logging/variable naming that may stem from incomplete understanding.

Test Steps:

1. Run ./gradlew testEnd2EndUP
2. Expect to see two reports created. One for the FULL_ELR workflow and the other for ELIMS.
3. Verify the receivers and sent file names in the output via curl --location 'http://localhost:7071/api/waters/report/[report_id]/history/'
4. Running in staging requires setting the AzureWebJobsStorage environment variable. Because that variable contains semicolons it will need to be set in CLI like export AzureWebJobsStorage="<key from azure>"

Changes

• Replaced the old UP e2e test with a test that posts a report. It then verifies the sent reports were sent to the correct receivers. It also pulls the resulting files from the blob store and verifies that the contents are what we expect.
• This allowed a reduction in local test run time from about 8 minutes to under 1.5 minutes.

Checklist

Testing

• Tested locally?
• Ran ./prime test or ./gradlew testSmoke against local Docker ReportStream container?

Process

• Are there licensing issues with any new dependencies introduced?
• Includes a summary of what a code reviewer should test/verify?
• Updated the release notes?
• Database changes are submitted as a separate PR?
• DevOps team has been notified if PR requires ops support?

Linked Issues

• Fixes Create Performant End-to-End UP Tests #13505

TODO

• https://app.zenhub.com/workspaces/platform-6182b02547c1130010f459db/issues/gh/cdcgov/prime-reportstream/14336

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details

Scanned Manifest Files

[github-actions]

Test Results

1 167 tests  ±0   1 163 ✅ ±0   7m 0s ⏱️ -16s
  145 suites ±0       4 💤 ±0 
  145 files   ±0       0 ❌ ±0 

Results for commit c509d01. ± Comparison against base commit 6ff05b9.

♻️ This comment has been updated with latest results.

[github-actions]

Integration Test Results

 62 files  ±0   62 suites  ±0   32m 23s ⏱️ - 1m 39s
392 tests ±0  382 ✅ ±0  10 💤 ±0  0 ❌ ±0 
395 runs  ±0  385 ✅ ±0  10 💤 ±0  0 ❌ ±0 

Results for commit c509d01. ± Comparison against base commit 6ff05b9.

♻️ This comment has been updated with latest results.

[mkalish]

Awesome stuff! Super exciting to see these getting improved. Let me know if you want to chat about any of my comments.

[arnejduranovic]

Looking a lot nicer than the other tests! Great job reducing the runtime.

[brick-green]

one question and one suggestion

[brick-green]

looking good to me!

[arnejduranovic]

My feeling is this quality filter may be too complicated for the test? Perhaps we put a very specific value in our test messages, and then have a simple filter like %messageId = "e2e_test_message_id"

[JFisk42]

I guess it just depends on what role if any you want for the quality filters in these tests. At the moment these exist simply because of the receivers I was copying from. I can modify it into something like you're suggesting and this would be an extra layer tested in these now.

[arnejduranovic]

Looks good and tests run locally in under 30 seconds! Prior to merging, let's verify these tests pass on staging and that deployment instructions are updated if needed.

[sonarcloud]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarCloud

[JFisk42]

Did final verification of the test in staging and updating deployment guide. Also noting here: The new step will be setting the AzureWebJobsStorage env var before trying to run the test in staging.