In 1052 add guardrails final runs

[jonavellecuerdo]

Purpose and background context

Add guardrails to prevent (or at least minimize) accidental executions of "final" runs of Alma SAP Invoices processing. This is achieved by creating a confirmation page that the Flask application directs users to when selecting to Execute a final run (see screenshots below). This PR also includes the following updates:

• improving the runtime of retrieving logs from CloudWatch (see Jira ticket IN-1057)(https://mitlibraries.atlassian.net/browse/IN-1057)
• updating webapp.config.Config to construct the ALMA_SAP_INVOICES_NETWORK_CONFIG variable from its components
  • @cabutlermit made this request as it was easier to provide the components of this config on their own (as opposed to a formatted string); the following environment variables are now provided: ALMA_SAP_INVOICES_ECS_GROUPS and ALMA_SAP_INVOICES_ECS_SUBNETS.
  • Also note that I removed the @property methods for environment variables. Remember that the original purpose of these now-deprecated methods was to load config vars into Flask's app.config model. Since we are not actually doing this, I opted to return webapp.config.Config to our standard format.

It's also worth noting the following discovery: "Expired" CloudWatch logs are still findable by boto3.CloudWatchLogs.Client.get_log_events; the client will return a successful 200 response but an empty list. This necessitated a few updates to the "status" page of the SAP Invoices web app. The "status" page displays the following results:

0. When page first loads.
   

1. Valid task ID and CloudWatch logs are not expired
   

2. Valid task ID and CloudWatch logs are expired
   

3. Invalid task ID (leads to error page)
   

How can a reviewer manually see the effects of these changes?

It is recommended that the changes be reviewed by commit! There aren't a lot of lines changed from these updates, but a lot of files did change.

Addition of confirmation page
Do one of the following:

1. Review screenshot(s):
   

2. Visit https://sapinvoices.dev1.mitlibrary.net/.

   • From the homepage, select Process invoices
   • Select to Execute a final run.

---

Improved runtime
This 1st screenshot shows the runtime of the request for retrieving logs from CloudWatch logs for an old task (i.e., not within the last hour), which is done via the data route before the changes:

This 2nd screenshot shows the runtime of the request ... for the same task ID after the changes:

Note that in the 2nd screenshot, the time is not increasing with each call to the data route.

---

Env variables

1. This unit test should be sufficient!

Includes new or updated dependencies?

NO

Changes expectations for external applications?

NO

What are the relevant tickets?

• https://mitlibraries.atlassian.net/browse/IN-1052
• https://mitlibraries.atlassian.net/browse/IN-1057

Developer

• All new ENV is documented in README
• All new ENV has been added to staging and production environments
• All related Jira tickets are linked in commit message(s)
• Stakeholder approval has been confirmed (or is not needed)

Code Reviewer(s)

• The commit message is clear and follows our guidelines (not just this PR message)
• There are appropriate tests covering any new functionality
• The provided documentation is sufficient for understanding any new functionality introduced
• Any manual tests have been performed and verified
• New dependencies are appropriate or there were no changes

[coveralls]

Pull Request Test Coverage Report for Build 10792977483

Details

• 21 of 42 (50.0%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-3.5%) to 87.106%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
webapp/config.py	5	7	71.43%
webapp/utils/init.py	0	7	0.0%
webapp/app.py	7	19	36.84%

Totals
Change from base Build 10515398903:	-3.5%
Covered Lines:	304
Relevant Lines:	349

---

💛 - Coveralls

[ghukill]

Overall, looking good. I particularly appreciate the detailed PR notes and examples.

Left a couple questions that likely could be resolved later, or not at all.

I did leave one question which prompted this "Request Changes" review, as it seems like it could be a bug? Unless I'm not understanding it?

[ehanson8]

Good changes and the context in the PR description is greatly appreciated as always! I'm also curious about the issues @ghukill raised so just commenting for now

[ehanson8]

Looks good!

[ghukill]

Looking great to me.

I left a comment around the get_task_status_and_logs method, as I did find it a little hard to follow, but I'm okay approving without changes there; it's mostly just stylistic / readability, and could be a personal preference.