MNTOR-3210, MNTOR-3211: E2E cron failure alerts and warning fixes.

[rafeerahman]

References:

Jira: MNTOR-3210, MNTOR-3211

Description

Adds an alert sent to #monitor-alerts in slack when the e2e cron fails. Fix for flaky tests and made retries for tests a 1 instead of 2.

Screenshot (if applicable)

Not applicable.

How to test

• Run this in the GHA tab against this branch.

Test pass except paypal ones

https://github.com/mozilla/blurts-server/actions/runs/9408352400

Checklist (Definition of Done)

• Localization strings (if needed) have been added.
• Commits in this PR are minimal and have descriptive commit messages.
• I've added or updated the relevant sections in readme and/or code comments
• I've added a unit test to test for potential regressions of this bug.
• If this PR implements a feature flag or experimentation, the Ship Behind Feature Flag status in Jira has been set
• Product Owner accepted the User Story (demo of functionality completed) or waived the privilege.
• All acceptance criteria are met.
• Jira ticket has been updated (if needed) to match changes made during the development process.
• Jira ticket has been updated (if needed) with suggestions for QA when this PR is deployed to stage.

[github-actions]

Preview URL 🚀 : https://blurts-server-pr-4640-mgjlpikfea-uk.a.run.app

[pdehaan]

Do we need a suite timeout, or could we remove it? (Random q: not PR blocking)

[rafeerahman]

Seems like we do because tests are stuck idling when they reach certain states. See https://github.com/mozilla/blurts-server/actions/runs/9421692534, the application error state on the webpage causes the tests to get stuck? Not sure why the test and job keeps running.

Also not sure why thats happening because we have a test timeout limit here, https://github.com/mozilla/blurts-server/blob/main/playwright.config.js#L27-L28.

[github-actions]

Cleanup completed - database 'blurts-server-pr-4640' destroyed, cloud run service 'blurts-server-pr-4640' destroyed