ci: Post comment after e2e smoke

[Cal-L]

Description

The purpose of these changes is to add a Github check for checking E2E smoke test run status (for PRs that have Run Smoke E2E label applied). Under the hood, the action does the following:

• Take the latest Bitrise comment, which is posted after the Run Smoke E2E label is applied
• Extract the commit hash from the comment body
• Compare the hash against a list of commits
• The list of commits to check against - Starting with the latest commit and moving backwards, it includes all merge from main (from pressing the update with main button) up to the latest non-merge from main commit (a commit that the contributor pushed up)
• If the comment commit hash does not match any from the list, the check fails since there is no comment associated with the list of commits
• If the comment commit hash does match something from the list, the action checks the build status listed in the comment
• If it's pending or failed, then the action fails and the PR is blocked from merge. If it's a success status, then the action passes and the PR is unblocked for merge

Related issues

Fixes: https://github.com/MetaMask/mobile-planning/issues/1552

Manual testing steps

Scenario Check Bitrise E2E status will fail if Bitrise status comment with last non-merge commit hash either doesn't exist, is pending, or failed

• GIVEN The Bitrise E2E status check runs
• AND The Run Smoke E2E label is applied
• AND A Bitrise status comment with last non-merge commit hash either doesn't exist, is pending, or failed
• THEN the Bitrise E2E status check will fail

Scenario Check Bitrise E2E status will pass if Bitrise E2E smoke comment with last non-merge commit hash is posted as a successful build

• GIVEN The Bitrise E2E status check runs
• AND The Run Smoke E2E label is applied
• AND A Bitrise status comment with last non-merge commit hash shows passed
• THEN the E2E comment check will pass and PR merge should be unblocked

Scenario Check Bitrise E2E status will pass if E2E smoke label is applied

• GIVEN The Bitrise E2E status check runs
• AND The Run E2E Smoke label is not applied to the PR
• THEN the E2E comment check will pass and PR merge should be unblocked

Screenshots/Recordings

Before

After

Check fails when comment shows pending
https://github.com/MetaMask/metamask-mobile/assets/10508597/01be00e3-a0e8-4c14-96eb-e884d9829507

Check fails since no Bitrise comment exists with latest commit
https://github.com/MetaMask/metamask-mobile/assets/10508597/ab0ac40d-322f-4f1f-9bff-ee2cc3ef436e

Check passes with Bitrise comment showing pass on latest commit
https://github.com/MetaMask/metamask-mobile/assets/10508597/9ffbd0c5-4b3b-426d-9396-cfb271da8422

Check passes with Bitrise comment showing pass on latest non-main merge commit
https://github.com/MetaMask/metamask-mobile/assets/10508597/415fb89c-a873-4a4f-86c5-afc06c0d6a90

Comment shows failed when Bitrise build fails. Check will also detect this and fail.
https://github.com/MetaMask/metamask-mobile/assets/10508597/aee06b95-06b8-4861-8611-d2cdda219fd4

Check passes with Bitrise comment showing pass on a later merge from main commit
https://github.com/MetaMask/metamask-mobile/assets/10508597/dacc0c99-e2a4-434c-be7a-00e63cb34c14

Pre-merge author checklist

• I’ve followed MetaMask Coding Standards.
• I've clearly explained what problem this PR is solving and how it is solved.
• I've linked related issues
• I've included manual testing steps
• I've included screenshots/recordings if applicable
• I’ve included tests if applicable
• I’ve documented my code using JSDoc format if applicable
• I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.
• I’ve properly set the pull request status:
  • In case it's not yet "ready for review", I've set it to "draft".
  • In case it's "ready for review", I've changed it from "draft" to "non-draft".

Pre-merge reviewer checklist

• I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
• I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

[github-actions]

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.60%. Comparing base (224f737) to head (af333f3).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8495   +/-   ##
=======================================
  Coverage   41.60%   41.60%           
=======================================
  Files        1267     1267           
  Lines       30722    30722           
  Branches     3062     3062           
=======================================
  Hits        12781    12781           
  Misses      17171    17171           
  Partials      770      770           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Cal-L]

Can you explain the issue that it fixes or point to a GH issue?

Updated description to describe behavior - #8495 (comment)
Issue is connected to ticket - https://github.com/MetaMask/mobile-planning/issues/1552

[github-actions]

Bitrise

✅✅✅ pr_smoke_e2e_pipeline passed on Bitrise! ✅✅✅

Commit hash: 6036a02
Build link: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/dee376b7-aec3-4afa-8550-b96c5e1ccf82

Note

• You can kick off another pr_smoke_e2e_pipeline on Bitrise by removing and re-applying the Run Smoke E2E label on the pull request

[github-actions]

Bitrise

✅✅✅ pr_smoke_e2e_pipeline passed on Bitrise! ✅✅✅

Commit hash: 1c3435f
Build link: https://app.bitrise.io/app/be69d4368ee7e86d/pipelines/8e9a0f82-0ee6-440f-8d87-93f818d1ed7b

Note

• You can kick off another pr_smoke_e2e_pipeline on Bitrise by removing and re-applying the Run Smoke E2E label on the pull request

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

[cortisiko]

This is massive! Good stuff Cal!!! 🌮 🌮 🌮 🌮