refactor: E2e Smoke Tests to Execute Across Multiple Virtual Machines on Bitrise

[Andepande]

Description

This pull request brings improvements to the smoke test execution process on Bitrise, leading to parallel test runs across multiple virtual machines. These enhancements aim to:

• Reduced Run Duration: Cut down the time required for e2e runs on Bitrise by executing smoke tests concurrently across various virtual machines.
• Simplified Fault Identification: By segregating different areas, it becomes simpler to identify test failures within their respective functional domains.
• Workflow Refinement: By isolating distinct areas, the necessity to retry workflows decreases, concentrating solely on specific areas where tests failed.

Changes Introduced

• Implemented parallel test runs across multiple virtual machines to optimise the smoke test execution process on Bitrise.
• Introduced new smoke tags (smokeConfirmations, smokeSwaps, smokeCore) to categorise and streamline testing efforts targeting specific functional areas.

These enhancements will significantly contribute to improving test efficiency, ensuring a more comprehensive evaluation of distinct functionalities via targeted smoke tests.

Observations for Discussion::

Bitrise Pipeline with Three Workers on Detox

• Credits Usage: ~622 credits
• Run Time: ~55 minutes
• Pipeline Link

2. Pipeline Utilising Two Workers
   • Credits Usage: ~556 credits
   • Run Time: ~45 minutes
   • Pipeline Link

If we reduce the number of parallel workers used to run tests in detox, builds may take longer and utilise more credits, even with optimised utilisation of virtual machines.

• Before

  After

  Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've clearly explained what problem this PR is solving and how it is solved.
  • I've linked related issues
  • I've included manual testing steps
  • I've included screenshots/recordings if applicable
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.
  • I’ve properly set the pull request status:
    • In case it's not yet "ready for review", I've set it to "draft".
    • In case it's "ready for review", I've changed it from "draft" to "non-draft".

  Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (e16c10f) 39.56% compared to head (e46de83) 39.57%.
Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8083   +/-   ##
=======================================
  Coverage   39.56%   39.57%           
=======================================
  Files        1231     1232    +1     
  Lines       29821    29825    +4     
  Branches     2841     2841           
=======================================
+ Hits        11800    11803    +3     
- Misses      17327    17328    +1     
  Partials      694      694           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cortisiko]

small comments

[sethkfman]

LGTM

[sonarcloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud