Run E2E tests only on prod deployments

[devksingh4]

Summary by CodeRabbit

• Chores
  • Restructured CI/CD deployment workflows with enhanced testing phases (unit tests and end-to-end tests now separated).
  • Improved QA build process and artifact management.
  • Refined post-deployment testing with live integration testing.
  • Streamlined deployment dependencies and workflow orchestration.

[coderabbitai]

Walkthrough

GitHub Actions workflows were refactored to improve test organization and deployment pipeline structure. Test jobs were split into separate unit and end-to-end test stages. QA build artifacts and testing steps were introduced into the production deployment workflow. Post-deploy testing logic was consolidated in the Makefile.

Changes

Cohort / File(s)	Summary
GitHub Actions workflow updates .github/workflows/deploy-prod.yml	Renamed test job to test-unit; introduced new test-e2e job with full E2E workflow including checkout, Node setup, Terraform setup, Yarn cache, artifact download, AWS credentials, QA deployment, health checks, and E2E tests; added QA build flow under build job with artifact uploads; updated deploy-prod job dependencies to require both test-unit and test-e2e.
GitHub Actions workflow updates .github/workflows/deploy-qa.yml	Removed AWS credentials configuration step from deploy-test-dev job; replaced "Run post-deploy testing (Live and E2E)" step with "Run live integration testing" step, changing command from make test_post_deploy -j 2 to make test_live_integration; removed PLAYWRIGHT_USERNAME and PLAYWRIGHT_PASSWORD environment variables.
Build configuration Makefile	Removed test_post_deploy target definition that previously triggered test_live_integration and test_e2e targets.

Sequence Diagram(s)

sequenceDiagram
    participant GitHub as GitHub Actions
    participant checkout as Checkout
    participant build as Build
    participant test_unit as test-unit Job
    participant test_e2e as test-e2e Job
    participant deploy as deploy-prod Job
    
    GitHub->>checkout: Trigger on push
    checkout->>build: Build & Build QA
    build-->>GitHub: Upload artifacts
    
    par Parallel Test Jobs
        GitHub->>test_unit: Run unit tests
        test_unit-->>GitHub: ✓ Complete
    and
        GitHub->>test_e2e: Run E2E tests
        test_e2e->>test_e2e: Deploy to QA<br/>Run health checks<br/>Run E2E tests
        test_e2e-->>GitHub: ✓ Complete
    end
    
    GitHub->>deploy: Deploy to prod<br/>(requires test-unit ✓ + test-e2e ✓)
    deploy-->>GitHub: ✓ Deployment complete

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Review the new test-e2e job structure and its integration with QA deployment workflow
• Verify deploy-prod dependencies are correctly updated to require both test-unit and test-e2e before production deployment
• Confirm the Makefile target removal doesn't break any existing local development workflows or other CI/CD references

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main change: restructuring the deployment workflow to run E2E tests exclusively during production deployments rather than post-deploy.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch dsingh14/e2e-only-prod-deploy

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to data retention organization setting

📥 Commits

Reviewing files that changed from the base of the PR and between 57e8a32 and fea82be.

📒 Files selected for processing (3)

• .github/workflows/deploy-prod.yml (4 hunks)
• .github/workflows/deploy-qa.yml (1 hunks)
• Makefile (0 hunks)

💤 Files with no reviewable changes (1)

• Makefile

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: Run Unit Tests
• GitHub Check: Build Application

🔇 Additional comments (5)

.github/workflows/deploy-prod.yml (4)

11-42: Approved: Job rename from test to test-unit.

Renaming to test-unit clarifies this job's scope and aligns with the new separation of concerns (unit vs. E2E tests).

---

91-107: Approved: QA build artifact addition.

Added QA build step that cleans and rebuilds with QA environment, then uploads as build-qa. This properly separates QA-destined artifacts from prod-destined artifacts (build-prod), allowing E2E tests to run on appropriate QA-configured code without affecting the prod deployment build.

---

183-184: Approved: Updated deploy-prod dependencies to require both test-unit and test-e2e.

Updated dependencies ensure both unit tests and E2E tests must pass before production deployment. This creates a proper quality gate: code correctness (unit tests) + integration/E2E verification (on QA environment) → prod deployment. Correct sequencing: test-unit and test-e2e run in parallel (both depend on build), then deploy-prod waits for both.

---

108-170: Approved: New test-e2e job introduces E2E testing gate before prod deployment.

New job structure correctly:

• Depends on build (has access to build-qa artifact)
• Uses "AWS QA" environment (appropriate for testing on QA)
• Deploys to QA, runs health checks, executes E2E tests with Playwright credentials
• Blocks deploy-prod if E2E tests fail

All required Makefile targets (deploy_qa, dev_health_check, test_e2e) exist and are properly defined. Artifact naming is consistent: test-e2e downloads build-qa, and deploy-prod downloads build-prod.

This aligns with the PR objective: E2E tests now gate production deployments.

.github/workflows/deploy-qa.yml (1)

145-148: test_live_integration target verified; AWS credentials claim in review is inaccurate.

The test_live_integration Makefile target exists (line 72) and the environment variable changes are correct—Playwright credentials were appropriately removed from this test step since E2E runs only on production.

However, the review comment's assertion that "AWS credentials configuration was also removed from this job" is inaccurate. AWS credentials remain in deploy-test-dev (line 130: aws-actions/configure-aws-credentials@v5), which is correct and necessary for the make deploy_qa step to function. The changes at lines 145-148 are sound.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

💰 Infracost report

Monthly estimate generated

_{This comment will be updated when code changes.}