Run automation tests on release (#4775)

* Test PR - running playwright tests after deployment * Test PR - running playwright tests after deployment * Rerun job * open draft PR * Add check * Add check * Add check * Add check * Add check * Add check * Add check * Add check * Update check * Update check * Update check * Update check * Update check * Update check * Update check * Update check * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Move create check to seperate file * Check if PR exist * Check if PR exist * Check if PR exist * Check if PR exist * restore data from snapshot * Allow user without app permission access apps view (#4738) * Allow user without app permission access apps view * Use css subgrid instead of multiple map loops * Add protection to queries and mutation that use App fragments * Allow user to see manage app view * Refactor app permission check to hook * Protect routes * shouldShowInstalledApps * Refactor AppListPage, write test for loading, empty state with checking manage_app perm * Change manage button label depend on manage_apps * Improve typing * Extract messages * ButtonWithTooltip * Refactor HeaderOptions, block action when no manage_apps * Improve buttons * Add changset * Adjust singlePermission tests * Remove required hasManagedAppsPermission when default * Improve action icons * Improve naming and refactor AppListCardInstallButton * Improve changset * Improve tests * [E2E] Fixing flaky e2e tests (#4751) * adding extra assertion to translation tests * updating assertion in attribute tests * modifying assertions for gift card tests * [E2E] Fixing another batch of flaky tests (#4756) * updating tests for extra assertions and adding skip to App test * updating tests for extra assertions * updating tests for extra assertions and temporarily commenting not working assertions * adding PW test for readonly Apps access (#4755) * [QA] Correcting e2e test case ids (#4764) * correcting test case ids * fixing flaky test * Update .changeset/serious-books-wave.md Co-authored-by: Jonatan Witoszek <jonatanwitoszek@gmail.com> --------- Co-authored-by: Jonatan Witoszek <jonatanwitoszek@gmail.com> * New github action for gathering metrics (#4762) * New github action for metrics gathering * metrics file name containing date * search query repo update * Update .github/workflows/repo-metrics.yml Co-authored-by: Mikolaj Stempniewicz <mikolaj@stempniewicz.net> * Update repo-metrics.yml * Update repo-metrics.yml * Update .github/workflows/repo-metrics.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> --------- Co-authored-by: Mikolaj Stempniewicz <mikolaj@stempniewicz.net> Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * Gathering pull requests metrics (#4766) * fix workflow dispatch trigger * fix workflow dispatch trigger * metrics for pr * removed date from file name * on push added * wip * date in issue name, workflow dispatch trigger * Create long-keys-compare.md * add siteSettings basic test (#4759) * Remove babel (#4711) * Remove babel * Add changeset * Fix showing empty tooltip when no content by bumping macaw-ui to newest (#4771) * Bump macaw * Add changeset * [E2E] Tests for page types (#4765) * add changeset * add test data * add tests and update data-test-id * code review fixes * spaces * adding test for permission groups (#4758) * fixing flaky tests (#4772) * Send tests results when running manually (#4737) * Send tests results when running manually * Send tests results when running manually * Send tests results when running manually * Check if it is working * Fix init testmo * Fix submit threads * Fix working directory when sending results to testmo * Add name and tag * Add name and tag * Fix run name * Add reports when tests run by cron * Fix messages skip ci * Fix messages [skip ci] * Sending playwright tests results to slack and testmo * Update .github/workflows/tests-nightly.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * changes fter review * fix changeset --------- Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * Fix version * Run tests * Run tests on correct branch * Update status check if job failed * Update status check if job failed * Manual order transaction refund (#4774) * Order manual transaction view init * Connect RHF * Add sending mutation * Refactor, add intl, fix form validation * extract messages * Add changeset * Refactor transactions tiles * Improve loading and disable state, fix radio group control * Hadle no transactions * Write tests * extract messages * Refactor fragments * Move mutation to form component * Fix test * fixing flaky tests (#4772) (#4778) * Show all transaction in manual refund view (#4779) * Order manual transaction view init * Connect RHF * Add sending mutation * Refactor, add intl, fix form validation * extract messages * Add changeset * Refactor transactions tiles * Improve loading and disable state, fix radio group control * Hadle no transactions * Write tests * extract messages * Refactor fragments * Move mutation to form component * Fix test * Show all transactions * Add changeset * Update tests * Extract messages * Fix typo * Improve tooltip * [E2E] [QA] Optimizing playwright setup and playwright.config.ts files (#4741) * adding condition to not create json file if it does exist * adding cache * adding cache * updating yml files * fixing GH action * fixing flaky test * adding randomization to tests execution * reverting * fixing flaky tests * updating other gh workflows consisting of pw test run * Fix version * Fix checkout * [E2E] Tests for product types (#4757) * initial commit with changelog * add test for update product type * add test to delete several product types * add test to remove product type with asigned product * fix numering and assertion after removing types * typo * update test data * add assertions * remove stripe and emails plugins app replacement message (#4777) * switch version to 3.19 * Fix submit results * Fix update check * Fix update check * Fix update check * Fix script * Fix status * test trigger run tests on release job * last test * last test * [E2E] Use dummy app in delete app test (#4789) * replace app * update test data * fixing flaky gift cards tests (#4785) * Fix workflow name * Fix names * Fix names * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * do not run cypress if higher version * test for lower versions run only cypress * test for lower versions run only cypress * test for lower versions run only cypress * test for lower versions run only cypress * test for lower versions run only cypress * test for lower versions run only cypress * test for lower versions run only cypress * test for lower versions run only cypress * test for higher versions - playwright * Delete test workflow * Add changeset * Update .github/actions/checks/create-pull-request-check/action.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * Update .github/workflows/prepare-release.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * use jq, don't use string interpolation * Check if still works after review changes * Fix crete check * Fix crete check * Fix crete check * Remove leftovers * Fix head sha * Fix head sha * Fix head sha * Fix head sha * cleanup after tests * Update .github/actions/checks/create-pull-request-check/action.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * tests if works after review changes * tests if works after review changes * Remove testing workflow * Update .github/actions/checks/update-pull-request-check/action.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * Update .github/workflows/run-tests-on-release.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * Update .github/actions/prepare-api-variables/action.yml Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> * merge deploy and prepare release workflows --------- Co-authored-by: Paweł Chyła <chyla1988@gmail.com> Co-authored-by: Renata <renata.gajzlerowicz@gmail.com> Co-authored-by: Jonatan Witoszek <jonatanwitoszek@gmail.com> Co-authored-by: wojteknowacki <124166231+wojteknowacki@users.noreply.github.com> Co-authored-by: Mikolaj Stempniewicz <mikolaj@stempniewicz.net> Co-authored-by: Mikail <6186720+NyanKiyoshi@users.noreply.github.com> Co-authored-by: M.Graczyk <michalina.graczyk@saleor.io> Co-authored-by: Anna Szczęch <30683248+szczecha@users.noreply.github.com> Co-authored-by: Lukasz Ostrowski <lukasz.ostrowski@saleor.io>
saleor · May 7, 2024 · 0b19eaa · 0b19eaa
1 parent a6ed457
commit 0b19eaa
Show file tree

Hide file tree

Showing 10 changed files with 557 additions and 150 deletions.
diff --git a/.changeset/hip-clocks-allow.md b/.changeset/hip-clocks-allow.md
@@ -0,0 +1,5 @@
+---
+"saleor-dashboard": patch
+---
+
+Run cypress or playwright tests on release
diff --git a/.github/actions/checks/create-pull-request-check/action.yml b/.github/actions/checks/create-pull-request-check/action.yml
@@ -0,0 +1,77 @@
+name: Create-check
+description: "Creates check for PR"
+inputs:
+  PR_HEAD:
+    description: "Head of PR on which create check"
+    required: true
+  CHECK_NAME:
+    description: "Name of check"
+    required: true
+  STATUS:
+    description: "Status of check"
+    required: true
+  TITLE: 
+    description: "Title of check"
+    required: true
+  DETAILS_URL: 
+    description: "Details url"
+    required: true
+
+outputs:
+  CHECK_ID:
+    description: "CHECK_ID"
+    value: ${{ steps.create-check.outputs.check_id }}
+
+runs:
+  using: composite
+  steps:
+    - name: Get Pull Request head_sha
+      shell: bash
+      env:
+        GH_TOKEN: ${{ github.token }}
+        PR_HEAD: ${{ inputs.PR_HEAD }}
+      run: |
+        gh api \
+        -H "Accept: application/vnd.github+json" \
+        -H "X-GitHub-Api-Version: 2022-11-28" \
+        "/repos/saleor/saleor-dashboard/pulls?head=saleor:${PR_HEAD}" > data.json
+        head_sha_with_quotes=$(jq '.[0] | .head.sha' data.json)
+        echo "PULL_REQUEST_HEAD_SHA=${head_sha_with_quotes//\"/}"  >> "$GITHUB_ENV"
+
+    - name: Create check
+      if: ${{ env.PULL_REQUEST_HEAD_SHA }}
+      shell: bash
+      id: create-check
+      env:
+        GH_TOKEN: ${{ github.token }}
+        NAME: ${{ inputs.CHECK_NAME }}
+        STATUS: ${{ inputs.STATUS }}
+        TITLE: ${{ inputs.TITLE }}
+      run: |
+        payload=$(\
+        jq --null-input \
+        --arg name "$NAME" \
+        --arg head_sha "$PULL_REQUEST_HEAD_SHA" \
+        --arg status "$STATUS" \
+        --arg title "$TITLE" \
+        --arg details_url "$DETAILS_URL" \
+        '{
+          "name": $name, 
+          "head_sha": $head_sha, 
+          "status": $status, 
+          "output": {
+            "title": $title,
+            "summary": "",
+            "text": "",
+            "details_url": $details_url
+         }
+        }'
+        )
+
+        check_id=$(curl -L \
+        -X POST \
+        -H "Accept: application/vnd.github+json" \
+        -H "Authorization: Bearer $GH_TOKEN" \
+        -H "X-GitHub-Api-Version: 2022-11-28" \
+        https://api.github.com/repos/saleor/saleor-dashboard/check-runs \
+        -d "$payload" | jq -r '.id') ; echo "check_id=$check_id" >> "$GITHUB_OUTPUT"
diff --git a/.github/actions/checks/update-pull-request-check/action.yml b/.github/actions/checks/update-pull-request-check/action.yml
@@ -0,0 +1,69 @@
+name: Create-check
+description: "Creates check for PR"
+inputs:
+  CHECK_ID:
+    description: "Check id"
+    required: true
+  CHECK_NAME:
+    description: "Name of check - must be the same here and in create check"
+    required: true
+  STATUS:
+    description: "Status of check"
+    required: true
+  CONCLUSION:
+    description: "Conclusion of check, required if status completed"
+    required: false
+  TITLE: 
+    description: "Title of check"
+    required: true
+  SUMMARY:
+    description: "Summary of check"
+    required: true
+  DETAILS_URL:
+    description: "Url with details to check"
+    required: true
+
+runs:
+  using: composite
+  steps:
+    - name: Update check
+      shell: bash
+      env:
+        GH_TOKEN: ${{ github.token }}
+        CHECK_ID: ${{ inputs.CHECK_ID }}
+        NAME: ${{ inputs.CHECK_NAME }}
+        STATUS: ${{ inputs.STATUS }}
+        CONCLUSION: ${{inputs.CONCLUSION}}
+        TITLE: ${{ inputs.TITLE }}
+        SUMMARY: ${{ inputs.SUMMARY }}
+        DETAILS_URL: ${{ inputs.DETAILS_URL }}
+      run: |
+      
+        payload=$(\
+        jq --null-input \
+        --arg name "$NAME" \
+        --arg status "$STATUS" \
+        --arg conclusion "$CONCLUSION" \
+        --arg title "$TITLE" \
+        --arg summary "$SUMMARY" \
+        --arg details_url "$DETAILS_URL" \
+        '{
+          "name": $name, 
+          "status": $status, 
+          "conclusion": $conclusion,
+          "output": {
+            "title": $title,
+            "summary": $summary,
+            "text": "",
+            "details_url": $details_url
+         }
+        }'
+        )
+
+        curl -L \
+        -X PATCH \
+        -H "Accept: application/vnd.github+json" \
+        -H "Authorization: Bearer $GH_TOKEN" \
+        -H "X-GitHub-Api-Version: 2022-11-28" \
+        "https://api.github.com/repos/saleor/saleor-dashboard/check-runs/$CHECK_ID" \
+        -d "$payload"
diff --git a/.github/actions/prepare-api-variables/action.yml b/.github/actions/prepare-api-variables/action.yml
@@ -5,6 +5,9 @@ inputs:
   MODE:
     description: "The mode of running tests (pull-request, release, main)"
     required: true
+  VERSION:
+    description: "Version if you want to run from different branch"
+    required: false
 
 outputs:
   BASE_URL:
@@ -36,6 +39,7 @@ runs:
         MODE: ${{ inputs.MODE }}
         PULL_REQUEST_NUMBER: ${{ github.event.number }}
         PREFIX: pr-
+
       run: |
         if [[ "$MODE" == 'pull-request' ]]; then
           echo "BASE_URL=https://${PREFIX}${PULL_REQUEST_NUMBER}.dashboard.saleor.rocks" >> $GITHUB_OUTPUT
@@ -73,6 +77,19 @@ runs:
           exit 0
         fi
 
+        if [[ "$MODE" == 'from-version' ]]; then
+          VERSION_SLUG=$(echo "${VERSION/./}")
+
+          echo "BASE_URL=https://v${VERSION_SLUG}.staging.saleor.cloud/dashboard/" >> "$GITHUB_OUTPUT"
+          echo "API_URL=https://v${VERSION_SLUG}.staging.saleor.cloud/graphql/" >> "$GITHUB_OUTPUT"
+          echo "BASE_URL_DOMAIN=v${VERSION_SLUG}.staging.saleor.cloud" >> "$GITHUB_OUTPUT"
+          echo "POOL_NAME=${VERSION} Staging" >> "$GITHUB_OUTPUT"
+          echo "POOL_INSTANCE=https://v${VERSION_SLUG}.staging.saleor.cloud/" >> "$GITHUB_OUTPUT"
+          echo "BACKUP_NAMESPACE=snapshot-automation-tests-${VERSION}" >> "$GITHUB_OUTPUT"
+
+          exit 0
+        fi
+
         echo "Unknown mode: $MODE" >&2
         exit 1
 
diff --git a/.github/actions/prepare-tests-variables/action.yml b/.github/actions/prepare-tests-variables/action.yml
@@ -7,6 +7,10 @@ inputs:
   CLI_TOKEN:
     description: "Saleor cli token"
     required: true
+  VERSION:
+    description: "Version in format X.XX, if run from other branch"
+    required: false
+
 outputs:
   BASE_URL:
     description: "Dashboard base url"
@@ -43,6 +47,7 @@ runs:
       uses: ./.github/actions/prepare-api-variables
       with:
         MODE: ${{ inputs.MODE }}
+        VERSION: ${{ inputs.VERSION || ''}} 
 
     - name: Generate backup
       id: backup

diff --git a/.github/workflows/deploy-staging.yaml → ...s/deploy-staging-and-prepare-release.yaml b/.github/workflows/deploy-staging.yaml → ...s/deploy-staging-and-prepare-release.yaml
@@ -1,26 +1,47 @@
 name: Deploy Dashboard to staging
 on:
   push:
-    tags:
-      # Matches stable and pre-releases
-      - "[0-9]+.[0-9]+.[0-9]+*"
     branches:
       # Matches release branches
       - "[0-9]+.[0-9]+"
-  workflow_dispatch:
-    inputs:
-      git_ref:
-        description: Git ref (tag, branch or commit hash) to deploy
-        required: true
-      trigger_release:
-        type: boolean
-        description: "Trigger release"
-        required: true
-        default: false
+
+concurrency: ${{ github.workflow }}-${{ github.ref }}
 
 jobs:
+  prepare-release:
+    name: Prepare release with Changesets
+    runs-on: ubuntu-22.04
+    outputs:
+      hasChangesets: ${{ steps.changesets.outputs.hasChangesets }}
+    permissions:
+      contents: write
+      pull-requests: write
+    steps:
+      - name: Checkout Repo
+        uses: actions/checkout@v4
+      - name: Setup Node
+        uses: actions/setup-node@v3
+        with:
+          node-version-file: ".nvmrc"
+      - name: Install Dependencies
+        run: npm ci
+      - name: Publishing version
+        uses: changesets/action@d89c1de63c7f28ac47ec85ed395f5f1d045d4697 # v1.4.4
+        id: changesets
+        with:
+          title: "Prepare release ${{ github.ref_name }}"
+          commit: "Release ${{ github.ref_name }}"
+          publish: "npm run release"
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
   build:
+    needs: prepare-release
+    if: always()
     runs-on: ubuntu-20.04
+    outputs:
+      MINOR_VERSION: ${{ env.MINOR_VERSION }}
+      CUSTOM_VERSION: ${{ env.CUSTOM_VERSION || env.VERSION }}
     env:
       API_URI: /graphql/
       APP_MOUNT_URI: /dashboard/
@@ -36,7 +57,7 @@ jobs:
       POSTHOG_KEY: ${{ secrets.POSTHOG_KEY }}
       POSTHOG_HOST: ${{ secrets.POSTHOG_HOST }}
     steps:
-      - uses: actions/checkout@v2
+      - uses: actions/checkout@v4
         with:
           ref: ${{ env.VERSION }}
 
@@ -45,24 +66,38 @@ jobs:
         run: |
           set -x
           environment=$(echo $VERSION | sed -n 's#\([0-9]\+\).\([0-9]\+\).*#saleor-staging-v\1\2#p')
-          echo "ENVIRONMENT=${environment}" >> $GITHUB_ENV
+          echo "ENVIRONMENT=${environment}" >> "$GITHUB_ENV"
       
       - name: Set custom version
-        if: ${{ startsWith(github.ref, 'refs/heads/') }}
+        env:
+          VERSION: ${{ github.event.inputs.git_ref || github.ref_name }}
         # Add commit hash to basic version number
         run: |
           set -x
           HASH=$(git rev-parse --short HEAD)
           CURRENT_VERSION=$(jq -r .version package.json)
           RELEASE="${CURRENT_VERSION}-${HASH}"
-          echo "CUSTOM_VERSION=${RELEASE}" >> $GITHUB_ENV
-          echo "SENTRY_RELEASE=${RELEASE}" >> $GITHUB_ENV
+          echo "CUSTOM_VERSION=${RELEASE}" >> "$GITHUB_ENV"
+          echo "SENTRY_RELEASE=${RELEASE}" >> "$GITHUB_ENV"
+          minor_version=$(echo "$VERSION" | sed -n 's#\([0-9]\+\).\([0-9]\+\).*#\1.\2#p')
+          echo "MINOR_VERSION=${minor_version}" >> "$GITHUB_ENV"
 
       - name: Setup Node
         uses: actions/setup-node@v3
         with:
           node-version-file: ".nvmrc"
 
+      - name: Create check in prepare release PR
+        if: needs.prepare-release.outputs.hasChangesets == 'true'
+        id: create-check
+        uses: ./.github/actions/checks/create-pull-request-check
+        with:
+          PR_HEAD: "changeset-release/${{env.MINOR_VERSION}}"
+          CHECK_NAME: "Deploy_staging"
+          STATUS: "in_progress"
+          TITLE: "Deploy staging"
+          DETAILS_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+
       - name: Package
         timeout-minutes: 15
         run: |
@@ -80,6 +115,7 @@ jobs:
           aws-region: us-east-1
 
       - name: Deploy
+        id: deploy
         run: |
           aws s3 sync build/dashboard "s3://${DEPLOYMENT_BUCKET}/${ENVIRONMENT}/static/"
           aws s3 cp build/dashboard/index.html "s3://${DEPLOYMENT_BUCKET}/${ENVIRONMENT}/"
@@ -93,8 +129,31 @@ jobs:
           DEPLOYMENT_BUCKET: ${{ secrets.AWS_CLOUD_STAGING_DEPLOYMENT_BUCKET }}
           CDN_DISTRIBUTIONS: ${{ secrets.AWS_CLOUD_STAGING_CDN_DISTRIBUTIONS }}
 
+      - name: Update check
+        if: always() && ${{ steps.create-check.outputs.CHECK_ID }}
+        uses: ./.github/actions/checks/update-pull-request-check
+        with:
+          CHECK_ID: ${{ steps.create-check.outputs.CHECK_ID }}
+          CHECK_NAME: "Deploy_staging"
+          STATUS: "${{ steps.deploy.outcome }}"
+          CONCLUSION: "${{ steps.deploy.conclusion }}"
+          TITLE: "Deploy staging"
+          SUMMARY: "Deploy finished with status ${{ steps.deploy.outcome }}. To check details: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          DETAILS_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+
+      - name: Notify Slack
+        if: ${{ always() }}
+        env:
+          JOB_DEPLOYMENT_KIND: staging
+          JOB_STATUS: ${{ job.status }}
+          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_CLOUD_DEPLOYMENTS_WEBHOOK_URL }}
+          SLACK_MENTION_GROUP_ID: ${{ secrets.SLACK_DASHBOARD_GROUP_ID }}
+          JOB_TITLE: "Dashboard deployment of ${{ env.CUSTOM_VERSION || env.VERSION }} to ${{ env.ENVIRONMENT }}"
+        run: |
+          python3 ./.github/workflows/notify/notify-slack.py
+
       - name: Prepare Sandbox release pull request
-        if: ${{ startsWith(github.ref, 'refs/tags/') || github.event.inputs.trigger_release == 'true' }}
+        if: needs.prepare-release.outputs.hasChangesets == 'false'
         run: |
           export GITHUB_TOKEN=$( \
             curl --request GET --url ${{ secrets.VAULT_URL}} --header "Authorization: JWT ${{ secrets.VAULT_JWT }}" | jq -r .token \
@@ -109,13 +168,11 @@ jobs:
               }
             }'
 
-      - name: Notify Slack
-        if: ${{ always() }}
-        env:
-          JOB_DEPLOYMENT_KIND: staging
-          JOB_STATUS: ${{ job.status }}
-          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_CLOUD_DEPLOYMENTS_WEBHOOK_URL }}
-          SLACK_MENTION_GROUP_ID: ${{ secrets.SLACK_DASHBOARD_GROUP_ID }}
-          JOB_TITLE: "Dashboard deployment of ${{ env.CUSTOM_VERSION || env.VERSION }} to ${{ env.ENVIRONMENT }}"
-        run: |
-          python3 ./.github/workflows/notify/notify-slack.py
+  run-tests-on-release:
+    needs: [prepare-release, build]
+    if: needs.prepare-release.outputs.hasChangesets == 'true'
+    uses: ./.github/workflows/run-tests-on-release.yml
+    with:
+      VERSION: ${{ needs.build.MINOR_VERSION }} # eg. "3.19"
+      CUSTOM_VERSION: ${{ needs.build.CUSTOM_VERSION }} # eg. "3.19.4-5a41d18"
+    secrets: inherit