Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow running of "Smoke test release" on draft releases #36997

Merged
merged 19 commits into from Mar 8, 2023
Merged

Allow running of "Smoke test release" on draft releases #36997

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
8fef083
Conditionally use authorization token when getting WC Zip download URL.
rodelgc Feb 28, 2023
f3a96be
Exponse error message
rodelgc Feb 28, 2023
3f5a8e4
Handle undefined authorization token more gracefully
rodelgc Feb 28, 2023
2d65274
Specify GH token in workflow
rodelgc Feb 28, 2023
3f26383
Add changelog
rodelgc Feb 28, 2023
1e811c1
Use E2E_GH_TOKEN instead of default github.token
rodelgc Feb 28, 2023
e0369af
Provide GitHub token to other jobs
rodelgc Feb 28, 2023
a6c7f44
Fix getting of tag name from triggered event
rodelgc Mar 2, 2023
7916b15
Use "inputs" context instead
rodelgc Mar 2, 2023
6d905d5
Add release version to workflow name
rodelgc Mar 2, 2023
d1da273
Correct concurrency group
rodelgc Mar 2, 2023
1f2d46a
Merge branch 'trunk' into e2e/release-include-drafts
rodelgc Mar 2, 2023
02d97b2
Fix workflow name
rodelgc Mar 2, 2023
8ee473f
Add --archive option
rodelgc Mar 2, 2023
d5d2ffb
Fix output setting
rodelgc Mar 2, 2023
a91073d
Fix script for verifying asset
rodelgc Mar 2, 2023
3d4cd21
Remove unnecessary 'uses' line
rodelgc Mar 2, 2023
3761267
Remove node 12 deprecation messages
rodelgc Mar 2, 2023
718ef83
Re-add "published" release type
rodelgc Mar 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
53 changes: 27 additions & 26 deletions .github/workflows/smoke-test-release.yml
View file
Open in desktop
@@ -1,14 +1,14 @@
name: Smoke test release
on:
release:
types: [published]
types: [released, prereleased]
Copy link
Contributor Author

@rodelgc rodelgc Mar 2, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Fix]

I noticed in the 7.4.1 and 7.5.0-beta.2 releases that this workflow wasn't triggered after they were made public. I figured it's because these releases may have already been published, and were just updated to be a pre-release or a release later on. Therefore, the published release type wasn't triggered because they were already in a published state to begin with.

Setting the release type to released and prereleased I think would be more appropriate. But, is it, @woocommerce/atlas? Thanks 😃

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured it's because these releases may have already been published, and were just updated to be a pre-release or a release later on.

I'm not sure that this is the case, as I always make sure to select one of those two before publishing a release... but if released and prereleased trigger this workflow as expected, I think this should be okay.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw this in the release event documentation:

Note: The prereleased type will not trigger for pre-releases published from draft releases, but the published type will trigger. If you want a workflow to run when stable and pre-releases publish, subscribe to published instead of released and prereleased.

Might this conflict with this change? 🤔

workflow_dispatch:
inputs:
tag:
description: 'WooCommerce Release Tag'
required: true
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.event.release.tag_name || inputs.tag }}
Copy link
Contributor Author

@rodelgc rodelgc Mar 2, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Fix]

I noticed that the concurrency group was erroneous because the main intention of this workflow is to group the release smoke tests based on the release version (tag) they're testing, not based on github.ref. Leaving the concurrency group to use github.ref will cause a smoke test for release 7.5.0, for example, to unexpectedly cancel the smoke test of another release version.

I fixed it here so that the release smoke tests are grouped by release tag, (whether this tag came from the release or workflow_dispatch event) thereby preventing concurrent release smoke tests on different tags/versions from cancelling each other.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! 🙌

cancel-in-progress: true
permissions: {}
env:
Expand All @@ -23,31 +23,31 @@ jobs:
contents: read
runs-on: ubuntu-20.04
outputs:
tag: ${{ steps.tag.outputs.result }}
tag: ${{ steps.get-tag.outputs.tag }}
created: ${{ steps.created-at.outputs.created }}
steps:
- name: Validate tag
if: ${{ github.event_name == 'workflow_dispatch' }}
env:
GH_TOKEN: ${{ github.token }}
run: gh release view "${{ github.event.inputs.tag }}" --repo=woocommerce/woocommerce
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
Copy link
Contributor Author

@rodelgc rodelgc Mar 2, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As noted in the PR description, I replaced the default github.token with E2E_GH_TOKEN. You would see similar diffs like this throughout this PR.

run: gh release view "${{ inputs.tag }}" --repo=woocommerce/woocommerce

- name: Get tag
uses: actions/github-script@v6
id: tag
with:
result-encoding: string
script: |
console.log( "${{ github.event_name }}" );
return "${{ github.event.release.tag_name }}" || "${{ github.event.inputs.tag }}"
- name: Get tag from triggered event
id: get-tag
env:
RELEASE_TAG: ${{ github.event.release.tag_name || inputs.tag }}
run: |
echo "Triggered event: ${{ github.event_name }}"
echo "Tag from event: $RELEASE_TAG"
echo "tag=$RELEASE_TAG" >> $GITHUB_OUTPUT
Comment on lines -35 to +42
Copy link
Contributor Author

@rodelgc rodelgc Mar 2, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Fix]

The return line was erroneous because the || should be inside ${{ }}. Fixed it here by doing just that. Also removed the use of the actions/github-script action in order to simplify the logic of this step and make it more readable (I hope 🤞) .


- name: Verify woocommerce.zip asset
env:
GITHUB_TOKEN: ${{ github.token }}
RELEASE_TAG: ${{ steps.tag.outputs.result }}
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
RELEASE_TAG: ${{ steps.get-tag.outputs.tag }}
run: |
gh release download $RELEASE_TAG --repo woocommerce/woocommerce
if [[ -f "woocommerce.zip" ]]
ASSET_NAMES=$(gh release view $RELEASE_TAG --repo woocommerce/woocommerce --json assets --jq ".assets[].name")
if [[ $ASSET_NAMES == *"woocommerce.zip"* ]]
then
echo "$RELEASE_TAG has a valid woocommerce.zip asset."
exit 0
Expand All @@ -59,8 +59,8 @@ jobs:
- name: Get 'created-at' of WooCommerce zip
id: created-at
env:
GH_TOKEN: ${{ github.token }}
run: echo "created=$(gh release view ${{ steps.tag.outputs.result }} --json assets --jq .assets[0].createdAt --repo woocommerce/woocommerce)" >> $GITHUB_OUTPUT
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
run: echo "created=$(gh release view ${{ steps.get-tag.outputs.tag }} --json assets --jq .assets[0].createdAt --repo woocommerce/woocommerce)" >> $GITHUB_OUTPUT

e2e-update-wc:
name: Test WooCommerce update
Expand Down Expand Up @@ -93,6 +93,7 @@ jobs:
CUSTOMER_PASSWORD: ${{ secrets.RELEASE_TEST_CUSTOMER_PASSWORD }}
CUSTOMER_USER: ${{ secrets.RELEASE_TEST_CUSTOMER_USER }}
DEFAULT_TIMEOUT_OVERRIDE: 120000
GITHUB_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
UPDATE_WC: ${{ needs.get-tag.outputs.tag }}
run: |
pnpm exec playwright test \
Expand All @@ -106,7 +107,7 @@ jobs:

- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1
uses: aws-actions/configure-aws-credentials@v1-node16
Comment on lines -109 to +110
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Fix]

This fixes the node v12 deprecation messages.

with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
Expand Down Expand Up @@ -183,7 +184,7 @@ jobs:

- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1
uses: aws-actions/configure-aws-credentials@v1-node16
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
Expand Down Expand Up @@ -284,7 +285,7 @@ jobs:

- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1
uses: aws-actions/configure-aws-credentials@v1-node16
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
Expand Down Expand Up @@ -393,7 +394,7 @@ jobs:

- name: Download release zip
env:
GITHUB_TOKEN: ${{ github.token }}
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
run: gh release download ${{ needs.get-wp-versions.outputs.tag }} --dir tmp

- name: Replace `plugins/woocommerce` with unzipped woocommerce release build
Expand Down Expand Up @@ -427,7 +428,7 @@ jobs:

- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1
uses: aws-actions/configure-aws-credentials@v1-node16
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
Expand Down Expand Up @@ -567,7 +568,7 @@ jobs:

- name: Download release zip
env:
GITHUB_TOKEN: ${{ github.token }}
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
run: gh release download ${{ needs.get-tag.outputs.tag }} --dir tmp

- name: Replace `plugins/woocommerce` with unzipped woocommerce release build
Expand All @@ -587,7 +588,7 @@ jobs:

- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1
uses: aws-actions/configure-aws-credentials@v1-node16
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
Expand Down
4 changes: 4 additions & 0 deletions plugins/woocommerce/changelog/e2e-release-include-drafts
View file
Open in desktop
@@ -0,0 +1,4 @@
Significance: patch
Type: dev

Support E2E testing of draft releases.
93 changes: 66 additions & 27 deletions plugins/woocommerce/tests/e2e-pw/tests/smoke-tests/update-woocommerce.spec.js
View file
Open in desktop
@@ -1,6 +1,6 @@
const axios = require( 'axios' ).default;
const wcApi = require( '@woocommerce/woocommerce-rest-api' ).default;
const { ADMINSTATE, UPDATE_WC } = process.env;
const { ADMINSTATE, GITHUB_TOKEN, UPDATE_WC } = process.env;
const { downloadZip, deleteZip } = require( '../../utils/plugin-utils' );
const { test, expect } = require( '@playwright/test' );

Expand All @@ -20,34 +20,67 @@ const skipTestIfUndefined = () => {
}, skipMessage );
};

/**
*
* Get download URL of a WooCommerce ZIP asset by sending a `GET` request to `List releases` GitHub API endpoint in the WooCommerce repository.
*
* If `GITHUB_TOKEN` is defined, use it as `Authorization` header.
*
* @returns Download URL of the WooCommerce ZIP. This URL depends on whether `GITHUB_TOKEN` was specified or not.
*
* If `GITHUB_TOKEN` was defined, this function assumes that you're trying to access all releases, including drafts (as draft releases don't show up in the response of an unauthenticated GET `List releases` request ).
* In this case, the returned value will be the `asset.url`.
*
* Otherwise, the returned value will be the `asset.browser_download_url`.
*
* @throws Error if:
* - 'List releases' request was unsuccessful, or
* - no release with the given tag was found, or
* - when a WooCommerce ZIP asset was not found.
*
*/
const getWCDownloadURL = async () => {
let woocommerceZipAsset;

const response = await axios
.get(
`https://api.github.com/repos/woocommerce/woocommerce/releases/tags/${ UPDATE_WC }`
)
.catch( ( error ) => {
console.log( error.toJSON() );
return error.response;
} );
const requestConfig = {
method: 'get',
url: 'https://api.github.com/repos/woocommerce/woocommerce/releases',
headers: {
Accept: 'application/vnd.github+json',
},
};

if ( GITHUB_TOKEN ) {
requestConfig.headers.Authorization = `Bearer ${ GITHUB_TOKEN }`;
}
Comment on lines +51 to +53
Copy link
Contributor Author

@rodelgc rodelgc Mar 2, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this update-woocommerce.spec.js file centers around this particular line. This makes sure that the supplied token will be used when downloading the woocommerce.zip file from a draft release.


const response = await axios( requestConfig ).catch( ( error ) => {
if ( error.response ) {
console.log( error.response.data );
}

if (
response.status === 200 &&
response.data.assets &&
response.data.assets.length > 0 &&
( woocommerceZipAsset = response.data.assets.find(
( { browser_download_url } ) =>
browser_download_url.match(
/woocommerce(-trunk-nightly)?\.zip$/
)
) )
) {
return woocommerceZipAsset.browser_download_url;
} else {
const error = `You're attempting to get the download URL of a WooCommerce release zip with tag "${ UPDATE_WC }". But "${ UPDATE_WC }" is an invalid WooCommerce release tag, or a tag without a WooCommerce release zip asset like "7.2.0-rc.2".`;
throw new Error( error );
throw new Error( error.message );
} );

const releaseWithTagName = response.data.find(
( { tag_name } ) => tag_name === UPDATE_WC
);

if ( ! releaseWithTagName ) {
throw new Error(
`No release with tag_name="${ UPDATE_WC }" found. If "${ UPDATE_WC }" is a draft release, make sure to specify a GITHUB_TOKEN environment variable.`
);
}

const wcZipAsset = releaseWithTagName.assets.find( ( { name } ) =>
name.match( /^woocommerce(-trunk-nightly)?\.zip$/ )
);

if ( wcZipAsset ) {
return GITHUB_TOKEN ? wcZipAsset.url : wcZipAsset.browser_download_url;
}

throw new Error(
`WooCommerce release with tag "${ UPDATE_WC }" found, but does not have a WooCommerce ZIP asset.`
);
};

skipTestIfUndefined();
Expand All @@ -58,7 +91,13 @@ test.describe.serial( 'WooCommerce update', () => {
test.beforeAll( async () => {
await test.step( 'Download WooCommerce zip from GitHub', async () => {
const url = await getWCDownloadURL();
woocommerceZipPath = await downloadZip( { url } );
const params = { url };

if ( GITHUB_TOKEN ) {
params.authorizationToken = GITHUB_TOKEN;
}

woocommerceZipPath = await downloadZip( params );
} );
} );

Expand Down
3 changes: 3 additions & 0 deletions plugins/woocommerce/tests/e2e-pw/utils/plugin-utils.js
View file
Open in desktop
Expand Up @@ -147,6 +147,9 @@ export const downloadZip = async ( {
};

response = await axios( options ).catch( ( error ) => {
if ( error.response ) {
console.error( error.response.data );
}
throw new Error( error.message );
} );

Expand Down