New UI: Cypress integration is missing.

[hackmajoris]

Environment information

Not relevant

Description

In the previous console UI, there was a 'TEST' step where Cypress tests were executed, and it generated comprehensive reports, including screenshots and videos, as detailed in this AWS blog post: https://aws.amazon.com/blogs/mobile/running-end-to-end-cypress-tests-for-your-fullstack-ci-cd-deployment-with-amplify-console/.

However, with the new UI, the process has changed significantly. While you can still run Cypress tests during the build stage, the reporting is not there. You only see whether the tests have passed or failed, even though the documentation states, "Amplify Hosting offers a deeper integration with Cypress that allows you to generate a UI report for your tests." The configurations for report generation are still present, as mentioned in the same AWS blog post: https://aws.amazon.com/blogs/mobile/running-end-to-end-cypress-tests-for-your-fullstack-ci-cd-deployment-with-amplify-console/.
This lack of detailed is not helpful. The documentation should be updated to reflect these changes, or the previous level of Cypress support should be reinstated.

[Jay2113]

Hi @hackmajoris 👋 , thanks for reaching out. We have identified this as a bug and are actively working on a fix for it. I'll keep the thread posted with any updates.

[Jay2113]

Hi @hackmajoris, thanks for your patience. We have released a fix for this feature and you should now be able to download the test artifacts and the Cypress results from the new Amplify console experience.

[hackmajoris]

Hi @hackmajoris, thanks for your patience. We have released a fix for this feature and you should now be able to download the test artifacts and the Cypress results from the new Amplify console experience.

Hey @Jay2113, while testing your fix, I can see that you can download those artifacts only for the successful builds. For the failing one I have just this option:

This is not very helpful, since usually you need to see those artifacts when the build fails.

The build may fail for different reasons and it can fail before executing the Cypress tests. In that case, the download of Cypress artifacts option can be hidden. But, if the build fails because of Cypress failed tests, the option to download the Cypress results should be in place.

[Tony-Donghua-Han]

Having the same issue, is there is workout for us to fetch the artifacts before we get another fix?

[hackmajoris]

Having the same issue, is there is workout for us to fetch the artifacts before we get another fix?

You can use this command:
aws amplify get-job --region eu-central-1 --app-id APP_ID --branch-name BRANCH_NAME --job-id JOB_NUMBER --profile AWS_PROFILE

[Jay2113]

@hackmajoris @Tony-Donghua-Han thank you for verifying the fix and providing feedback on allowing test artifact downloads upon failure. We are investigating into how we can support this functionality to improve the experience.

[Jay2113]

@hackmajoris @Tony-Donghua-Han we have prioritized and are actively working on a fix to enable downloading artifacts upon test failures. I'll update the thread when the fix is released. Thanks for your continued patience.

[Jay2113]

@hackmajoris @Tony-Donghua-Han 👋 , we have deployed another fix and you should now be able to download the test artifacts even when the tests fail. Can you please verify and confirm the same? Thanks!

[richardwadsworth]

@Jay2113 Do I need to do anything to make use of both of these fixes?

On successful builds I still do not have access to the test artifacts

And on failed build I only see the option to download the logs.

App id dsj8xdikh9gco

[richardwadsworth]

@Jay2113 Do I need to do anything to make use of both of these fixes?

On successful builds I still do not have access to the test artifacts

And on failed build I only see the option to download the logs.

App id dsj8xdikh9gco

Ok, I can see the changes if I deploy a new app. Can I pull those changes into an existing app? I'd rather not have to re configure my app based on the domain url that would be created if I deployed a new app.

[Jay2113]

Hi @richardwadsworth, I see that your app dsj8xdikh9gco has a couple of branches develop and feature/e2e. The tests are configured for the feature/e2e branch, are you unable to download the test artifacts for it? Can you share a screenshot?

[richardwadsworth]

Hi @richardwadsworth, I see that your app dsj8xdikh9gco has a couple of branches develop and feature/e2e. The tests are configured for the feature/e2e branch, are you unable to download the test artifacts for it? Can you share a screenshot?

Hi @Jay2113 , the tests are being run in the feature/e2e branch. I can download the test artefacts zip using "aws amplify get-job" but the options to download the test artefacts via the UI (on a successful or failed build) are not available on the UI. Refer to the screen shot.

[Jay2113]

Hi @richardwadsworth, thanks for your continued cooperation. To expedite the troubleshooting process, we would like to setup a call with you. Can you share your availability via a DM on our community discord server. My username is Jay Raval.

[hackmajoris]

@Jay2113 I can now download the artifacts for the failed builds. @richardwadsworth Try to re-run the build again. The download of artifacts it's not possible for the historical builds. Also, make sure that your build is not failing because of different reasons, other than the Cypress tests. The download of artifacts it's possible only if the Cypress was executed, which is the expected scenario.

[hackmajoris]

Thanks for this fix. Good job!