Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: Resolve OBJ create and delete E2E test flake #10245

Merged
merged 7 commits into from
Mar 7, 2024
Merged

test: Resolve OBJ create and delete E2E test flake #10245

merged 7 commits into from
Mar 7, 2024

Conversation

jdamore-linode
Copy link
Contributor

@jdamore-linode jdamore-linode commented Mar 1, 2024 •
edited

Description 📝

This resolves the test failures that we've recently been seeing in object-storage.e2e.spec.ts. There are a couple issues at play that contribute to these failures:

  • After a Bucket has been created, the landing page re-renders (presumably in response to some API request resolving), causing occasional failures when the re-render occurs after Cypress has located some element (e.g. the "Delete" table row button) but before it has interacted with it. (I think this is irrelevant from a user experience point of view)
  • Less frequently, the test fails due to an issue with the Buckets landing page where the page fails to display a newly created Bucket if the Bucket was created before the initial requests to fetch Buckets have all completed (see M3-7833)

This PR only seeks to resolve test failures related to these behaviors, and it does so by waiting for all related API requests to resolve before attempting to create a Bucket. Because Cloud makes an indeterminate number of requests to fetch Buckets, the test waits for the loader to disappear instead of intercepting and waiting on the requests directly.

I was able to reproduce these failures easily on develop, and after making these changes I ran the test on repeat 50 times and all 50 runs passed.

Changes 🔄

  • Wait for loader to disappear and for network utilization data to be fetched before creating an OBJ bucket

How to test 🧪

We can lean on CI for this, but you can use this command to run the tests locally:

yarn cy:run -s "cypress/e2e/core/objectStorage/object-storage.e2e.spec.ts"

If you want to run them repeatedly, you can use this command if you're using zsh as your shell:

repeat 5 yarn cy:run -s "cypress/e2e/core/objectStorage/object-storage.e2e.spec.ts"

As an Author I have considered 🤔

Check all that apply

  • 👀 Doing a self review
  • ❔ Our contribution guidelines
  • 🤏 Splitting feature into small PRs
  • ➕ Adding a changeset
  • 🧪 Providing/Improving test coverage
  • 🔐 Removing all sensitive information from the code and PR description
  • 🚩 Using a feature flag to protect the release
  • 👣 Providing comprehensive reproduction steps
  • 📑 Providing or updating our documentation
  • 🕛 Scheduling a pair reviewing session
  • 📱 Providing mobile support
  • ♿ Providing accessibility support

@jdamore-linode jdamore-linode self-assigned this Mar 1, 2024
@jdamore-linode jdamore-linode requested a review from a team as a code owner March 1, 2024 16:50
@jdamore-linode jdamore-linode requested review from cliu-akamai and removed request for a team March 1, 2024 16:50
@jdamore-linode jdamore-linode requested a review from a team as a code owner March 1, 2024 16:54
@jdamore-linode jdamore-linode requested review from cpathipa and jaalah-akamai and removed request for a team March 1, 2024 16:54
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 1, 2024 •
edited

Coverage Report: ✅
Base Coverage: 81.38%
Current Coverage: 81.38%

@jaalah-akamai
Copy link
Contributor

Screenshot 2024-03-01 at 2 00 27 PM

@jdamore-linode
Copy link
Contributor Author

(Test failure picture)

Thanks @jaalah-akamai, pushed a fix for that! Very glad you caught that as it would've been a guaranteed failure for all of our test accounts with no buckets 🙏

mjac0bs
mjac0bs approved these changes Mar 5, 2024
View reviewed changes
Copy link
Contributor

@mjac0bs mjac0bs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The repeat command is useful, thanks for including that!

This test passed consistently for me, and your 50x good runs are good assurance that the flake is resolved. 🚀

@mjac0bs mjac0bs added Approved Multiple approvals and ready to merge! and removed Ready for Review labels Mar 5, 2024
@mjac0bs
Copy link
Contributor

mjac0bs commented Mar 5, 2024

One note, though: in some subsequent runs I'd kicked off in the background after that initial 5, I've seen this failure occasionally (twice within ~20 runs):

image

I haven't seen it when watching the test run in debug mode, didn't get to the video output before it was overwritten, and not sure why this would be happening if we've got our wait for @uploadObject, anyway. 🤔

@jdamore-linode
Copy link
Contributor Author

One note, though: in some subsequent runs I'd kicked off in the background after that initial 5, I've seen this failure occasionally (twice within ~20 runs):

(Screenshot failure)

I haven't seen it when watching the test run in debug mode, didn't get to the video output before it was overwritten, and not sure why this would be happening if we've got our wait for @uploadObject, anyway. 🤔

Thanks @mjac0bs! I'll take a look at this before merging, and if I can reproduce it I'll try to fix it 👍 I was using cy.only when I was testing my changes so this totally fell under the radar.

(Also worth pointing out that I'm experimenting with some Jenkins pipeline changes using this PR, so if you see unusual results for the cloud-manager-tests run, that's probably why!)

@jdamore-linode
Copy link
Contributor Author

@mjac0bs I've been totally unable to reproduce this locally (tried something like 80 times 😭) but I did find a recording of it happening in Cypress. I wrote up M3-7850 to revisit this soon!

@jdamore-linode jdamore-linode merged commit caf10cb into linode:develop Mar 7, 2024
18 checks passed
vrajesh73 added a commit to vrajesh73/manager that referenced this pull request Mar 12, 2024
@vrajesh73
Merge branch 'develop' of https://github.com/vrajesh73/manager into f…
521e8b2 
…eature/namespace-create

* 'develop' of https://github.com/vrajesh73/manager: (89 commits)
  fix: [M3-7269] - Display parent email in user menu when no company name is available for restricted parent user (linode#10248)
  fix: [M3-7817] - Show correct status of Child Account Enabled column for parent users (linode#10233)
  upcoming: [M3-7616] - Add Placement Groups Events and Notifications (linode#10221)
  upcoming: [M3-7816-v2] - Adjust logic for when to show Switch Account button (linode#10266)
  fix: [M3-7831] - Persisting error messages in ACLB delete dialogs (linode#10254)
  upcoming: [M3-7842] - Update Assign Linode Drawer and improve query skipping (linode#10263)
  upcoming: [M3-7704] - Disable Cloning, Private IP, Backups for edge regions (linode#10222)
  test: Fix test flake for Images landing page test (linode#10267)
  fix: [M3-7824] - ACLB TCP Rule Creation and other fixes (linode#10264)
  refactor: [M3-7687] - Linodes Restricted User Experience 2/2 (linode#10227)
  test: Resolve OBJ create and delete E2E test flake (linode#10245)
  upcoming: [M3-7723] - Placement Group feature flag as object (linode#10256)
  chore(deps): Bump sanitize-html from 2.11.0 to 2.12.1 (linode#10247)
  change: [M3-7813] - Allow the disabling of the TypeToConfirm input (linode#10251)
  upcoming: [M3-7839] - Change Business Partner to Parent User (linode#10259)
  upcoming: [M3-7835] - Adjust user table column count (linode#10252)
  upcoming: [M3 7738] - Update Placement Group Create & Edit Drawers (linode#10205)
  refactor: [M3-7437] - Use `@lukemorales/query-key-factory` for Profile Queries (linode#10241)
  fix: React Query `updateInPaginatedStore` helper function not working as expected (linode#10249)
  test: [M3-7497] - Add tests for child user verification banner (linode#10204)
  ...

# Conflicts:
#	packages/manager/src/MainContent.tsx
#	packages/manager/src/dev-tools/FeatureFlagTool.tsx
This was referenced Mar 18, 2024
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mjac0bs mjac0bs mjac0bs approved these changes

@bnussman-akamai bnussman-akamai bnussman-akamai approved these changes

@cliu-akamai cliu-akamai Awaiting requested review from cliu-akamai cliu-akamai is a code owner automatically assigned from linode/frontend-sdet

@cpathipa cpathipa Awaiting requested review from cpathipa cpathipa is a code owner automatically assigned from linode/frontend

@jaalah-akamai jaalah-akamai Awaiting requested review from jaalah-akamai jaalah-akamai is a code owner automatically assigned from linode/frontend

Assignees

@jdamore-linode jdamore-linode

Labels
Approved Multiple approvals and ready to merge!
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@jdamore-linode @jaalah-akamai @mjac0bs @bnussman-akamai