Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated test to account for flakiness. #54874

Merged
merged 4 commits into from
Nov 15, 2023
Merged

Conversation

jmkulwik
Copy link
Contributor

@jmkulwik jmkulwik commented Nov 14, 2023

We've been hitting an issue in our testing system where the sprites are sometimes doubled. This does not repro on test or production, and it appears to be an issue with either and outdated version of browsers in our testing system or with the web driver that's being used. We suspect that updating the versions will address the issue. In the meantime, to unblock the pipeline, we are ok accepting any number of sprites that is at least 10. That way we can be sure sprites have been generated and also not block on this intermittent (but frequent) issue.

Slack details where this was reported: https://codedotorg.slack.com/archives/C0T0PNTM3/p1699578533080019

Image of doubled sprites
Screenshot 2023-11-09 at 5 23 17 PM

Follow-up task: https://codedotorg.atlassian.net/browse/LABS-241

This is a follow-up from #54876

Links

Testing story

Deployment strategy

Follow-up work

Privacy

Security

Caching

PR Checklist:

  • Tests provide adequate coverage
  • Privacy and Security impacts have been assessed
  • Code is well-commented
  • New features are translatable or updates will not break translations
  • Relevant documentation has been added or updated
  • User impact is well-understood and desirable
  • Pull Request is labeled appropriately
  • Follow-up work items (including potential tech debt) are tracked and linked

@jmkulwik jmkulwik requested review from davidsbailey and a team November 14, 2023 00:24
Copy link
Member

@breville breville left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable for a temporary fix, so long as we know it's not a real bug.

It might be worth a comment just above the changed line, maybe using the # TODO: format.

@jmkulwik jmkulwik merged commit 9f34315 into staging Nov 15, 2023
2 checks passed
@jmkulwik jmkulwik deleted the jmkulwik/fix-flaky-test branch November 15, 2023 00:22
@fisher-alice fisher-alice mentioned this pull request Jan 28, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants