Skip to content
This repository has been archived by the owner. It is now read-only.

"sign up, wait for confirmation screen, click resend" test is being rate-limited #4537

Closed
vladikoff opened this issue Dec 21, 2016 · 12 comments
Closed
Assignees

Comments

@vladikoff
Copy link
Contributor

@vladikoff vladikoff commented Dec 21, 2016

CancelError: Timeout reached on firefox on any platform - confirm - sign up, wait for confirmation screen, click resend
  at null._onTimeout  <node_modules/intern/lib/Test.js:132:20>
  at Timer.listOnTimeout  <timers.js:92:15>
  at Command.then  <node_modules/intern/node_modules/leadfoot/Command.js:542:10>
  at Command.finally  <node_modules/intern/node_modules/leadfoot/Command.js:566:15>
  at <node_modules/intern/lib/Test.js:206:37>
  at Promise.cancel  <node_modules/intern/browser_modules/dojo/Promise.ts:343:37>
  at null._onTimeout  <node_modules/intern/lib/Test.js:134:21>
  at Timer.listOnTimeout  <timers.js:92:15>

This one test failed 3 times in a row:
image

https://tc-test.dev.lcip.org/viewType.html?buildTypeId=FxaLatest_FunctionalTests

@shane-tomlinson shane-tomlinson added this to the FxA-0: quality milestone Dec 21, 2016
@vladikoff vladikoff changed the title TeamCity Flaky Test: confirm - sign up, wait for confirmation screen, click resend TeamCity Flaky Test: confirm - sign up, wait for confirmation screen, click resend Dec 22, 2016
@shane-tomlinson shane-tomlinson self-assigned this Dec 22, 2016
@shane-tomlinson
Copy link
Member

@shane-tomlinson shane-tomlinson commented Dec 22, 2016

I found this nugget in visibleByQSA:

// If the element is animating, try again after a delay. Clicks
// do not always register if the element is in the midst of
// an animation.

The '#resend' element fades in, using a CSS animation.

@shane-tomlinson
Copy link
Member

@shane-tomlinson shane-tomlinson commented Dec 23, 2016

I'm going to re-open this until we see results on tc-test.

@vladikoff
Copy link
Contributor Author

@vladikoff vladikoff commented Dec 28, 2016

Issue is due to rate limiting, #4561 (comment)

@shane-tomlinson
Copy link
Member

@shane-tomlinson shane-tomlinson commented Jan 3, 2017

Issue is due to rate limiting, #4561 (comment)

Great catch. Do we need to add mailinator to the list on latest too?

@vladikoff
Copy link
Contributor Author

@vladikoff vladikoff commented Jan 3, 2017

@shane-tomlinson i don't think we have a "list" like that for latest. We have this list: https://github.com/mozilla/fxa-customs-server/blob/9b94bd1bdd012217e322d2e008e16f3d7e57c22c/lib/config/config.js#L176 that is used everywhere and I'm not sold about adding mailinator to that list.

@shane-tomlinson
Copy link
Member

@shane-tomlinson shane-tomlinson commented Jan 3, 2017

Hmm, from that list, who is using mockmyid.com? Persona is no more, mockmyid.com shouldn't work anymore.

@vladikoff
Copy link
Contributor Author

@vladikoff vladikoff commented Jan 3, 2017

@shane-tomlinson I remember seeing Nick Alexander using it for testing some Fennec Sync. @rfk or @jrgm might know of other uses.

@shane-tomlinson
Copy link
Member

@shane-tomlinson shane-tomlinson commented Jan 3, 2017

I'm not sold about adding mailinator to that list.

I wouldn't want to add mailinator globally, but I want to get latest functional tests passing. The two obvious workarounds are to add mailinator to the list for latest, or use a different email that won't be rate limited but still provides the functionality we need.

@vladikoff
Copy link
Contributor Author

@vladikoff vladikoff commented Jan 3, 2017

@shane-tomlinson maybe we can just use @mockmyid in that test and leave it in the list.

@shane-tomlinson
Copy link
Member

@shane-tomlinson shane-tomlinson commented Jan 3, 2017

@shane-tomlinson maybe we can just use @mockmyid in that test and leave it in the list.

Seems like a winner.

@vladikoff vladikoff self-assigned this Jan 3, 2017
@rfk
Copy link
Member

@rfk rfk commented Jan 3, 2017

I remember seeing Nick Alexander using it for testing some Fennec Sync

Yes, I vaguely recall this being one reason. I've also used it in some sync loadtests in the past because it's easy to generate valid assertions for. But given the demise of persona (and the fact that it isn't a Mozilla-owned domain) we should probably work towards replacing it anywhere it's being used.

Removing it from the customs bypass list might be a good way to flush out anyone still using it :-)

@vladikoff
Copy link
Contributor Author

@vladikoff vladikoff commented Jan 3, 2017

Ok let's remove it! However then we need another domain we can use in https://github.com/mozilla/fxa-content-server/pull/4583/files

@vladikoff vladikoff changed the title TeamCity Flaky Test: confirm - sign up, wait for confirmation screen, click resend "sign up, wait for confirmation screen, click resend" test is being rate-limited Jan 10, 2017
shane-tomlinson pushed a commit that referenced this issue Jan 12, 2017
Replace `mailinator.com` with `restmail.dev.lcip.org`. mailinator addresses
are rate limited, restmail.dev.lcip.org is a mozilla controlled server and
won't be rate limited.

I have not used the `fixes` meta tag below because we have to update
the config on latest to not rate limit `restmail.dev.lcip.org`

issue #4537
shane-tomlinson pushed a commit that referenced this issue Jan 12, 2017
Replace `mailinator.com` with `restmail.dev.lcip.org`. mailinator addresses
are rate limited, restmail.dev.lcip.org is a mozilla controlled server and
isn't rate limited.

fixes #4537
@shane-tomlinson shane-tomlinson changed the title "sign up, wait for confirmation screen, click resend" test is being rate-limited "sign up, wait for confirmation screen, click resend" test is being rate-limited Jan 12, 2017
vladikoff added a commit that referenced this issue Jan 12, 2017
… r=vladikoff

Replace `mailinator.com` with `restmail.dev.lcip.org`. mailinator addresses
are rate limited, restmail.dev.lcip.org is a mozilla controlled server and
isn't rate limited.

fixes #4537
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants