Flaky tests after resource changes

[cramforce]

To repro:

Scroll to YT videos.
Click reload.
The document doesn't load uniformly.

Lets find out why. This should not happen.

CC @dvoytenko

[dvoytenko]

Hmm. Tried few times. I don't see it jumping around. Are you trying on your phone? Or emulator?

[erwinmombay]

same here, BUT i do see amp-twitter jumping around. ill ping you guys video through mail.

edit: @dvoytenko explained that this was a known problem

[dvoytenko]

So, twitter jumping is due to some scale/resize sequence. We tried to address it before, but I guess we didn't do so fully. It jumping around however would not cause any relayouts and thus I think what Malte is describing is a different thing.

One thing does cause some concern is how we apply "media" attribute. This could possibly cause a risk condition where we apply "media" attribute a little too late which would cause a visible effect. Not sure if this is the reason here.

[cramforce]

We should definitely make sure to apply media as early as possible.
On Sep 10, 2015 10:31 PM, "Dima Voytenko" notifications@github.com wrote:

So, twitter jumping is due to some scale/resize sequence. We tried to
address it before, but I guess we didn't do so fully. It jumping around
however would not cause any relayouts and thus I think what Malte is
describing is a different thing.

One thing does cause some concern is how we apply "media" attribute. This
could possibly cause a risk condition where we apply "media" attribute a
little too late which would cause a visible effect. Not sure if this is the
reason here.

—
Reply to this email directly or view it on GitHub
#61 (comment).

[cramforce]

The amp-img test for media queries is consistently failing for me. Lets try to fix that ASAP.

Also the test runner that @erwinmombay fixed is broken again. Tests basically only run once and then require a restart.

[erwinmombay]

@cramforce noticed that yesterday. not sure what happened, I'll take a look again.

[erwinmombay]

@cramforce will also investigate and fix failing test

[cramforce]

Also the iframe tests are flaky

[cramforce]

@erwinmombay This is most likely due to @dvoytenko's change. I think he already had an idea how to fix the media queries.

[erwinmombay]

#104 turned off the flaky tests for now.

as @cramforce has linked we'll need to fix this before preview.

[cramforce]

The tests still run locally, so we don't forget!

Also I haven't turned them all of I think. There are some tests that are super flaky on Travis. Those no longer run there, but some tests are flaky everywhere (maybe 10%) and those I left on :)

[erwinmombay]

@cramforce this should be good to close? or do we still have some really bad flaky tests? (think you've fixed it for the most part)

[erwinmombay]

@dvoytenko has been running into some consistent ones that only he can repro (carousel ones specifically), will take a look when i get a chance and might reopen this.