-
Notifications
You must be signed in to change notification settings - Fork 9.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests(smoke): update CLS-AF expectation #12353
Conversation
Interesting that this affects us. From the change description, at least, I wouldn't expect handling zoom would change things for our smoke test. |
This was the full bisect list: I didn't notice any other suspect changes. |
oh, I think you're definitely right, I'm just trying to understand why our number would change. I have no idea what the larger implications of |
lighthouse-cli/test/smokehouse/test-definitions/perf/expectations.js
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's some weirdness here that @adamraine, @paulirish, and I looked into a little bit but haven't figured out yet. e.g. when I try out the crbug's demo https://output.jsbin.com/wegixex using m91 and m92 (where the change landed), I'm not getting different CLS score
or weightedScoreDelta
in Lighthouse (have to hack around NO_FCP
errors). If this change implies that our defaults run into a CLS change with m92, I would have expected a change on that page as well, but maybe there's something about the iframe relative sizing that just happens to work out?
However, when we've landed clsAF smoke changes before (#11874 and #12034) I don't think we actually verified the numbers, we just trusted what Chrome gave us. Should we do the same here to unblock CI and file a follow-up issue to make sure we do our due diligence before making all-frame stuff visible in the report?
@patrickhulce pointed out that the demo page has a |
CLS weighted scores were erroneous and should be fixed in ToT:
https://chromium.googlesource.com/chromium/src/+/042fbfb4cc6a675da0dff4bf3fc08622af42422b