[meta] List of sites that give Lighthouse trouble

[patrickhulce]

To Investigate

• https://h5.m.taobao.com/trip/home/index.html, from No domContentLoaded event in trace #3838, "no DCL in trace", "no screenshots in trace" 100% reproducible

Axe failures

• http://visiondirect.com (from GTech) 🐌 🐌 🐌 invalid selector, tracked by aXe fails on XHTML documents with prefixed element names dequelabs/axe-core#563

Service worker/offline issues

• https://www.lenskart.com/ tracked by Problems with multiple service workers #3724
• https://next.weekplan.net
• https://www.flipkart.com from Failed Audits: Myntra and Flipkart #3736

Stopped tracing too early

• https://www.instagram.com/p/BNQ98s4BTEa/ (from comment) tracked by Network timestamps differ significantly from NetworkRecorder observed time #3734
• https://weather.com (from comment)
• https://www.davidemaggio.it/ (from comment)
• http://lampsplus.com (from GTech)
• http://valuecityfurniture.com (from GTech)

Site misbehavior (aborted request, failed document, extremely slow server, etc)

• https://app.jalantikus.com/ (from comment)
• http://createspace.com (from GTech)
• http://gifts.com (from GTech)
• https://music.amazon.com/ (from CDS) tracked by More gracefully handle aborted requests #3735

Working but needed more than 30s

• http://mcdcc.org/ 😢
• http://pwa.voot.com/ (from comment) 💨 🐌
• http://m.aliexpress.com (from comment) needs ~60+ seconds!
• https://nytimes.com (from comment) 💨 🐌
• http://hollisterco.com (from GTech) 🐌
• http://petfooddirect.com (from GTech) 🐌 needs ~60+ seconds!
• http://potterybarnkids.com (from GTech) 🐌
• http://rei.com (from GTech) 🐌
• http://shopjustice.com (from GTech) 🐌
• http://soma.com (from GTech) 🐌
• http://sportsauthority.com (from GTech)
• http://sterlingjewelers.com (from GTech)
• http://tailgateclothing.com (from GTech)
• http://thehappinessplanner.com (from GTech) 🐌 needs ~60 seconds!
• http://toddsnyder.com (from GTech)
• http://victoriassecret.com (from GTech)
• http://zulily.com (from GTech)
• http://www.gmarket.co.kr/ 🐌 🐌 🐌 🐌 axe takes ~30s, needs 60+ seconds

Working as of 2.6

• https://m.forbes.com (from Can't complete LH run on forbes PWA #2254) 💨
• https://q-citrix.salesnext.com/login (from [A11y Audit] Audit Error: ... Protocol error (Runtime.evaluate): Object reference chain is too long #2265)
• http://veerle.duoh.com/ (from More "is fast enough on 3g" flakiness #2170) (polish: do not fail loadfast4pwa for internal redirects #2296)
• https://airhorner.com (from comment)
• https://react-hn.appspot.com/ (from comment) 💨
• https://www.nasa.gov (from comment)
• http://contactsdirect.com (from GTech)
• http://skinstore.com (from GTech)
• http://sunglasshut.com (from GTech)
• http://theshadestore.com (from GTech)
• http://trunkclub.com (from GTech)
• http://walgreens.com (from GTech) 🐌 🐌
• http://weddingtonway.com (from GTech)
• http://whitehouseblackmarket.com (from GTech) 🐌
• http://toysrus.com (from GTech) 💨 🐌 🐌 🐌 axe can fail

Legend

💨 = can be flaky (received a bad trace or axe failure on at least one run, but not reproducible)
😢 = undesirable metric result (but correct according to definition)
🐌 = Lighthouse is slow, i.e. multiple seconds spent in gathering/auditing

[paulirish]

• https://app.jalantikus.com/ didn't finish its load once or twice

perf diagnostics

• https://www.instagram.com/p/BNQ98s4BTEa/ is surprisingly slow to load. are we identifying the top perf concern?
• http://pwa.voot.com/ is v slow to load. are we identifying the top perf conern?

[paulirish]

classics

• https://airhorner.com
• http://m.aliexpress.com
• about:blank (yes, someone tried it yesterday and expected it to be fine.)

[brendankenny]

• https://react-hn.appspot.com/ we stop too soon and only get TTCI ~50% of the time, TTFI ~80-90% of the time. Has an unusual SSR/delayed hydration load scheme that may be tricking us

[brendankenny]

• https://www.vevo.com/ may fall into the same timing oddness category as react-hn where we're cutting off too early. In my latest plots/ run, it usually had end-of-trace at 20s, but neither TTCI or TTFI were found. There were four trace outliers, however, where it went for the full 30s timeout, and for those, TTFI was found in all four, TTCI was in three. (polish: listen for network idle after DCL #2271)

30s may be too short

all these struggled with TTI, but all also hit the 30s timeout

• https://nytimes.com was fine with TTFI but struggled with TTCI. I assume new nytimes failures are from throttling changes from Update network throttling to simulate more realistic network conditions #2238
• https://weather.com/ fine with TTFI but never reached TTCI
• https://www.nasa.gov/ Found TTFI 1/6 of the time, never reached TTCI. onload (loadEventEnd) was usually not found in the trace taken

[patrickhulce]

30s may be too short

the new throttling is going to make this true for a lot of sites now, we will basically fail on all sites from deep's TTI variability study that had TTI of >25s, which is a lot

[patrickhulce]

re: https://airhorner.com I'm not seeing anything fishy, what fails usually?
re: https://react-hn.appspot.com/ to provide more color here, this app actually renders in response to websocket messages and fixing would require a CPU quietness signal over the protocol since we can't reasonably guess when to stop tracing with the current network request signals alone

[patrickhulce]

• https://plustimer.firebaseapp.com/ (fails the perf metrics with an atob is not defined debug string) (polish: handle traces with base64 encoded args #2300)

[benschwarz]

• Bloomberg was having trouble too

[patrickhulce]

@benschwarz could you elaborate with what issues you're seeing there? On a couple runs the only issue I'm seeing is running up to the 30s timeout before TTCI is computed

[benschwarz]

@benschwarz could you elaborate with what issues you're seeing there? On a couple runs the only issue I'm seeing is running up to the 30s timeout before TTCI is computed

Ok, if it runs at all it must've been fixed in the last two weeks. 👍
The 30s timeout is a whole other thing, which is being tracked elsewhere, right?

[ebidel]

• https://www.davidemaggio.it/ fails to finish.

• large number of network requests: ~300

• huge number of DOM nodes: 11,180 to be precise. Lighthouse spends a lot time in evaluating getEventListeners for ExternalAnchorsUseRelNoopenerAudit and other related audits.

[upugo-dev]

@patrickhulce any update on these issues? we are experiencing TTI issues quite frequently during our testing

[patrickhulce]

@upugo-dev these were mostly addressed by 3.0 and is quite old, probably a candidate for closing. If you have a specific, reproducible error on a particular URL then file it in a separate issue 👍

[brendankenny]

these were mostly addressed by 3.0 and is quite old, probably a candidate for closing

closing! We can have other issues for sites that give Lighthouse trouble without confusing things mixing in pre-3.0 issues :)