Add queries for bfcache

[westonruter]

I was curious to find out how often bfcache is disabled on WordPress pages. It turns a third of pages have it disabled. In the following table of results from bfcache-score-counts.sql, the score comes from the bf-cache Lighthouse audit, where for this audit it is binary, with 1 meaning bfcache is enabled and 0 meaning it was disabled for 1 or more reasons:

Score	Count
1	3,695,471
0	1,820,553
null	113,350

(1820553/(3695471+1820553) = 0.330048056 ≈ 33%)

In addition to score, the Lighthouse audit also includes a displayValue that summarizes the number of issues found. I also queried for that instead of the score and found a single failure is 3x more common than two failure reasons, and two failure reasons are 3x more common than three failure reasons. The following shows the top 10 most displayValue:

displayValue	count
1 failure reason	1156186
2 failure reasons	354844
3 failure reasons	103371
4 failure reasons	46228
12 failure reasons	25102
15 failure reasons	23863
5 failure reasons	21625
6 failure reasons	13372
13 failure reasons	12638
10 failure reasons	10924

However, when there is 1 failure reason it isn't always the same reason.

To get specific, I wrote bfcache-failure-reasons.sql to extract the underlying failure reasons:

Reason	Count	Percentage
Pages whose main resource has cache-control:no-store cannot enter back/forward cache.	527756	22.14%
The page has an unload handler in the main frame.	499127	20.93%
Pages that use Media Device Dispatcher are not eligible for back/forward cache.	445422	18.68%
The page has an unload handler in a sub frame.	445149	18.67%
Pages with WebSocket cannot enter back/forward cache.	161201	6.76%
Pages that use SpeechSynthesis are not currently eligible for back/forward cache.	85148	3.57%
Pages that use WebXR are not currently eligible for back/forward cache.	79395	3.33%
Pages that use WebLocks are not currently eligible for back/forward cache.	50942	2.14%
The page cannot be cached because it has a BroadcastChannel instance with registered listeners.	18614	0.78%
An iframe on the page started a navigation that did not complete.	18564	0.78%
The page did not finish loading before navigating away.	16158	0.68%
Pages that have requested sensor permissions are not currently eligible for back/forward cache.	10029	0.42%
Pages that use WebHID are not currently eligible for back/forward cache.	7921	0.33%
Pages that have inflight fetch() or XHR are not currently eligible for back/forward cache.	6475	0.27%
Internal error.	6373	0.27%
Back/forward cache is disabled due to a keepalive request.	2393	0.10%
The page was evicted from back/forward cache because an active network request involved a redirect.	844	0.04%
Pages that use WebAuthetication API are not eligible for back/forward cache.	493	0.02%
Pages that use WebDatabase are not currently eligible for back/forward cache.	454	0.02%
Modal dialog such as form resubmission or http password dialog was shown for the page upon navigating away.	317	0.01%
Popup blocker was present upon navigating away.	195	0.01%
ServiceWorker was unregistered while a page was in back/forward cache.	185	0.01%
Only pages with a status code of 2XX can be cached.	173	0.01%
Back/forward cache is disabled due to an IndexedDB event.	135	0.01%
Pages that use MediaSession API and set action handlers are not eligible for back/forward cache.	133	0.01%
Pages that use WebOTPService are not currently eligible for bfcache.	119	0.00%
Back/forward cache is disabled due to a document error.	97	0.00%
The page was evicted from back/forward cache due to a service worker activation.	90	0.00%
Pages that do not have a valid response head cannot enter back/forward cache.	67	0.00%
The page was evicted from the cache because an active network connection received too much data. Chrome limits the amount of data that a page may receive while cached.	65	0.00%
JsNetworkRequestReceivedCacheControlNoStoreResource	43	0.00%
Only pages loaded via a GET request are eligible for back/forward cache.	20	0.00%
Chrome Password Manager was present upon navigating away.	15	0.00%
Pages that use SharedWorker are not currently eligible for back/forward cache.	15	0.00%
The page timed out entering back/forward cache (likely due to long-running pagehide handlers).	13	0.00%
Pages that use IdleManager are not currently eligible for back/forward cache.	10	0.00%
Only pages whose URL scheme is HTTP / HTTPS can be cached.	10	0.00%
Pages with WebRTC cannot enter back/forward cache.	10	0.00%
Chrome detected an attempt to execute JavaScript while in the cache.	9	0.00%
Pages that use SpeechRecognizer are not currently eligible for back/forward cache.	7	0.00%
Pages that use Keyboard lock are not currently eligible for back/forward cache.	5	0.00%
WebSocketSticky	3	0.00%
Chrome restarted and cleared the back/forward cache entries.	2	0.00%
The page was claimed by a service worker while it is in back/forward cache.	2	0.00%
Pages that have requested MIDI permissions are not currently eligible for back/forward cache.	2	0.00%
The cache was intentionally cleared.	1	0.00%

The presence of an unload event handler amounts to ~21% of all reasons for bfcache being disabled. This may be reduced by Core-55491. But the top reason for bfcache being disabled is the site sending Cache-Control: no-store. That seems like the easiest failure to look for in the ecosystem and to fix.

Lastly, in heartbeat-script-presence.sql I check to see how common the Heartbeat script actually is:

Presence	Count
true	14,095
false	5,615,279

So only 0.25% of pages (14,095/(5,615,279+14,095)) have the Heartbeat present. Therefore, fixing Core-55491 is not really going to make a dent.

[adamsilverstein]

Looks good to me overall. Looking at the UNNEST and COUNTs, I was wondering if any of those need a DISTINCT clause?

[adamsilverstein]

Looks good!

[westonruter]

Looking at the UNNEST and COUNTs, I was wondering if any of those need a DISTINCT clause?

@adamsilverstein Using a DISTINCT for which SELECT?

[adamsilverstein]

Not sure - I'll test and get back to you.

[adamsilverstein]

Not sure - I'll test and get back to you.

The query looks fine, I was just double checking that you had considered possible row duplication with UNNEST!

[felixarntz]

Thanks @westonruter. The queries mostly look good, though they can be simplified, see feedback below.

[westonruter]

@felixarntz I've applied the changes and re-run the queries.

[felixarntz]

Thanks @westonruter!