CSS loaded as incorrect charset unless explicitly declared

[edent]

Provide the steps to reproduce

1. Run LH on https://shkspr.mobi/blog/
2. When the page reloads, the ✍ emoji incorrectly renders as âœ.

Before LH

After LH

What is the current behavior?

Why this occurs:

• My style.css contains content: "✍";
• It is served as Content-Type: text/css; with no charset.
• When the devtools reloads the page after Lighthouse runs, it corrupts the text - I think into Windows-1252
• If I add @charset "UTF-8"; at the start of the CSS file, it renders correctly after Lighthouse runs.

What is the expected behavior?

The CSS spec says, if a text encoding can't be determined, assume UTF-8.

(I am unable to change the server to serve Content-Type: text/css; charset=utf-8)

Environment Information

• Lighthouse version: Whatever ships with Chromium 78
• Chrome version: Version 78.0.3904.108 (Official Build) snap (64-bit)
• Operating System: Ubuntu

Related issues

[connorjclark]

On Mac and can't reproduce. Perhaps it is just Linux.

[edent]

I've reproduced it on Mac. Mojave. Chrome Version 78.0.3904.108 (Official Build) (64-bit)

[dwm]

I successfully reproduced this on Windows with my default configuration:

disabled slowdown
desktop mode
OS: Windows 10
Chrome: Version 78.0.3904.108 (Official Build) (64-bit)

See image:

My default configuration includes the HTTPS Everywhere extension (https://chrome.google.com/webstore/detail/https-everywhere/gcbommkclmclpchllfjekcdonpmejbdp). When I direct HTTPS Everywhere to stop operating and re-run the test, this misbehaviour does not occur.

[bearpig]

On Mac (10.14.6) & Chrome (78.0.3904.108) - have reproduced.

See image.

edit - although this doesn't happen in Incognito Mode, and I do run a fair few chrome extensions normally, so might be a caching conflict somewhere?

[connorjclark]

@edent can you confirm @bearpig's extension theory by trying this in Incognito?

[edent]

Reproduced in 79 on Mac in incognito mode. Reproducible example at http://shkspr.mobi/test/test.html

[patrickhulce]

@edent I'm not able to reproduce this behavior in a clean Chrome profile and Incognito mode on m79. Is it possible you still have some Chrome extensions running in Incognito mode?

[edent]

Just installed a fresh Chrome on Linux. No extensions. Still fails.

Settings are Desktop, Accessibility, No Throttling, Clear Storage.

Chrome Version 79.0.3945.117 (Official Build) (64-bit)
Generated by Lighthouse 5.5.0

The only thing which could be different is that my language settings are English UK / en_gb

[jpwsutton]

Hi, I have the same issue on macos
Chrome Version 79.0.3945.88 (Official Build) (64-bit)
Lighthouse 5.5.0

[edent]

Here's a completely fresh Canary build on OSX. No extensions loaded.

[patrickhulce]

Aha! Canary + Desktop did the trick thanks folks I'm able to reproduce now on macOS.

[connorjclark]

Canary 81.0.4026.0, OSX. Still can't repro. @patrickhulce is it fixed for you?

[edent]

Still an issue on 4026.

Fresh install. No plugins. UK / EN_GB language. Accessibility Desktop Audit.

[patrickhulce]

Confirmed still an issue on 81.0.4026.0 for me as well on macOS.

@connorjclark you're running with exactly these settings, right?

[connorjclark]

yes

what rendered font does your system use for the element w/ the hand emoji?

[patrickhulce]

not that :)

also worth noting that simply refreshing the page fixes the issue and uses Apple Color Emoji font again

[connorjclark]

i dunno then. Can't repro so this is all you atm :)

[patrickhulce]

Feels eerily similar to #10067 , P1.5 because whatever's going on with DevTools audits deserves a look.

[edent]

We've passed the 1 year anniversary of this bug - and it is still a problem. Could someone please help me fix it?

Screencast.from.20-12-20.22.53.16.mp4

To reproduce:

• Visit https://shkspr.mobi/test/test.html
• Use an incognito window with no extensions, on the latest version of Chrome
• Run a Lighthouse report as a Desktop device
• You may need to be set to en_gb
• Reproduced on Mac and Linux and Windows - see above

Minimum Viable Code:

HTML

<!doctype html>
<html lang="en">
<head>
	<meta charset="utf-8">
	<link rel="stylesheet" href="test.css">
 </head>
 <body>
 	<hr class="emoji"/>
 </body>
 </html>

CSS

.emoji:after {
    content: "✍";
    font-size: 2em;
    display: inline-block;
    position: relative;
    top: -0.33em;
}

[connorjclark]

Noticed that the CSS doesn't show up in the Network tab after the LH run. First clear the Network panel, then run LH, then go back to Network panel and you see:

(blocking the CSS resource on a normal page load doesn't create the same effect–in that case it's just a page with no text)

Also: this only happens if "Disable Cache" is not checked. That's the default in a new chrome profile, which is why Incognito repros so well. Also doesn't repro with just "Performance" checks (seems it only happens when LH does multiple passes?)

[patrickhulce]

Nice find @connorjclark , possibly

lighthouse/lighthouse-core/config/default-config.js

Line 178 in 96eda60

blockedUrlPatterns: ['*.css', '*.jpg', '*.jpeg', '*.png', '*.gif', '*.svg', '*.ttf', '*.woff', '*.woff2'],

leftover? I wonder if LH cleaned up its blocked patterns instead of just disconnecting it would fix it.

[connorjclark]

Big hunch: perhaps Chrome is correctly (as the spec says) assuming UTF-8 when it gets a CSS response w/ no charset, but when it saves to cache and reads from it later, makes no such assumption about encoding. This might be enough information to file a chromium bug–what do others think?

[connorjclark]

@patrickhulce I don't see how that would matter–if the resource were blocked, the page would have no text (because text is only added via the CSS). It's an encoding issue, not a missing resource.

But I'll try to comment that out locally in DT.

[patrickhulce]

If the source of the issue is a subtle change in resource loading assumptions and one part of Chrome thinks this request should be blocked while another part doesn't actually block it, seems like there's at least a chance it's related.

The cache route also seems promising, if so, should be able to repro with a simple local server and no LH involved (which makes a far more compelling CRbug IMO :)).

[connorjclark]

But I'll try to comment that out locally in DT.

Still happens even if removing the blockedUrlPatterns.

Minimal settings for repro: do not check "Disable Cache", only check PWA category, desktop config.

[paulirish]

The CSS is served by the memory cache in these "corrupted" cases.

At the protocol-level, we don't see a Network.requestWillBeSent (typically), which is why it doesn't show up in the Network panel. Normally, even memory-cache served items have one of these.

Also funny: the Network.requestServedFromCache event indicates it's served from memory cache, and it's emitted entirely separately from the .responseReceived payload which signals about 3 different caches.

[paulirish]

A workaround is that we run cleanBrowserCaches at the end of our gathering, this'd clear the memory cache and ensure this subsequent load would refetch and be OK.

[connorjclark]

I agree it must be serving from cache, but the protocol doesn't say that explicitly.

There's a "served from cache" event but that's from before Lighthouse ends, as far as I can tell. I highlighted the last request for the HTML document. No request follows for the CSS (cache or otherwise), according to the protocol.

[connorjclark]

Oh, and loading normally w/ cache enabled and devtools open (refreshing, don't run LH), you can see the CSS served by "memory cache" but it never "corrupts" :/

[paulirish]

I took a look at the data in chrome://net-export/ but nothing worthwhile there. Kinda makes sense to me, given that the memory cache lives in the renderer and net-layer is way out in the browser process.

[sbenstewart]

Replicated the issue with the following setup.
OS - Windows 11 Home Single Language 21H2
Chrome - Version 96.0.4664.45 (Official Build) (64-bit)

And, this is the second anniversary of this bug!

[adamraine]

I don't have a minified repro, but I was able to prevent the issue with changes in this branch:
https://github.com/GoogleChrome/lighthouse/compare/stop-charset-change

It looks like this changes too much with the page load, but reenabling full-page-screenshot will still produce the charset problem. I suspect it's an issue with the Page CDP domain messing with the cache or something.

[edent]

Three years later and this bug is still present on Version 110.0.5481.96

Do you have a Patreon or something I can donate to in order to get this fixed?

[amirsarfarW]

Same issues happening here! :(

[amirsarfarW]

Three years later and this bug is still present on Version 110.0.5481.96

Do you have a Patreon or something I can donate to in order to get this fixed?

Did you find any solution?