Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate if changing the font loading mechanism is better for users #1437

Open
nickcolley opened this issue Jun 6, 2019 · 13 comments
Open

Investigate if changing the font loading mechanism is better for users #1437

nickcolley opened this issue Jun 6, 2019 · 13 comments

Comments

@nickcolley
Copy link
Contributor

@nickcolley nickcolley commented Jun 6, 2019

We currently use font-display: fallback to load our fonts:

This is somewhere in between block and swap. The text is invisible for a short period of time (100ms). Then if the custom font hasn't downloaded, the text is shown in a fallback font (for about 3s), then swapped after the custom font loads.

Matt's new font work he has suggested that we change fallback to swap, which is also recommended by performance community members, however when testing in our development environment we have noticed that when the page is switching fonts text "jumps" more than before.

Font-display: swap

After some research I have changed the font-display CSS property from fallback to swap. To quote from Addy Osmani:

...self-hosted Web Fonts have a zero second block period and an infinite swap period. Browsers would draw text immediately with a fallback used if the font face isn't loaded, but swap as soon as it does load.

More discussion about this change can be seen here, here and here.

We should determine what the difference is to users in the following scenarios, in order of priority:

  1. Users on unreliable slow network connections, with a production optimised server.
  2. Users on a reliable fast network connection, with a production optimised server.

Reference: https://font-display.glitch.me/

@Nooshu

This comment has been minimized.

Copy link
Member

@Nooshu Nooshu commented Jun 7, 2019

This sounds like a good plan. We can create a few test pages and look at the impact on perceived performance using WPT or sitespeed.io. We should also test this under both HTTP/1.1 and HTTP/2 since h2 is planned for the future (and dependent services will be a mixture of both).

aliuk2012 pushed a commit that referenced this issue Jun 7, 2019
Alistair Laing
We need to have a better understanding and test what it means for the
font-display to use `swap` instead of fallback.

A new issue has been created:
#1437
aliuk2012 pushed a commit that referenced this issue Jun 7, 2019
Alistair Laing
We need to have a better understanding and test what it means for the
font-display to use `swap` instead of fallback.

A new issue has been created:
#1437
aliuk2012 pushed a commit that referenced this issue Jun 14, 2019
Alistair Laing
We need to have a better understanding and test what it means for the
font-display to use `swap` instead of fallback.

A new issue has been created:
#1437
aliuk2012 pushed a commit that referenced this issue Jun 14, 2019
Alistair Laing
We need to have a better understanding and test what it means for the
font-display to use `swap` instead of fallback.

A new issue has been created:
#1437
@36degrees 36degrees added this to Upcoming sprints in Design System Sprint Board via automation Sep 23, 2019
@36degrees 36degrees moved this from Upcoming sprints to Sprint Backlog in Design System Sprint Board Sep 23, 2019
@nickcolley nickcolley moved this from Sprint Backlog to In progress in Design System Sprint Board Sep 24, 2019
@nickcolley nickcolley assigned nickcolley and unassigned nickcolley Sep 24, 2019
@nickcolley nickcolley moved this from In progress to Sprint Backlog in Design System Sprint Board Sep 24, 2019
@36degrees 36degrees moved this from Sprint Backlog to Upcoming sprints in Design System Sprint Board Sep 25, 2019
@kellylee-gds kellylee-gds removed this from Upcoming sprints in Design System Sprint Board Oct 3, 2019
@Nooshu

This comment has been minimized.

Copy link
Member

@Nooshu Nooshu commented Feb 21, 2020

Note

This comment overrides any comments / considerations made in PR 1607. The testing method I used in that PR was flawed so the end results weren't accurate. The method used was to run a very small number of test runs (usually 9), then base results of the median result. The limited number of runs doesn't give an accurate representation of real-world performance, and can easily introduce large variations.

The updated testing method relies on us using our own WebPageTest instance which gives us the ability to run a lot more tests than the public instance (201 vs 9). This should give us a much more accurate representation.

Hypothesis

Changing font-display: fallback to font-display: swap will improve perceived performance for users.

Reasons for Hypothesis

According to the specification:

swap - Gives the font face an extremely small block period (100ms or less is recommended in most cases) and an infinite swap period.

fallback - Gives the font face an extremely small block period (100ms or less is recommended in most cases) and a short swap period (3s is recommended in most cases).

Now although these two settings look very similar in terms of the specification, practical implementation by browsers is different:

  • With swap there is a 0s blocking period, meaning the fallback font is displayed immediately on first paint. So a user can start reading the content as soon as the page renders. The browser then gets unlimited time to swap out the fallback font for the web font once loaded.
  • With fallback the browser will wait ~100ms before using the fallback font (Arial), in that time the browser will render invisible placeholder text. This 100ms delay to view any content will have a small, but noticeable impact on perceived performance. The browser is given 3 seconds to replace the fallback font with the web font once loaded, else the fallback font will be used for the lifetime of the page.

Others have suggested changing font-display from fallback to swap too. To quote from Addy Osmani:

...self-hosted Web Fonts have a zero second block period and an infinite swap period. Browsers would draw text immediately with a fallback used if the font face isn't loaded, but swap as soon as it does load.

More discussion about this can be seen here, here and here.

Testing

I will be running 201 test runs over each of 4 GOV.UK pages on production:

The tests will be conducted at 3 connection speeds on Chrome:

  • Cable (5/1 Mbps 28ms RTT)
  • 3G (1.6 Mbps/768 Kbps 300ms RTT)
  • 3G slow (780 Kbps/330 Kbps 200ms RTT)

This should give a fairly representative idea of user experience, since we have a high percentage of users on a 3G or greater effective connection type.

Results

Homepage - Cable

homepage-cable

Test link (GDS internal only access)

Observations

  • swap: full page painted with the fallback font visible immediately at 500ms. Web font loaded at 900ms.
  • fallback: page started paint at 600ms, invisible text for 100ms the fallback font used. Web font loaded at 900ms.

Winner: swap

Homepage - 3G

homepage-3g

Test link (GDS internal only access)

  • swap: page rendered complete with fallback font at 2.9 seconds. Web font loaded at 4.4 seconds.
  • fallback: page rendered complete with fallback font at 2.8 seconds. Web font loaded at 4.4 seconds.

Winner: fallback

Homepage - 3G Slow

homepage-3gslow

Test link (GDS internal only access)

  • swap: page rendered complete with fallback text at 5.1 seconds. Web font loaded at 9.2 seconds.
  • fallback: page rendered complete with fallback text at 5.2 seconds. Web font wasn't rendered as it passed the 3 second cut-off point.

Winner: swap


Guidance page - Cable

guidance-cable

Test link (GDS internal only access)

  • swap: full page painted with the fallback font visible immediately at 600ms. Web font loaded at 1100ms.
  • fallback: page started paint at 600ms, invisible text for 100ms the fallback font used. Web font loaded at 1100ms.

Winner: swap

Guidance page - 3G

guidance-3g

Test link (GDS internal only access)

  • swap: complete page rendered with fallback text at 2.5 seconds. Web font loaded at 4.2 seconds.
  • fallback: page rendered with invisible text at 2.5 seconds. 100ms later the fallback text was rendered. Web font loaded at 4.1 seconds.

Winner: swap

Guidance page - 3G Slow

guidance-3gslow

Test link (GDS internal only access)

  • swap: page rendered complete with fallback font at 4.3 seconds. Web font loaded at 9.1 seconds.
  • fallback: page rendered with invisible text at 4.2 seconds. 100ms later fallback text rendered. Web font wasn't rendered as it passed the 3 second cut-off point.

Winner: fallback


Start page - Cable

start-cable

Test link (GDS internal only access)

  • swap: at 600ms page renders with fallback text immediately visible. Web font finished loading at 800ms.
  • fallback: initial page renders at 400ms with invisible text. At 500ms fallback test is rendered. Web font finished loading at 700ms.

Winner: fallback

Start page - 3G

start-3g

Test link (GDS internal only access)

  • swap: page rendered with fallback text immediately seen at 2.8 seconds. Web font loaded at 4.2 seconds.
  • fallback: page rendered with invisible text at 2.8 seconds. 100ms later the fallback font was rendered. Web font loaded at 4.2 seconds.

Winner: swap

Start page - 3G Slow

start-3gslow

Test link (GDS internal only access)

  • swap: page rendered with fallback font at 4.1 seconds. Web font rendered at 8.5 seconds.
  • fallback: page rendered with invisible text at 5.4s. 100ms later fallback text is displayed. Web font wasn't rendered as it passed the 3 second cut-off point.

Winner: swap


Speech page - Cable

speech-cable

Test link (GDS internal only access)

  • swap: page renders with fallback text immediately visible at 500ms. Web font loads at 800ms.
  • fallback: initial page renders with invisible text at 500ms. Fallback text visible at 600ms. Web font loads at 800ms.

Winner: swap

Speech page - 3G

speech-3g

Test link (GDS internal only access)

  • swap: page rendered with fallback text immediately seen at 2.5 seconds. Web font loaded at 4.1 seconds.
  • fallback: page rendered with invisible text at 2.5 seconds. 100ms later the fallback font was rendered. Web font loaded at 4.1 seconds.

Winner: swap

Speech page - 3G Slow

speech-3gslow

Test link (GDS internal only access)

  • swap: page rendered with fallback text immediately at 3.9 seconds. Web font loads at 9.2 seconds.
  • fallback: page rendered with invisible text at 3.9 seconds. 100ms later the fallback text was rendered. Web font wasn't rendered as it passed the 3 second cut-off point.

Winner: swap

Summary

The table below gives a summary of the results above. Which font-display property results in the best perceived performance:

Page Cable 3G 3G Slow
Homepage swap fallback swap
Guidance Page swap swap fallback
Start Page fallback swap swap
Speech Page swap swap swap

Chrome performed as per the spec:

  • swap: shows a 0s blocking period and shows fallback text immediately. The browser then waits as much time as needed to load the web fonts before swapping them out (as seen in the 3G Slow tests).
  • fallback: in almost all cases the initial page is rendered with invisible text, then 100ms later the text is replaced with the fallback text. We can see the 3 second cut-off point in action in the 3G Slow tests, where the web font is downloaded by never actually rendered.

Recommendation: change the font-display value from fallback to swap. Our users will be able to see the fallback font (Arial) immediately, without having to see FOIT for 100ms as seen with fallback.

This shouldn't hugely effect other areas since users will already be seeing the fallback font before the web font has loaded, only after 100ms rather than immediately. This also gives slow connections the chance to render the web font. Since they are already downloading it, they may as well see it at some point. If the last point is a point of contention then we should consider using font-display: optional. This shows exactly the same 100ms of invisible text as seen with fallback, only the browser has the option to choose not to download the font at all. It's worth noting that this value is hardly used across the web, so I question its effectiveness.

In regards to HTTP/1.1 vs HTTP/2, this shouldn't really come into the decision. The use of HTTP/2 will make it more likely that the web font is delivered in a decent time, but it won't change the upfront rendering as controlled by swap vs fallback.

@nickcolley

This comment has been minimized.

Copy link
Contributor Author

@nickcolley nickcolley commented Feb 21, 2020

Just to note this is an excellent investigation thanks for putting together this Matt.

This also gives slow connections the chance to render the web font. Since they are already downloading it, they may as well see it at some point.

Are we sure that if a user has been reading content for 6+ seconds and then the font changes that this is a good user experience? You could argue that the current experience where the font doesnt load if it's very slow is the least disruptive. The font will still benefit from being downloaded for future pages the user goes to.

@Nooshu

This comment has been minimized.

Copy link
Member

@Nooshu Nooshu commented Feb 21, 2020

Are we sure that if a user has been reading content for 3+ seconds and then the font changes that this is a good user experience? You could argue that the current experience where the font doesnt load if it's very slow is the least disruptive.

That's a good point, but I actually struggled to see the difference between 'nta' and 'Arial' in the tests at times. I had to focus of particular glyphs (usually 'l') to identify when the web font had loaded. So I'd hope it isn't hugely jarring between the font swap. And with version 2 of the font, now that the em-box settings now match Arial, it should be much smoother than we saw in version 1.

I guess the question is: would we prefer the majority of users to see the content instantly on paint with no invisible text and visible repaint to the fallback font (which is quite disruptive), as we have now

OR

prefer the fallback 3s cutoff to stop repainting later, that only happens for users on a 3G slow connections and below (which looks to be in the minority?

From this data, we have 0% on 2G or 2G Slow according to Effective Connection Type (ECT). I fail to believe that we have nobody in the UK on a ECT of 2G and below, but this is the only data we have to back up the decision at the moment.

Personally I'd prefer users not to see the invisible text and repaint at the start of the page load, as I feel it would be more noticeable and disruptive.

@nickcolley

This comment has been minimized.

Copy link
Contributor Author

@nickcolley nickcolley commented Feb 21, 2020

I think I would trade having a stable page that doesnt risk reflowing and moving text as our users are reading it over a very short 100ms window of time during the loading of the page which users will be used to things incrementally loading in as they browse across the web.

When it comes to perception of speed the main priority is that users see something happening, which is the same for both swap and fallback.

If people think that the risk of the text reflowing or disrupting people reading is negligible then I'm not going to block this as I agree removing the 100ms gap would be nice, however when we tested the fonts I seem to remember fallback fonts not being perfectly aligned with GDS Transport which meant that content could shift and reflow. That is something that would be annoying if you're mid way through reading a page to have it shift and lose your place.

@nickcolley

This comment has been minimized.

Copy link
Contributor Author

@nickcolley nickcolley commented Feb 21, 2020

@Nooshu

This comment has been minimized.

Copy link
Member

@Nooshu Nooshu commented Feb 21, 2020

If it is that we aren't too concerned about the first 100ms for text content, then maybe we should be considering optional?

This has is the same setup as fallback, but it hands the power back to the browser to make the decision if it should load the font at all. Would be very useful for users on a slow connection.

@nickcolley

This comment has been minimized.

Copy link
Contributor Author

@nickcolley nickcolley commented Feb 21, 2020

Is it true that optional will decide not to load the font on fast connections though? I think that may be why we avoided it? Worth considering if it doesnt make the experience on fast connections worse.

@Nooshu

This comment has been minimized.

Copy link
Member

@Nooshu Nooshu commented Feb 21, 2020

According to the spec:

An optional font must never cause the layout of the page to "jump" as it loads in. A user agent may choose to slightly delay rendering an element using an optional font to give it time to load from a possibly-slow local cache, but once the text has been painted to the screen with a fallback font instead, it must not be rendered with the optional font for the rest of the page’s lifetime.

And the example given sounds like something we are aiming for:

For example, body text is perfectly readable in one of the browser default fonts, though a downloadable font face might be more attractive and mesh with the site’s aesthetics better. First time visitors to a site generally care far more about the site being quickly usable than they do about the finer points of its display, and optional provides a good behavior for them. If they return later, the desired font faces might have finished downloading, giving them the "intended" experience without slowing down either their first or subsequent visits.

Users on very slow connections might not ever receive the "intended" experience, but optional ensures they can actually use the site, rather than quitting and going elsewhere because the site takes too long to load.

We can certainly test the results on simulated Cable / 4G / 3G connections to identify what happens.

@Nooshu

This comment has been minimized.

Copy link
Member

@Nooshu Nooshu commented Feb 24, 2020

I've just investigated the impact of font-display: optional on the font loading:

Cable connection

4G connection

3G connection

So out of the 1200 test runs the web font wasn't rendered on initial page load. Of the 1200 repeat view tests, the web font was displayed every time. So optional is very agressive on first load and doesn't allow the web font to be swapped out once the fallback test has rendered. But on page reload (or page navigation) the font is then visible as it has been cached.

So here's where we stand for all font-display settings:

Font display Notes Use this?
auto Browser, default. Most likely block.
swap Immediate render but will swap out the font at any time so can be disruptive to users.
fallback Render after 100ms, will stop the web font rendering after 3 seconds. Better for very slow connections (3G Slow and below)
optional Stops the rendering of the web font on initial load (even on fast connections). Web font only seen on repeat view.
block Invisible text rendered on page load. Text not displayed until web font has loaded.

At the moment it is difficult to decide which setting is best for users without more information about their connection speeds. Decision: ??

@nickcolley

This comment has been minimized.

Copy link
Contributor Author

@nickcolley nickcolley commented Feb 24, 2020

@Nooshu do you know if Chrome's recent work to avoid pages jumping when images load could prevent the issues I'm worried about?

@36degrees

This comment has been minimized.

Copy link
Member

@36degrees 36degrees commented Feb 25, 2020

We could revisit font-size-adjust to see if we can match the x-height of Arial to Transport, which might help reduce the reflow effect?

(This was previously implemented in alphagov/govuk_frontend_toolkit#271 but had to be reverted due to the way we implemented tabular numbers – see alphagov/govuk_frontend_toolkit#300)

@nickcolley

This comment has been minimized.

Copy link
Contributor Author

@nickcolley nickcolley commented Feb 25, 2020

Definitely worth a think about regardless of this pull request, since even with fallback there's a smaller window of time where the font can replace a fallback.

That feature seems to have minimal support in firefox: https://caniuse.com/#search=font-display when compared with the font-display feature: https://caniuse.com/#search=font-size-adjust

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.