Audit: Serve <video> instead of animated GIFs

[addyosmani]

Suggested audit
Serve <video> instead of animated GIFs

Background

Animated GIFs are suboptimal for quality and performance. While they are a popular option for creative expression, they are awful for web performance - they're large in size, impact cellular data bills, require additional CPU and memory, cause repaints and kill battery. Often, GIFs can be up to 10-12x larger than H.264 videos and take more energy to load and be displayed in mobile browsers. Those resources are being spent on something that has severe color limitations.

Using the <video> tag is significantly better for performance than animated GIFs. GIF sites have been shipped videos down instead of GIFs for years and back in 2014, Twitter even added animated GIF support using MP4s. They transcoded GIFs to MP4 on the fly, delivering them in <video> tags. This is a trend we're seeing, but are still running into plenty of examples of sites accidentally shipping megs of animated GIFs down. The AMP team recently published a blog post that included 52MB of animated GIFs without realizing this.

Explanation of how it’s different from other audits

Current audits will capture unoptimized images and resources with large payloads but do not specifically call out animated GIFs that could be served down as <video> instead. This is a more targeted recommendation than current audits expose.

GIFs alone would not be flagged by this audit (we probably don't want to flag 1x1 pixel tracking pixels..) so detecting if an image is an animated GIF using animated-gif-detector, is-animated or detectanimation might be worthwhile.

We've been considering implementing a feature policy for this idea so having something related in Lighthouse could be useful.

What % of developers/pages will this impact (estimates OK, data points preferred)

According to Android stable, GIFs account for approximately 4-5% of images loaded by Chrome. Although we do not have statistics on animated GIFs as a subset of this population, shifting animated GIFs over to video has the potential to impact a large number of requests on the web.

According to HTTP Archive, GIFs account for 23% of the total image formats being served on sites tracked by HA.

Advanced

If we wanted to do something really clever here, we could use FFMPEG.js (or similar) to try encoding animated GIFs as videos to demonstrate the byte savings, however, this is probably going to be too expensive for most pages.

What is the resourcing situation (who will create the audits, maintain the audits, and write/maintain the documentation)

This is an audit that could be authored by our Lighthouse contractor @wardpeet with input from myself and the image compression team (and Bonsai team). We have staffing to assist with documentation if needed and are planning to write up a best practice doc on using <video> vs animated GIFs for /web.

Do you envision this audit in the Lighthouse report or the full config in the CLI? If in the report, which section?

The default Lighthouse report. I would place this in the Opportunities section alongside other image optimization opportunities.

How much support is needed from the Lighthouse team?

Evaluation of the technical approach for the audit will be needed. Although there are OSS modules available to assist with detecting whether an image is an animated GIF, these may be 1) too large/unsuitable, 2) require additional thought on the performance of the solution for a page with a number of animated GIFs present.

Note: I noticed this audit was also filed in #2586. I am refiling here using the "new audits criteria" doc that the team published in the docs repo.

[malchata]

+1 to @addyosmani for this idea. This is one area of performance optimization that can have a significant impact. This involves a bit more than simply checking for GIFs, as there are static and animated GIF images. This is the process I would attempt (attempt, mind you, I've never written a Lighthouse audit before):

1. Scan the DOM for <img> elements with src or srcset attributes ending in .gif.
2. Examine the referenced image. Multiple images are delimited in animated GIFs by an octet of 2C: http://giflib.sourceforge.net/whatsinagif/bits_and_bytes.html#image_descriptor_block
3. If multiple images are found, add them to the audit.

I would probably also consider having a separate audit to check if videos sources include a WebM version. But that's a different story for a different time.

[wardpeet]

Just going to write a possible solution for this problem. Feel free to change/improve the solution.

Idea here is to fetch all rendered gif images (so use currentSrc if it's a srcset). In the browser we should render all these images inside canvas so we can get the Buffer from getImageData . When we get the buffer we can use any js lib we want to detect a gif. (I like the simplicity of https://gist.github.com/marckubischta/261ad8427a214022890b)

The gatherer gets all animated gifs and the audit just does the comparison. Not sure how long canvas and getImageData take to load so the gatherer might take some time.

[patrickhulce]

+1 on the overall audit, love the concept and huge potential savings to surface! ❤️

Quick note on implementation cost, if there's a hard requirement on examining the buffer to determine animation, it will likely need to be implemented on the Chrome side if we want it in the default run as we had to abandon the earlier approach to optimize images for this same reason (and those weren't even hopping over the protocol!).

A suggestion for a workaround, we should be flagging GIFs as an inferior format no matter how they're used! :) so how about the audit flags all GIFs above a reasonable size, suggesting webm, and documenting that if they don't have animation, consider PNG/WebP/etc.

The original reason we filter out GIFs from our next-gen formats audit at all is specifically because they might contain animation, so addressing this kills two birds with one stone 👍

[wardpeet]

@patrickhulce what reasonable size are we talking about? 1MB? Also would it be lots of work to introduce it in the protocol? If need be I could take a stab at it, just need someone to put me in the right direction but it might not be worth it.

[patrickhulce]

what reasonable size are we talking about? 1MB?

Oh much lower, I was talking about something more inline with our existing ignore thresholds (~4KB or so). Just to address the tracking pixel case @addyosmani highlighted.

Also would it be lots of work to introduce it in the protocol?

Depends on how comfortable you are with C++ ;) The actual code shouldn't be that bad if all we're doing is determining if the GIF is animated. Honestly though given we should flag all GIFs regardless I personally don't think it's worth the effort when the user should easily be able to tell if they need to make it webm or png on their own.

[wardpeet]

fine by me. I'm not comfortable at C++. I've played with it 5 years ago but you can't call it programming. I'm a pretty good script kiddy, copy and pasting + adapting. I just thought maybe I could implement the progressive jpg one at the same time but i'm all for leaving that for another day. As I think we can just start with flagging all GIFs

[addyosmani]

Thanks for the context on having to fall back into C++ if we wanted to forge ahead with this path, @patrickhulce. That's very useful to know!.

A suggestion for a workaround, we should be flagging GIFs as an inferior format no matter how they're used! :) so how about the audit flags all GIFs above a reasonable size, suggesting webm, and documenting that if they don't have animation, consider PNG/WebP/etc.

Flagging GIFs as an inferior format (factoring in some minimal byte threshold) sounds pretty reasonable to me if we think the cost of doing something more advanced is not worth it. I have a few follow-up questions if we head down this path instead of detecting if a GIF is animated:

1. Is there enough meat here for this to be a separate audit for GIFs?
2. Should this be merged into the existing "next-gen image formats" audit?

The one point I want to ensure we convey is that <video>/WebM should be considered instead of animated GIFs. My main question with 2) is whether we think we have enough flexibility with the text to convey this (alongside WebP/JPEG2000 etc).

Also interested in whether @paulirish or @brendankenny have input on the direction here.

[paulirish]

Is there enough meat here for this to be a separate audit for GIFs?

Yes. I'm a fan of a dedicated audit for "Use better formats than animated GIF".

Should this be merged into the existing "next-gen image formats" audit?

Nah, because it's about video typically. Plus this is actionable in different ways than the next-gen one. You can serve video to all browsers, no UA or request headers necessary.

Lastly, +1 to implementing this natively, either a protocol method or adding details in trace events. jk jk, we can just use network records for this. based on me and patrick's research, it appears contentType + fileSize is all we need. We can use a function to estimate savings. Patrick is about to comment with everything I'm saying right now...

[patrickhulce]

Did a bit of digging in HTTP archive stats around this and here's what I've found.

• Approx GIFs: ~5M
• Approx GIFs over 4kb (our threshold for savings): ~507k

Already nearly all seem to be tracking pixels, of the remaining I downloaded a random sample of ~100 and did some analysis. No GIF <10kb had savings above our thresholds, so if we just focus on those...

• Approx GIFs over 10kb: ~314k
• Approx % of GIFs over 10kb that are animated: ~64%
• Average % savings available on GIFs over 10kb by converting to webm: ~66%

In general the larger the GIF the greater percentage that can be saved, probably since compression overhead is dwarfed by compression savings i.e. ~2.4mb GIF with large dimensions few frames becomes ~220k webm but ~100k GIF with small dimensions many frames becomes ~50k webm.

Relevant code snippets

SELECT url FROM [httparchive:requests.2018_02_01_desktop] 
WHERE JSON_EXTRACT_SCALAR(payload, "$._contentType") = 'image/gif' 
  AND INTEGER(JSON_EXTRACT_SCALAR(payload, "$.response.bodySize")) > 4000 
  AND RAND() < 500/500000
LIMIT 1000

cat ./gifs-list-from-httparchive.csv | xargs -n 1 curl -O
brew install ffmpeg --with-libvpx
for f in *.gif; do ffmpeg -i "$f" -c:v libvpx -crf 12 -b:v 500K -auto-alt-ref 0 "$f.webm"; done
npm install file-type is-animated
node do-the-cool-analysis.js

var fs = require('fs')
var fileType = require('file-type')
var isAnimated = require('is-animated')

let totalAnimated = 0
let totalNon = 0
let animatedSize = []
let nonSize = []
let savings = []
fs.readdirSync(__dirname).forEach(x => {
  try {
  const buffer = fs.readFileSync(x)
  const type = fileType(buffer)
  if (!type || !/image/.test(type.mime)) return
  if (buffer.length < 10000) return

  if (isAnimated(buffer)) {
const webmBuffer = fs.readFileSync(x + '.webm')
  savings.push({name: x, orig: buffer.length, saved: buffer.length - webmBuffer.length})
  animatedSize.push(buffer.length)
totalAnimated++
  console.log('ANIMATED!!')
} else {
nonSize.push(buffer.length)
totalNon++
 console.log('NOT ANIMATED!!')
}  console.log(x, type.mime)

  } catch (err) {
  console.error(err)
}
})

let filteredSav = []
for (const entry of savings) {
  if (entry.saved < 4000) continue
  if (entry.orig < 100 * 1000) continue
  const percent = entry.saved / entry.orig
  console.log(percent)
  filteredSav.push(percent)
}

animatedSize = animatedSize.sort((a,b) => a - b)
nonSize = nonSize.sort((a,b) => a -b)
savings = savings.sort((a,b)=> a.saved - b.saved)
console.log({animatedSize, nonSize, savings, totalAnimated, totalNon})
console.log(filteredSav.reduce((a,b)=>a+b,0) / filteredSav.length)
~

---

In response to your question @addyosmani

I definitely think there's enough evidence here to say we should have a separate audit to flag them for conversion to webm. Additionally, we see that large GIFs are most commonly used for animated content (~64% is good enough in my book in this case 😄) and, from my perspective, could be flagged by the audit even if they're not animated at the beginning.

I'd propose the following approach to not block this on Chrome-side work:

1. Implement an audit that identifies GIFs from network records by contentType and flags all GIFs over 10kb with some title "Use a video format for animated content" (or something much better someone comes up with 😄), description calls out that if your GIF is not animated consider PNG/WebP
2. Conduct broader analysis on GIF savings to WebM to get rough estimate of savings based on dimensions + file size.
3. (Later when capacity allows) implement Chrome-side instrumentation/violation for animated GIFs that will allow this audit to filter out non-animated GIFs and allow the webp audit to start flagging them instead.

@paulirish @brendankenny does this capture what we discussed as well?

[wardpeet]

@patrickhulce not 100% sure about how to work on bullet 2. How should I analyse this? (I don't mind doing it). I can start with bullet 1.

[addyosmani]

@wardpeet If this direction is fair by @paulirish and @brendankenny, let's make a start on bullet 1 and follow up with 2 and 3 once the initial implementation for the GIF identification audit is written up. Does that sound reasonable?

[wardpeet]

For me it does :)

[paulirish]

Some followup research that I did based on Patrick's initial run: https://docs.google.com/spreadsheets/d/1eeY1OZE_P1ilXyHtTEMNzg9v5M7d5ElZh4h6cK5_Lx0/edit?usp=sharing

Looked at 1020 real GIFs sourced from HTTPA. We used this to determine the 100kb threshold.

Based on the data, brendan eyeballed out a rough estimation of savings for gifs above 100kb.

getPercentSavings = bytes => 29.1 * Math.log10(bytes) - 100.7

[abdonrd]

I was trying to follow this recommendation.
I also read Replace Animated GIFs with Video by @malchata.
But it seems that it does not work well on mobile phones, right?

• Example: https://jsfiddle.net/L3u6zg9j/
• Example result: https://jsfiddle.net/L3u6zg9j/embedded/result/

Demo with a Google Pixel:
The video does not autoplay. The video starts if I click on it (you can see the script). Also, you can see the speaker icon in the status bar (despite muted) & the Cast icon on top of the video.

---

Edit 1: Cast button removed with disableRemotePlayback.

• Example result: https://jsfiddle.net/L3u6zg9j/5/embedded/result/

---

Edit 2: Now I just tried with an iPhone 6S Plus, and it works well:

[addyosmani]

Could you file a new bug report with this feedback? Fyi both links above autoplay for me in Chrome 67. I'm curious if any recent changes in site engagement scoring or otherwise are causing issues here.

[mathiasbynens]

I can reproduce this for both fiddles in Chrome 67.0.3396.87 on my Pixel phone. (It's a Pixel 1.)

Both fiddles work fine in Chrome Canary 69.0.3481.0 on the same Pixel phone.

Update: Ugh, it was the Data Saver!

[malchata]

No issues on desktop in any browsers for me on macOS. But I can't get autoplay to work on any browser in iOS.

My test: https://jlwagner.net/ext/vid2.html

Edit: macOS chrome version is 69.0.3477.0. Tested on latest stable Chrome on iOS, Safari, and UC Browser.

[abdonrd]

@addyosmani Chrome 67 on Android works for you? 😮

I have used Stable & Canary:

---

About the new bug? Here https://crbug.com/wizard?

[malchata]

@abdonrd IIRC, autoplaying of videos may be blocked if you have data saver turned on. Try turning that off if you have it enabled.

[abdonrd]

@malchata ouch! Is that! The data saver was on! 🤦‍♂️

[mathiasbynens]

@malchata I can confirm Data Saver was the culprit. Turning it off makes the fiddles work as expected even on Chrome 67.

[abdonrd]

@malchata @mathiasbynens exactly! Both demos works now with the Data Saver disabled.

[addyosmani]

Thanks for helping us debug this folks. Looks like we might want to better document checking for data saver mode being on when trying this recommendation :)

[malchata]

I'll update the relevant guide today, @addyosmani.

[abdonrd]

Thank you all! 🙂

It would be great to mention the data saver on the article, yes!

[abdonrd]

@malchata I would also like to propose that disableRemotePlayback be mentioned.

<video autoplay loop muted playsinline disableRemotePlayback></video>

<video autoplay loop muted playsinline disableRemotePlayback>
  <source src="oneDoesNotSimply.webm" type="video/webm">
  <source src="oneDoesNotSimply.mp4" type="video/mp4">
</video>

[malchata]

@addyosmani, any thoughts on disableRemotePlayback as @abdonrd has mentioned here? I'm not overly familiar with it and how it may affect autoplay.

[abdonrd]

I don't know if it affects something else, but if we don't add disableRemotePlayback , we will have two things that I think we should not have if we want to simulate a GIF:

• Cast button on top of the video
• Chrome speaker notification

[abdonrd]

And what is the fallback recommendation if the data saver is enabled? Just the poster, right?

<video autoplay loop muted playsinline disableRemotePlayback poster="oneDoesNotSimply.png">
  <source src="oneDoesNotSimply.webm" type="video/webm">
  <source src="oneDoesNotSimply.mp4" type="video/mp4">
</video>

[midzer]

@abdonrd I've experienced letting browser load preview image via (now Chromium default) preload="metadata" to be less kbyte heavy in most cases than loading a custom poster.

[huygn]

hey @midzer I am interested in your method (I added preload="metadata" but didn't see any preview image. on Chrome 67), would you mind to share a short snippet to demonstrate? Thanks in advance 👏

[midzer]

Hmm, strange. What browser are you using, @huygn ?

Using preload="metadata" is pretty straightforward. You can check my snippet here https://github.com/midzer/eisolzried/blob/master/_includes/video.html

For more info https://www.html5rocks.com/en/tutorials/video/basics/

[patrickhulce]

Seems like we should file this as a bug against data saver :) Blindly loading large GIFs while blocking the smaller video versions is incompatible with a data saving mindset.

Also, I seem to be able to avoid the data saver issue somehow when the video is standalone and not in a <source>, i.e. <video src="https://storage.googleapis.com/webfundamentals-assets/videos/chrome.mp4" autoplay loop muted></video> still autoplays.

Is this true for anyone else? That might be another workaround if so.

[addyosmani]

Seems like we should file this as a bug against data saver :) Blindly loading large GIFs while blocking the smaller video versions is incompatible with a data saving mindset.

If it's useful, happy to kick off a thread with the Data Saver team to discuss.

Is this true for anyone else? That might be another workaround if so.

Hmm. Edge case in their heuristics? :) This appears to work fine for me with data saver on.