Standardize YouTube embed rewriting?

[RByers]

Apparently chromium, Gecko and WebKit all have code for rewriting YouTube flash embeds to use HTML instead (so they work when flash is disabled, such as on mobile):

• Chromium: ChromeContentRendererClient::OverrideFlashEmbedWithHTML (discussion)
• Gecko: nsObjectLoadingContent::MaybeRewriteYoutubeEmbed (bug)
• WebKit: YouTubePluginReplacement

If this is now effectively required for web compat, perhaps we should work to align our implementations and specify it in HTML just to make this a little more predictable / sane? For automated testing purposes, perhaps we should include in the definition an additional hostname like youtube.web-platform.test?

@mounirlamouri can you share any data on how much of the web benefits from this? I imagine there may be other cases where we want to ask whether this is worth doing. So we may be setting a precedent here. I assume we'd only ever consider something like this when it's has truly massive positive impact on the user experience. I think it's worth having some discussion on what magnitude of usage qualifies (eg. as a fraction of page views, and/or a fraction of HTTP Archive pages).

[mounirlamouri]

According to our data, this is affecting somewhere between 1-3% of our users on desktop and less than that on mobile (probably half). It's not huge because it's mostly for old websites, which is also the reason why mobile is less affected.

Is there any other data you would like to see?

[zcorpan]

From https://nerdydata.com/ ("We index approximately 160 million websites") searching in "Deep Web":

src="//www.youtube.com/v/
Showing 1 - 50 (of approx. 22,090 found)

src="http://www.youtube.com/v/
Showing 1 - 50 (of approx. 281,530 found) 

src="https://www.youtube.com/v/
Showing 1 - 50 (of approx. 151,810 found) 

data="//www.youtube.com/v/
Showing 1 - 50 (of approx. 3,290 found) 

data="http://www.youtube.com/v/
Showing 1 - 50 (of approx. 28,200 found) 

data="https://www.youtube.com/v/
Showing 1 - 50 (of approx. 1,175 found) 

Adding these together and assuming the worst case (1 match is 1 site), that is ~0.3%.

[zcorpan]

For comparison, we have <image>:

   <dt>A start tag whose tag name is "image"</dt>
   <dd>
    <!-- As of 2005-12, studies showed that around 0.2% of pages used the <image> element. -->
    <p><span>Parse error</span>. Change the token's tag name to "img" and reprocess it. (Don't
    ask.)</p>
   </dd>

[RByers]

Thanks guys! As expected, this is pretty compelling data that this is important enough to justify some sort of fix. Here's some HTTP Archive data about the popularity of different embed origins/types (of the top 500k sites). So YouTube is present on ~0.27% of top sites, with histats.com behind that at about a tenth of that at 0.029%.

Rank	Origin	Type	Page count
1	http://www.youtube.com	application/x-shockwave-flash	1087	 
2	https://www.youtube.com	application/x-shockwave-flash	234	 
3	http://s10.histats.com	application/x-shockwave-flash	143	 
4	http://player.youku.com	application/x-shockwave-flash	102	 
5	http://www.xatech.com	application/x-shockwave-flash	98	 
6	http://	application/x-shockwave-flash	71	 
7	http://www.clocklink.com	application/x-shockwave-flash	69	 
8	http://blog.naver.com	application/x-shockwave-flash	66	 
9	http://www.dailymotion.com	application/x-shockwave-flash	61	 
10	http://img.i2i.jp	application/x-shockwave-flash	49	 
11	http://www.	application/x-shockwave-flash	49	 
12	http://www.smartadserver.com	application/x-shockwave-flash	48	 
13	http://home.uchat.co.kr	application/x-shockwave-flash	45	 
14	http://content.pop6.com	application/x-shockwave-flash	43	 
15	http://embed.scribblelive.com	application/x-shockwave-flash	41	
16	http://www.ustream.tv	application/x-shockwave-flash	32	 
17	http://chat.53kf.com	application/x-shockwave-flash	31	 
18	http://www.flickr.com	application/x-shockwave-flash	28	 
19	http://static-hw.xvideos.com	application/x-shockwave-flash	27	 
20	http://static.video.qq.com	application/x-shockwave-flash	27	 
21	http://pic.img80.com	application/x-shockwave-flash	26	 
22	http://vimeo.com	application/x-shockwave-flash	26	 
23	http://s3.rotaban.ru	application/x-shockwave-flash	24	 
24	https://embed-ssl.wistia.com	application/x-shockwave-flash	23	 
25	http://s4.rotaban.ru	application/x-shockwave-flash	23	 
26	http://h2.flashvortex.com	application/x-shockwave-flash	21	 
27	http://www.youtube-nocookie.com	application/x-shockwave-flash	19	 
28	http://c.brightcove.com	application/x-shockwave-flash	19	 
29	http://ws.amazon.com	application/x-shockwave-flash	19	 
30	http://localtimes.info	application/x-shockwave-flash	19	 

Query:

SELECT 
  REGEXP_EXTRACT(body, r'<embed[^>]*src=[\'"]?(http[s]?://[^/ "\']*)') AS embedsrc,
  REGEXP_EXTRACT(body, r'<embed[^>]*type=[\'"]?([^ "\']*)') AS type,
  COUNT(DISTINCT bodies.page) as count
  FROM [httparchive:har.2017_01_15_chrome_requests_bodies] 
  AS bodies  
JOIN (
  SELECT 
    page, url,
    JSON_EXTRACT(payload, '$._contentType') AS contentType
  FROM [httparchive:har.2017_01_15_chrome_requests]
) AS requests ON requests.url=bodies.url AND requests.page=bodies.page
WHERE REGEXP_MATCH(body, r'<embed[^>]*src=[\'"]?http[s]?://')
GROUP BY embedsrc, type
ORDER BY count DESC
LIMIT 5000

[rniwa]

I would be surprised this was required for compatibility.

[qdot]

Our firefox telemetry had it at somewhere around 1% of users also when we implemented embed rewriting. It's now down to about 0.5% on our release branch.

https://telemetry.mozilla.org/new-pipeline/dist.html#!cumulative=0&end_date=2017-01-25&keys=__none__!__none__!__none__&max_channel_version=release%252F51&measure=YOUTUBE_REWRITABLE_EMBED_SEEN&min_channel_version=null&processType=*&product=Firefox&sanitize=1&sort_keys=submissions&start_date=2017-01-18&table=0&trim=1&use_submission_date=0

[miketaylr]

I would be surprised this was required for compatibility.

If there's no Flash plugin on the device or platform, this is how you get the embedded videos to keep working, right?

[zcorpan]

Couldn't most of the "rewriting" be done by YouTube instead of in the browser engine, though? That is, if the only hack in the browser engine is type="application/x-shockwave-flash" -> type="text/html" for youtube.com/v/.

(And maybe eventually, do such type rewriting for any URL...?)

[RByers]

Yeah, we've been discussing a bit a new generic web platform feature that could enable YouTube (and everyone else in a similar position) to solve this problem themselves (basically upgrading a flash embed to an iframe). It's not clear to me what the exact requirements would be for YouTube to be able to use such a feature. If people here feel that is worth pursuing, I can see if we can find someone at Google to drive this.

[qdot]

Update: @mounirlamouri has pointed out that flash embeds are now completely deprecated by youtube. We've still got per-browser solutions to rewriting. Should we revisit this?

[karlcow]

Is there a counter somewhere for

<embed src="https://www.youtube.com/…"
       type="application/x-shockwave-flash"
…