Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize YouTube embed rewriting? #2390

Open
RByers opened this issue Feb 23, 2017 · 10 comments
Open

Standardize YouTube embed rewriting? #2390

RByers opened this issue Feb 23, 2017 · 10 comments
Labels
compat Standard is not web compatible or proprietary feature needs standardizing interop Implementations are not interoperable with each other topic: embed and object topic: media

Comments

@RByers
Copy link

RByers commented Feb 23, 2017

Apparently chromium, Gecko and WebKit all have code for rewriting YouTube flash embeds to use HTML instead (so they work when flash is disabled, such as on mobile):

If this is now effectively required for web compat, perhaps we should work to align our implementations and specify it in HTML just to make this a little more predictable / sane? For automated testing purposes, perhaps we should include in the definition an additional hostname like youtube.web-platform.test?

@mounirlamouri can you share any data on how much of the web benefits from this? I imagine there may be other cases where we want to ask whether this is worth doing. So we may be setting a precedent here. I assume we'd only ever consider something like this when it's has truly massive positive impact on the user experience. I think it's worth having some discussion on what magnitude of usage qualifies (eg. as a fraction of page views, and/or a fraction of HTTP Archive pages).

@annevk annevk added compat Standard is not web compatible or proprietary feature needs standardizing topic: media labels Feb 23, 2017
@mounirlamouri
Copy link
Member

According to our data, this is affecting somewhere between 1-3% of our users on desktop and less than that on mobile (probably half). It's not huge because it's mostly for old websites, which is also the reason why mobile is less affected.

Is there any other data you would like to see?

@zcorpan
Copy link
Member

zcorpan commented Feb 24, 2017

From https://nerdydata.com/ ("We index approximately 160 million websites") searching in "Deep Web":

src="//www.youtube.com/v/
Showing 1 - 50 (of approx. 22,090 found)

src="http://www.youtube.com/v/
Showing 1 - 50 (of approx. 281,530 found) 

src="https://www.youtube.com/v/
Showing 1 - 50 (of approx. 151,810 found) 

data="//www.youtube.com/v/
Showing 1 - 50 (of approx. 3,290 found) 

data="http://www.youtube.com/v/
Showing 1 - 50 (of approx. 28,200 found) 

data="https://www.youtube.com/v/
Showing 1 - 50 (of approx. 1,175 found) 

Adding these together and assuming the worst case (1 match is 1 site), that is ~0.3%.

@zcorpan
Copy link
Member

zcorpan commented Feb 24, 2017

For comparison, we have <image>:

   <dt>A start tag whose tag name is "image"</dt>
   <dd>
    <!-- As of 2005-12, studies showed that around 0.2% of pages used the <image> element. -->
    <p><span>Parse error</span>. Change the token's tag name to "img" and reprocess it. (Don't
    ask.)</p>
   </dd>

@RByers
Copy link
Author

RByers commented Feb 24, 2017

Thanks guys! As expected, this is pretty compelling data that this is important enough to justify some sort of fix. Here's some HTTP Archive data about the popularity of different embed origins/types (of the top 500k sites). So YouTube is present on ~0.27% of top sites, with histats.com behind that at about a tenth of that at 0.029%.

Rank	Origin	Type	Page count
1	http://www.youtube.com	application/x-shockwave-flash	1087	 
2	https://www.youtube.com	application/x-shockwave-flash	234	 
3	http://s10.histats.com	application/x-shockwave-flash	143	 
4	http://player.youku.com	application/x-shockwave-flash	102	 
5	http://www.xatech.com	application/x-shockwave-flash	98	 
6	http://	application/x-shockwave-flash	71	 
7	http://www.clocklink.com	application/x-shockwave-flash	69	 
8	http://blog.naver.com	application/x-shockwave-flash	66	 
9	http://www.dailymotion.com	application/x-shockwave-flash	61	 
10	http://img.i2i.jp	application/x-shockwave-flash	49	 
11	http://www.	application/x-shockwave-flash	49	 
12	http://www.smartadserver.com	application/x-shockwave-flash	48	 
13	http://home.uchat.co.kr	application/x-shockwave-flash	45	 
14	http://content.pop6.com	application/x-shockwave-flash	43	 
15	http://embed.scribblelive.com	application/x-shockwave-flash	41	
16	http://www.ustream.tv	application/x-shockwave-flash	32	 
17	http://chat.53kf.com	application/x-shockwave-flash	31	 
18	http://www.flickr.com	application/x-shockwave-flash	28	 
19	http://static-hw.xvideos.com	application/x-shockwave-flash	27	 
20	http://static.video.qq.com	application/x-shockwave-flash	27	 
21	http://pic.img80.com	application/x-shockwave-flash	26	 
22	http://vimeo.com	application/x-shockwave-flash	26	 
23	http://s3.rotaban.ru	application/x-shockwave-flash	24	 
24	https://embed-ssl.wistia.com	application/x-shockwave-flash	23	 
25	http://s4.rotaban.ru	application/x-shockwave-flash	23	 
26	http://h2.flashvortex.com	application/x-shockwave-flash	21	 
27	http://www.youtube-nocookie.com	application/x-shockwave-flash	19	 
28	http://c.brightcove.com	application/x-shockwave-flash	19	 
29	http://ws.amazon.com	application/x-shockwave-flash	19	 
30	http://localtimes.info	application/x-shockwave-flash	19	 

Query:

SELECT 
  REGEXP_EXTRACT(body, r'<embed[^>]*src=[\'"]?(http[s]?://[^/ "\']*)') AS embedsrc,
  REGEXP_EXTRACT(body, r'<embed[^>]*type=[\'"]?([^ "\']*)') AS type,
  COUNT(DISTINCT bodies.page) as count
  FROM [httparchive:har.2017_01_15_chrome_requests_bodies] 
  AS bodies  
JOIN (
  SELECT 
    page, url,
    JSON_EXTRACT(payload, '$._contentType') AS contentType
  FROM [httparchive:har.2017_01_15_chrome_requests]
) AS requests ON requests.url=bodies.url AND requests.page=bodies.page
WHERE REGEXP_MATCH(body, r'<embed[^>]*src=[\'"]?http[s]?://')
GROUP BY embedsrc, type
ORDER BY count DESC
LIMIT 5000

@rniwa
Copy link
Collaborator

rniwa commented Feb 25, 2017

I would be surprised this was required for compatibility.

@miketaylr
Copy link
Member

I would be surprised this was required for compatibility.

If there's no Flash plugin on the device or platform, this is how you get the embedded videos to keep working, right?

@zcorpan
Copy link
Member

zcorpan commented Feb 27, 2017

Couldn't most of the "rewriting" be done by YouTube instead of in the browser engine, though? That is, if the only hack in the browser engine is type="application/x-shockwave-flash" -> type="text/html" for youtube.com/v/.

(And maybe eventually, do such type rewriting for any URL...?)

@RByers
Copy link
Author

RByers commented Mar 2, 2017

Yeah, we've been discussing a bit a new generic web platform feature that could enable YouTube (and everyone else in a similar position) to solve this problem themselves (basically upgrading a flash embed to an iframe). It's not clear to me what the exact requirements would be for YouTube to be able to use such a feature. If people here feel that is worth pursuing, I can see if we can find someone at Google to drive this.

@qdot
Copy link

qdot commented Sep 8, 2017

Update: @mounirlamouri has pointed out that flash embeds are now completely deprecated by youtube. We've still got per-browser solutions to rewriting. Should we revisit this?

@annevk annevk added the interop Implementations are not interoperable with each other label Sep 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compat Standard is not web compatible or proprietary feature needs standardizing interop Implementations are not interoperable with each other topic: embed and object topic: media
Development

No branches or pull requests

7 participants