Support for multimedia embeds

[brookerose1312]

Does not display all media in the post
https://twitter.com/punsandbunsgg/status/1578095299460943872

This feature was added a few days ago, but I just wanted to make the issue so that you had a ticket to work off of

[dangeredwolf]

Unfortunately due to technical limitations this is unlikely to be possible with current OpenGraph standards.

[Deer-Spangle]

I guess the way to solve this would be to combine them all into one video with ffmpeg or something?.. But maybe that's outside the ability of a cloudflare worker? Maybe the mosaic server could do it?

[dangeredwolf]

Oh 100% impossible for a worker, and in fact I can't see this being feasible at all.

Like Cloudflare Workers already don't have enough compute time to handle compositing images within their allotted CPU time. So video is an absolute no-go for workers. Just like photos also didn't make sense.

And even for self-hosting like we do for Mosaic, doing it for video will consume way more bandwidth and compute to the point where I'd probably exceed both my VPS and/or Cloudflare bandwidth caps.

Furthermore, Cloudflare's Terms of Use technically forbids Video streaming without paying for a separate plan. They don't care too much for small videos, for a small amount of visitors, on small sites, but implementing this in something the scale of FixTweet would basically guarantee require forking over more money to Cloudflare. An effect that could stack if Cloudflare gets wary about the amount of bandwidth used as well.

Lastly, while it would make a cool demo, it's not a practical thing users would want implemented that way. Like, why would you want multiple videos stacked next to each other if their audio will drown each other out making certain content measurably worse than if only a single video were displayed. Or videos play one after the other and you can scrub along the main video to scrub parts of each video. But suddenly the bandwidth required per video shoots up even more when it already can't be handled.

I cannot see this working unless platforms and/or OpenGraph come up with a way to attach multiple kinds of media as I cannot envision a present-day solution that makes sense at scale and even would appeal to users.

[Deer-Spangle]

Ahh, okay, yeah, sounds not so possible then.

Like Cloudflare Workers already don't have enough compute time to handle compositing images within their allotted CPU time. So video is an absolute no-go for workers.

That's fair enough! It would certainly be some processing power, and add quite some delay. I imagine the telegram/discord web preview workers don't give long to reply before they just give up and display no preview?

Cloudflare's Terms of Use technically forbids Video streaming without paying for a separate plan.

I feel like video streaming is a separate matter to video hosting, the former requiring you send in blocks and all? Twitter limits video lengths to a minute or something, so sending the whole thing as one file seems like it might be okay? But might not be, I dunno!

Like, why would you want multiple videos stacked next to each other if their audio will drown each other out making certain content measurably worse than if only a single video were displayed.

Ahhh, I wasn't sure whether multiple videos with audio tracks could be posted in one tweet? I've only seen it used with one video and 3 photos, or a video and a gif and a photo, or something. Never more than one video and/or one gif.
Though, checking the twitter announcement tweet now (Frustrating that they don't put these announcements in any of their release notes or change logs !!)
I can see in the replies, someone has posted 4 videos in a tweet: https://twitter.com/jaayro/status/1578194246229643265
I see twitter implements that by playing one video, then the next, then the next. I guess that would be theoretically implementable, if not for the cloudflare and processing power restrictions

I cannot see this working unless platforms and/or OpenGraph come up with a way to attach multiple kinds of media as I cannot envision a present-day solution that makes sense at scale and even would appeal to users.

That would be absolutely dreamy! I think the Arrays section of the OpenGraph protocol site implies that a site can have multiple images (though it's a bit unclear whether it's meant to be used for multiple resolutions of the same image), but telegram just doesn't seem to support that, which is incredibly frustrating.
Sure would be cool if Telegram was actually open source, like they claim to be 🙃 Then we could just fix it at the root

[dangeredwolf]

Telegram and Discord don't currently support image arrays afaik so it'd just be vendor implementation then

[Deer-Spangle]

Yeah, definitely vendor implementation. Which is a shame.

One other option my partner just suggested is to mosaic using the thumbnail of the video (I'm guessing twitter API provides a thumbnail image for videos?), with a play button overlaid. So it gives a good preview image, but is clear that part of it is a video.

[bb010g]

Telegram and Discord don't currently support image arrays afaik so it'd just be vendor implementation then

It still might be a good idea to implement OpenGraph tag arrays so that platforms that do support them will work better, and so that if/when Telegram & Discord do implement support FixTweet will just work. (You might want a way to explicitly opt out of mosaics along with this too?)