Origin-Relay-Edge Assistance

[cwpenhale]

Hey @getroot!

Opening up a new issue based on the feedback you gave me in #766.

I've got a use case I'm hoping I can get your insight on. We've got limited bandwidth (35Mbit up) from our broadcast site.

Here's what we're trying to accomplish:

1. We're publishing to origin via RTMP via OBS at rtmp://local.site.organization.tld✅
2. ome-origin is exposing OVT at site.organization.tld:9000 to ome-edge edge.cdn.organization.tld, doing hardware transcoding in docker for 2 additional profiles, 1080p and 720p ✅
3. ome-edge edge.cdn is reaching ome-origin site via OVT when an incoming, upstream OVT request is made from ome-edge cdn ❌
4. ome-edge bound to ip-anycasted https://cdn.organization.tld:443 is reaching edge.cdn.organization.tld:9000 over OVT when a player in step 5 requests an LLHLS playlist ✅
5. OvenPlayer requests https://cdn.organization.tld/app/live_UHD/llhls.m3u8 and plays the video in near real time ✅

We've got graphics cards on-site, but want to use commodity hardware in our cloud setup, and are trying to avoid paying for GPUs in the cloud, so we've got our origin behind our firewall. So we're using edge.cdn to consolidate the requests from all of the actual edge servers receiving anycasted requests globally. If we receive requests from multiple edge servers to our origin at site, we won't have bandwidth. We only have enough bandwidth to serve one edge server, and need to consolidate behind one edge.

In step 3, ome-edge crashes the second the upstream edge requests a stream, so I figured this was just unsupported behavior. I started down the path for Varnish, but I found out it doesn't cache HTTP/2 on the backend. I would love to use HTTP/2 end to end with this LLHLS solution! Any thoughts on how to solve this?

Here's a gist with my configs and logs as requested: https://gist.github.com/cwpenhale/99255d5ae1f5a3b2264f5e1fcf26f5ea

Looking forward to helping test.

Thanks, and great work!
Connor

Originally posted by @cwpenhale in #766 (comment)

[basisbit]

what version / commit of OME are you using? LL-DASH is not yet usable in any of the stable releases.

[cwpenhale]

Hi @basisbit! I've been able to reproduce this behavior with HLS as well, using the 0.13.2 tag and I just reproduced against latest dev. I'm about to run this with debug logging enabled to try to get more information.

[getroot]

Please test with the latest master branch. I fixed the crash problem in relay.

[cwpenhale]

Hey @getroot! Working without the crash and with ABR. However, we've lost <CrossDomains> along the way. I'll bring it back to #766 for that issue. Here's the edge config, concentrator, and origin XML I ended up with: https://gist.github.com/cwpenhale/3564493210c668f01fe6d902b7bc45c1

[cwpenhale]

At this point all of my issues are resolved. Working through iOS ABR in #777 but that's unrelated, AFAICT.