[QUESTION] Change starting offset time in custom manifest loader

[mkochman]

Hi 👋
Is it possible to change start position during custom manifest loading?

So we want to start stream from certain position using loadVideo method, e.g.

Set base loadVideo config:

{
  autoPlay: true,
  startAt: { position: 10 },
  transport: "dash",
  url: "some url",
  manifestLoader: customManifestLoader
}

Parse manifest data and modify offset:

customManifestLoader = (manifestData, callback) => {
	const { url } = manifestInfo;
        const xhr = new XMLHttpRequest();
	const baseTime = performance.now();

  	xhr.onload = (r) => {
			if (200 <= xhr.status && xhr.status < 300) {
				let streamDuration = null;

			        function parseISODuration(duration: string): number {
				        .....
				        // parser logic
			        }
				const data = xhr.response;
				const mpdElement = data?.querySelector("MPD");
				streamDuration = mpdElement?.getAttribute("mediaPresentationDuration");
				...
				// Some logic based on duration
				...
		                // New start position value
		                // Best way to set a new start position?
			}
		};
  // Some manifest stuff
}

So far I have logic where loadVideo is called and then after stream is ready seekTo, so yep it's not the most efficient.

I will be grateful for your help ;)

[peaBerberian]

Hi,

So if I get it right your start position calculation is too complex for what startAt allows?

Can you tell us an example of such calculation?
The better way would probably for the API to be able to integrate it into startAt in some ways (or even make it a e.g. function)

[mkochman]

I need this for the live stream, for normal vod I can calculate proper position because I know range for the stream.
For live stream it is much more complicated, there is a DVR window and the best performance is when passed start position is in that range, if for example passed start time position is before or after dvr window rx player is quite struggling with stream start.
It starts but it takes quite a lot of time, what is quite not acceptable for TV devices.
And unfortunately, information about DVR window range is only available when manifest is downloaded :(
So far I've tested logic like:
Start live stream and do seek, it works but on older TV devices starting time is quite slow, then I've tried start stream with manifest parsing and loadVideo again and from testing it is faster and it reduces additional seeking.
So I wonder if there is a good way to e.g. dynamically overwrite startAt position while manifest data is parsed and new position is counted in DVR window range 🤔

[mkochman]

Summarizing the main problem, live streaming is tricky due to a DVR window. Best results occur when the start position is within this range. If the start time is outside this range, it can cause the player to struggle.

[peaBerberian]

Yes when you start further than the DVR window, the player will have to wait because media segments aren't available yet for that position, you'll normally also have warning events with the error code MEDIA_TIME_AFTER_MANIFEST that will be sent regularly during that time.

Where do you need to start? For example the startAt option allows to tell that you want to play something like 10 seconds before the end of the DVR window (fromLastPosition: -10).

[mkochman]

Yes, I want to reduce that time, when position is outside DVR window, it works a lot of better when position is inside DVR window range, then player doesn't need to download not ready segents.

When position is after the end of DVR window I'm using fromLastPosition, when is before I'm using fromFirstPosition. So far I;ve created logic in custom manifest load and I'm doing loadVideo call with new startAt value and abort xhr of first started manifest call.

Also, about catching it in warning, I see that doing that in manifest load is a little bit faster, also it will reduce complexity with detection for this case in warning 🤔

So my questions:

• Can I overwrite startAt value in custom manifest load? (So I will not need to abort first manifest load, but update startAt value)
• Shouldn't RX Player itself do something like this? I mean you are checking live stream DVR window from manifest and if passed value in startAt position is before then start from begging of DVR window and after DVR window, then start from the end of DVR window?

[peaBerberian]

Can I overwrite startAt value in custom manifest load? (So I will not need to abort first manifest load, but update startAt value)

Right now, no.
But I think the best way would be for an application to call seekTo on a MEDIA_TIME_BEFORE_MANIFEST or MEDIA_TIME_AFTER_MANIFEST warning.
This way it can do what it wants, does not have to re-call loadVideo (and to have to re-wait for Manifest loading and so on) and the API can stay "simpler".

Though I'll have multiple things to check, which I'll try to do soon (tomorrow?):

• I have to ensure that those warnings are sent as soon as possible, e.g. as soon as the Manifest is parsed in your case

• I have to ensure that API like getMinimumPosition / getMaximumPosition (that provide the minimum/maximum positions that can be seeked to) are callable. Alternatively we could add things like fromFirstPosition / fromLastPosition to seekTo, I'll see.

• And perhaps more importantly, I'll have to check that seekTo can be called and will be functional at that point (because it might be in conflict with startAt at that - early - point).

---

So to summarize for me the right solution would be something like:

rxPlayer.addEventListener("warning", (err) => {
  if (err.code === "MEDIA_TIME_BEFORE_MANIFEST") {
    const minimumPosition = rxPlayer.getMinimumPosition();
    if (minimumPostion != null) {
      rxPlayer.seekTo(rxPlayer.getMinimumPosition() + 5);
    }
  } else if (err.code ===  "MEDIA_TIME_AFTER_MANIFEST") {
    const maximumPosition = rxPlayer.getMaximumPosition();
    if (maximumPostion != null) {
      rxPlayer.seekTo(rxPlayer.getMaximumPosition() - 5);
    }
  }
});

But I have to check that it's both performant (as in: called as soon as we can) and that it works as expected.

Shouldn't RX Player itself do something like this? I mean you are checking live stream DVR window from manifest and if passed value in startAt position is before then start from begging of DVR window and after DVR window, then start from the end of DVR window?

We avoid doing that ourselves because we don't know what the application would want in that case (e.g. applications could prefer displaying a warning if a particular program is not available anymore and play live content instead).

There's also questions about what should we do if the end user pauses a content enough time that the position now fell behind the DVR window: Should we seek at the minimum position and play? Or seek and pause with the risk of very soon have to re-do it? Do nothing as the current frame is still visible anyway and wait for a play?
Note that even at Canal+, there's multiple applications using the RxPlayer that don't have the same answers for these questions.

Because of all those questions, we ended up preferring the exposition of an unopinionated lower-level API where we try to give an application the means to do what it wants to do.
So in this case, we signal to the application that the current position is currently not in the Manifest's DVR window with the idea that the application can then use our other API to do what it wants to do (seek close to the start of the DVR window, to the live edge, stop the content etc.).

[peaBerberian]

Though we understand that this is a situation that's easy to miss and that it can be assumed that the RxPlayer should be the one handling it.

We do often hear developers who would have preferred that some specific logic ot be handled by the RxPlayer, so we're not totally closed to the idea of doing things our way in some cases.
But for now, we prefer the MEDIA_TIME_BEFORE_MANIFEST / MEDIA_TIME_AFTER_MANIFEST - based solution for that scenario.

[peaBerberian]

I looked more into it and there seem to actually be some bouding to the Manifest's minimum and maximum positions going down specifically for the startAt option:

rx-player/src/main_thread/init/utils/get_initial_time.ts

Lines 88 to 131 in 2ca74e6

	export default function getInitialTime(
	manifest: IManifestMetadata,
	lowLatencyMode: boolean,
	startAt?: IInitialTimeOptions,
	): number {
	if (!isNullOrUndefined(startAt)) {
	const min = getMinimumSafePosition(manifest);
	const max = getMaximumSafePosition(manifest);
	if (!isNullOrUndefined(startAt.position)) {
	log.debug("Init: using startAt.minimumPosition");
	return Math.max(Math.min(startAt.position, max), min);
	} else if (!isNullOrUndefined(startAt.wallClockTime)) {
	log.debug("Init: using startAt.wallClockTime");
	const ast =
	manifest.availabilityStartTime === undefined ? 0 : manifest.availabilityStartTime;
	const position = startAt.wallClockTime - ast;
	return Math.max(Math.min(position, max), min);
	} else if (!isNullOrUndefined(startAt.fromFirstPosition)) {
	log.debug("Init: using startAt.fromFirstPosition");
	const { fromFirstPosition } = startAt;
	return fromFirstPosition <= 0 ? min : Math.min(max, min + fromFirstPosition);
	} else if (!isNullOrUndefined(startAt.fromLastPosition)) {
	log.debug("Init: using startAt.fromLastPosition");
	const { fromLastPosition } = startAt;
	return fromLastPosition >= 0 ? max : Math.max(min, max + fromLastPosition);
	} else if (!isNullOrUndefined(startAt.fromLivePosition)) {
	log.debug("Init: using startAt.fromLivePosition");
	const livePosition = getLivePosition(manifest) ?? max;
	const { fromLivePosition } = startAt;
	return fromLivePosition >= 0
	? livePosition
	: Math.max(min, livePosition + fromLivePosition);
	} else if (!isNullOrUndefined(startAt.percentage)) {
	log.debug("Init: using startAt.percentage");
	const { percentage } = startAt;
	if (percentage > 100) {
	return max;
	} else if (percentage < 0) {
	return min;
	}
	const ratio = +percentage / 100;
	const extent = max - min;
	return min + extent * ratio;
	}

But this is actually worse here than doing nothing here, not only we don't have the warnings initially, but:

• moving silently the initial position at the live edge when startAt would be after it may still lead to some wait, the time for the current segments to be made available
• moving silently the initial position to the minimum position when startAt would be before is too risky, as old segments from live contents are usually continuously removed. We will usually have a MEDIA_TIME_BEFORE_MANIFEST after some time but not immediately here.

In most cases, at least multiple seconds should be added/removed from the max/min position for more efficient content start-up.
If I remove that bounding logic for live contents, the MEDIA_TIME_BEFORE_MANIFEST warning is sent as early as we know it, though the other issue is that seeking at that point in time does not work for now.

So we're right now discussing on two elements on our side:

• if we can remove that IMO hurtful manifest-bounding security from the startAt handling code. The problem is that here some applications might (I guess unknowingly) rely on it, even if the resulting behavior is far from ideal.

• we'll try to make seekTo work at that point.