Live DASH multiperiod SSAI not working in shaka 3.x.x because segment template indexes extend forever

[caridley]

Have you read the FAQ and checked for duplicate open issues?
Yes

What version of Shaka Player are you using?
3.0.10
The problem does not seem to happen with 2.5.20

Can you reproduce the issue with our latest release version?
Yes

Can you reproduce the issue with the latest code from master?
Yes with code from this commit 7af44ef from 4/19/2021

Are you using the demo app or your own custom app?
Custom app

If custom app, can you reproduce the issue using our demo app?
Yes

What browser and OS are you using?
Chrome on MAC OS
Problem is also being seen on Chromecast

For embedded devices (smart TVs, etc.), what model and firmware version are you using?

What are the manifest and license server URIs?
Will send e-mail with info on accessing streams

What configuration are you using? What is the output of player.getConfiguration()?
See player-config.json in the attached zip file
shakaLiveDai404_TMFJ.zip

What did you do?
Watch one of our live dash DAI streams through a few ad breaks

What did you expect to happen?
Playback should progress smoothly through the ad breaks.

What actually happened?
Playback stalls at the end of the first because of a 404 response caused by making an extra segment request at the end of the ad.

Playback then resumes using segments from the period prior to the ad break, even though that period is no longer visible in the manifest. The segment request are successful because the main video streams is continuous.

Meanwhile thousands of these error messages are emitted "Assertion failed: Should not see evicted segments after available segments"

Debug logs and manifest are in attached .zip file

[caridley]

I just e-mailed stream access instructions to shaka-player-issues@google.com

[caridley]

I tried adding these logging statement in generateSegmentIndexFromDuration_() in lib/dash/segment_template.js and found that the player was continuing to add segments to periods that had already ended. I suspect this could be caused by the fact that our live manifest do not have explicit period end times, but instead expect it to be inferred from the next period start time.

    for (let position = minPosition; position <= maxPosition; ++position) {
      const reference = createReference(position);
      shaka.log.info('AIDAN adding segment reference for period ' + context.period.id + 
      ' type=' + context.adaptationSet.mimeType +
      ' bandwidth=' + context.bandwidth + 
      ' position=' + position);
      references.push(reference);
    }


        while (nextPosition <= maxPosition) {
          const reference = createReference(nextPosition);
          shaka.log.info('AIDAN adding more segment reference for period ' + context.period.id + 
          ' type=' + context.adaptationSet.mimeType +
          ' bandwidth=' + context.bandwidth + 
          ' position=' + nextPosition);
          references.push(reference);
          nextPosition++;
        }

11:54:34.177 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=149000 position=2
11:54:34.178 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=899000 position=2
11:54:34.178 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=4200000 position=2
11:54:34.178 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=2100000 position=2
11:54:34.178 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=5652000 position=2
11:54:34.198 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=audio/mp4 bandwidth=96000 position=17
11:54:34.199 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=149000 position=17
11:54:34.199 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=899000 position=17
11:54:34.199 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=4200000 position=17
11:54:34.199 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=2100000 position=17
11:54:34.199 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=5652000 position=17
11:54:34.330 segment_template.js:430 AIDAN adding more segment reference for period orig_0_1-947046 type=audio/mp4 bandwidth=98800 position=947209
11:54:34.333 segment_template.js:430 AIDAN adding more segment reference for period orig_0_1-947046 type=audio/mp4 bandwidth=98800 position=947209
11:54:34.333 segment_template.js:430 AIDAN adding more segment reference for period orig_0_1-947046 type=video/mp4 bandwidth=149952 position=947209
11:54:34.333 segment_template.js:430 AIDAN adding more segment reference for period orig_0_1-947046 type=video/mp4 bandwidth=899968 position=947209
11:54:34.333 segment_template.js:430 AIDAN adding more segment reference for period orig_0_1-947046 type=video/mp4 bandwidth=1400000 position=947209
11:54:34.333 segment_template.js:430 AIDAN adding more segment reference for period orig_0_1-947046 type=video/mp4 bandwidth=2400000 position=947209
11:54:34.333 segment_template.js:430 AIDAN adding more segment reference for period orig_0_1-947046 type=video/mp4 bandwidth=3400000 position=947209
11:54:36.179 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=audio/mp4 bandwidth=96000 position=3
11:54:36.179 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=149000 position=3
11:54:36.179 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=899000 position=3
11:54:36.180 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=4200000 position=3
11:54:36.180 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=2100000 position=3
11:54:36.180 segment_template.js:430 AIDAN adding more segment reference for period ad_2-1 type=video/mp4 bandwidth=5652000 position=3
11:54:36.202 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=audio/mp4 bandwidth=96000 position=18
11:54:36.202 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=149000 position=18
11:54:36.202 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=899000 position=18
11:54:36.202 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=4200000 position=18
11:54:36.202 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=2100000 position=18
11:54:36.202 segment_template.js:430 AIDAN adding more segment reference for period ad_1-1 type=video/mp4 bandwidth=5652000 position=18

[caridley]

It looks to me like there is no way for the code that adds more references to a live stream segment index to detect a change in the period end time when the next period appears in the manifest.

[caridley]

When this problem occurs we will also see massive number of these error message: Assertion failed: Should not see evicted segments after available segments

[caridley]

if we allow the player to continue after encountering the a 404 response when requesting an extra segment at the end of the first ad, the player will resume fetching and playing content from the main content from the period prior to the ad break, despite the fact that period is no longer in the manifest.

Playback of live multi-period content seems to be badly broken in shaka 3.x.x

The segment indexes need to have the period end time updated when the following period comes into view in the manifest so that they do not continue adding references for ever.

This behavior is also having an impact on application performance because a large collection or segments indexes stay alive for the duration of playback adding references and logging errors.

[caridley]

I am able to improve behavior with an extremely ugly hack that makes the updated period duration available to the index update timer.

In parsePeriods_() around line 574 in dash_parser.js I add code stash the period duration

      const period = this.parsePeriod_(context, baseUris, info);
      periods.push(period);

      window.shakaPeriodDurations[period.id] = periodDuration;

In compute computeAvailablePeriodRange() in segment_template.js I make use of the stashed period duration

const computeAvailablePeriodRange = () => {
      const updatedPeriodEnd = window.shakaPeriodDurations[context.period.id] ? 
        (periodStart + window.shakaPeriodDurations[context.period.id]) :
        periodEnd;
      return [
        Math.max(
            presentationTimeline.getSegmentAvailabilityStart(),
            periodStart),

        Math.min(
            presentationTimeline.getSegmentAvailabilityEnd(),
            updatedPeriodEnd),
      ];
    };

With these changes the segment index references for live periods no longer grow without bounds, and I no longer see a gazillion of these error messages "Assertion failed: Should not see evicted segments after available segments"

I do however see indications that multiple segment indexes are being created for the a single representation within a period. When a new period appears segment indexes for existing periods seem to be duplicated.

[caridley]

I looks like the extra duplicate segment indexes are created by this line in extendExistingOutputStream_()

    // We need to create all the per-period segment indexes and append them to
    // the output's MetaSegmentIndex.
    await Promise.all(matches.map((match) => match.createSegmentIndex()));

I think these calls to createSegmentIndex should be be restricted to matches in new periods like this

    await Promise.all(matches.slice(firstNewPeriodIndex).map((match) => match.createSegmentIndex()));

[caridley]

Because the periodEnd is not being updated in the segment template index when the period end becomes known, the appendWindowEnd value is not set correctly for live multi-period streams. If a period ends partway through a segment we will end up with an overlap in the timeline. I wonder if this might be causing looping behavior that I have occasionally observed with our streams.

[caridley]

@joeyparrish I believe that we have a contributors agreement in place and I would like to submit a merge request to fix this problem. I could use use some guidance on how to best provide the updated period duration and end times to the segment index. I have prototyped two approaches.

One hack in which I "published" a map of the latest period end times in window.shakaPeriodDurations.

I also tried adding a map of period duration to the dash_parser and then defined a getter for the duration field in the periodInfo and made segment_template.js compute the period end on very use

  this.periodDurations[period.id] = periodDuration;
  Object.defineProperty(info, 'duration', {
    get: () => this.periodDurations[period.id]
  });

Does this second approach seem acceptable.

[joeyparrish]

Getting caught up on this thread, and then I'll try to provide the guidance you requested. Sorry for letting this one slip our attention!

[joeyparrish]

found that the player was continuing to add segments to periods that had already ended. I suspect this could be caused by the fact that our live manifest do not have explicit period end times, but instead expect it to be inferred from the next period start time.

There is a check since v3.0.8 that looks like this:

        if (presentationTimeline.getSegmentAvailabilityEnd() >= periodEnd &&
            !references.length) {
          // Signal stop.
          return null;
        }

The periodEnd variable is set by context.periodInfo.duration, which is derived in lib/dash/dash_parser.js like so:

        if (nextStart != null) {
          periodDuration = nextStart - start;
        }

So by the time we generate segment references, the period duration has already been calculated based on the start of the next period in the manifest.

It looks to me like there is no way for the code that adds more references to a live stream segment index to detect a change in the period end time when the next period appears in the manifest.

This may be the root of the issue. Since the callback that generates references has captured the value of the local variable periodEnd, this would have to be replaced by a new callback to get a new value for the end of the period. Ideally, we would do something like you suggested and use some structure to allow updates to that value later.

I'm not especially keen on using Object.defineProperty for this. If SegmentTemplateInfo object is already captured by the callback, why wouldn't we just update that directly?

Just answered my own question reviewing the code. That object is recreated on each update in createStreamInfo, and the returned generateSegmentIndex function will never be called again. So updating that doesn't make much sense with the current structure.

How about this instead? We maintain and pass segmentIndexMap into each createStreamInfo call. In addition to that, we could maintain and pass a map of period IDs to ending times. That new map could be sent along to generateSegmentIndexFromDuration_, and used instead of the locally-calculated periodEnd value.

Because the periodEnd is not being updated in the segment template index when the period end becomes known, the appendWindowEnd value is not set correctly for live multi-period streams. If a period ends partway through a segment we will end up with an overlap in the timeline. I wonder if this might be causing looping behavior that I have occasionally observed with our streams.

That sounds like a reasonable conclusion. Thanks for all the hard work you've done tracking this down! We're very grateful for your help.