Can mux.js be configured not to change timestamps?

[joeyparrish]

We use mux.js, if present, to convert TS to MP4 in Shaka Player on platforms that don't have native TS support. However, when we do that, we've already parsed the timestamps from TS segments to build an accurate segment index. What we find, though, is that the timestamps in the output are not the same as the input timestamps. See shaka-project/shaka-player#1102 for more.

To support both native TS implementations and mux.js-based transmuxing to MP4, we need everything to be consistent. For our architecture, the timestamps in the MP4 output should be the same as the TS input segments.

Can this be done today with mux.js? Or would this constitute an enhancement?

[mjneil]

Hi @joeyparrish

No there currently isn't a way to configure it to not change the timestamps so that would consitute an enhancement. However, If I follow correctly, you may be able to work around it with the cuirrent build of mux.js.

Is the problem that say you have a segment with timestamp 100, which you parse out and use to set SourceBuffer.timestampOffset, but the mp4 output has a timestamp of 0, which causes it to be incorrectly placed on the timeline when appending to the buffer?

If this is the case, then I think you might be able to use Transmuxer.setBaseMediaDecodeTime which is used to shift the timestamp of the output mp4 so that setting SourceBuffer.timestampOffset is not needed

[joeyparrish]

You are close. We don't set the timestampOffset for live streams. We build a segment index (for seeking) that allows us to locate a segment by its timestamp. From that, we also build our model of what timestamps are viewable/bufferable.

The trouble is that Transmuxer causes what is actually buffered to have different timestamps than what we have in our index. We don't want our index-building and manifest-parsing logic to have to know about transmuxing, and on some platforms with native TS (Edge, for example), we won't be using Transmuxer at all.

But it sounds like Transmuxer.setBaseMediaDecodeTime may help. I'll check it out and let you know how it turns out. Thanks!

[mjneil]

Another thing that might be helpful is listening to gopInfo on the transmuxer

transmuxer.on('gopInfo', (info) => {
  // info is an array of objects of the form { pts, dts, byteLength } representing each GOP in the segment
});

The output mp4 will still have the shifted timestamps, but the values emitted by this event are the values before the time shift.

Another note: I'm unsure if you have come across this already or not, but mux.js also accounts for the 33bit timestamp rollover. If you are not account for that on your end, you might see a difference in the input timestamp and the output timestamp as well

[joeyparrish]

Roll-over shouldn't be an issue. We only look at the timestamp of the first segment in the playlist. After that, so long as we update the playlist frequently enough that there are some segments we recognize in the updated version, we will be computing the timestamps of new segments based on the timestamps of old segments, without parsing the segments themselves.

[joeyparrish]

For some reason, setBaseMediaDecodeTime doesn't seem to affect timestamps in the output. I'll try to put together a simple case to reproduce it.

[joeyparrish]

I must be doing something wrong. When I put together a simple test page, it's working correctly.

[joeyparrish]

I was setting the time in seconds, not timescale units. Whoops! Everything is working now.

@mjneil, would you be open to an option for this behavior so we don't have to manage the base time ourselves?

[mjneil]

I'm glad you got that working for you!

And yea, I think an initialization option to not change the timestamps would be a good option to add. If you want to put together a PR for it I'd be happy to review and help get that merged.

[joeyparrish]

Thank you so much for helping us with this. What is your release schedule like? We are, of course, eager to adopt 4.3.3 with the new option.

[mjneil]

Of course, thanks for helping get the contribution in. I was going to make a release now, but it looks like our TravisCI Master branch build is unable to run due to some issues Travis is having, so I will probably make a release tomorrow after thats all been resolved. This will also be versioned to 4.4.0 since this is a new backwards compatible feature.

[mjneil]

4.4.0 has been released

[joeyparrish]

Thanks!