Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a remoting buffer state attribute to RemotePlayback #126

Closed
wants to merge 1 commit into from

Conversation

takumif
Copy link

@takumif takumif commented Sep 3, 2019

This PR proposes to add a remoting buffer state attribute to RemotePlayback, which will indicate the remote buffer state when using MSE / media remoting mode (as opposed to media/URL flinging).

Here is sample code showing how RemotePlayback may interact with MSE using the new enum:

const video = document.querySelector('#my-video');
const mediaSource = new MediaSource();
let sourceBuffer;
let isBufferingSegments = false;
mediaSource.addEventListener('sourceopen', onMediaSourceOpen);
video.src = window.URL.createObjectURL(mediaSource);
video.remote.addEventListener('remotingstatechanged', onRemotingStateChanged);

function onRemotingStateChanged() {
  switch (video.remote.remotingBufferState) {
    case 'insufficient-data':
      lowerResolution();
      startBufferingSegments();
      break;
    case 'enough-data':
      startBufferingSegments();
      break;
    case 'too-much-data':
    case 'not-remoting':
      stopBufferingSegments();
      break;
  }
}

async function onMediaSourceOpen() {
  sourceBuffer = mediaSource.addSourceBuffer('video/mp4; codecs="avc1.4d401f"');
  await startBufferingSegments();
  video.play();
}

async function startBufferingSegments() {
  if (isBufferingSegments) {
    return;
  }
  isBufferingSegments = true;
  sourceBuffer.addEventListener('updateend', bufferNextSegment);
  await bufferNextSegment();
}

function stopBufferingSegments() {
  if (!isBufferingSegments) {
    return;
  }
  isBufferingSegments = false;
  sourceBuffer.removeEventListener('updateend', bufferNextSegment);
}

async function bufferNextSegment() {
  const videoChunk = await getNextVideoChunk();
  sourceBuffer.appendBuffer(new Uint8Array(videoChunk));
}

async function getNextVideoChunk() {
  // Fetches and returns a video chunk.
}

Preview | Diff

@mfoltzgoogle
Copy link
Contributor

Adding this PR to the topic for the TPAC agenda.

@pthatcherg
Copy link

Read over the PR and the example. It looks good to me. It's interesting that the use of the enum reduced down to "I'm remoting; please append to buffer" and not. I wonder if we could reduce it to that. Can we think of a case where we need the additional states?

@takumif
Copy link
Author

takumif commented Sep 4, 2019

Can we think of a case where we need the additional states?

There may be a situation where the receiver side's status is insufficient-data, even though the sender side's HTMLMediaElement.readyState is HAVE_ENOUGH_DATA. This may happen if the network connection between the sender and the receiver is poor (maybe the sender has an ethernet connection so it can download media fast, but the receiver doesn't). In this case, we may want to lower the bit rate of the media. I've updated the example to include this.

@mounirlamouri
Copy link
Member

@takumif according to the TPAC discussion, it's not clear whether there is a strong need for this change at the moment. It seems that the first step is to update OSP to have a backpressure signal.

I wonder if it would make sense to move this into an issue where we could talk about the problem this is trying to solve instead of how to solve it? WDYT?

@takumif
Copy link
Author

takumif commented Jan 31, 2020

Sure, filed #132.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants