Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mediastream support #1634

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from
Draft

Conversation

rafern
Copy link

@rafern rafern commented Dec 8, 2022

Feature

There is currently no way to play WebRTC audio in Howler. This PR lets you provide a MediaStream as a source, since you can create a MediaStream object from WebRTC audio tracks.

If someone wanted to play WebRTC audio in the current state of the library, they would have implement it separately from Howler, which means re-implementing most of Howler's features, such as spatial audio, etc..., hence the need for this in the core library.

Related Issues

Fixes #581 and fixes #1314

Solution

Instead of making a new class for streams, this PR lets you provide a MediaStream as a source for a Howl object. If a Howl object is using a MediaStream as a source, then it will:

  • have infinite duration
  • no sprite, seek, loop nor playback rate support, since streams don't buffer
  • pausing a stream will not record any audio (pausing a stream is similar to muting), since streams don't buffer

If using Web audio:

  • MediaStream is connected to audio context by replacing the bufferSource with the result of createMediaStreamSource, instead of using an actual buffer source
  • A temporary audio element with the stream attached is created to work around a chromium bug

If using HTML5 audio:

  • MediaStream is attached to an audio element by using srcObject

I'm not 100% happy with this solution, as the API feels wrong, so I'm leaving this PR as a draft. Ideally there would be a new class for streams since they are more limited than sounds, and a buffered stream class which can be paused/seeked, but this PR is more of a starting point.

Testing

  1. Get MediaStream (either get it from WebRTC (see mediasoup-demo or other WebRTC projects) or make one by capturing an audio file)
  2. Create a new Howl object with the MediaStream as the src (or part of the src array)
  3. play it, pause it, etc...

Breaking Changes

No breaking changes as far as I know

Other stuff

The build script didn't work on my system (there were some errors with sed), so I replaced the build script with esbuild. I didn't include this in the PR, and you can find it in the master branch of our repository, but I think it would be better if #1518 got merged (please!)

@rudrabhoj
Copy link

Any updates on actually merging this pull request?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create howl from MediaStream Compatibility with streaming audio from WebRTC
2 participants