Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Capture media for specified WebContents #4776

Open
MarshallOfSound opened this issue Mar 13, 2016 · 19 comments
Open

Capture media for specified WebContents #4776

MarshallOfSound opened this issue Mar 13, 2016 · 19 comments

Comments

@MarshallOfSound
Copy link
Member

@MarshallOfSound MarshallOfSound commented Mar 13, 2016

  • Electron version: 0.37.0
  • Operating system: All

I've been trying for weeks now to figure this out, I need to somehow capture the audio output from my electron application. But the desktopCapturer API appears to exclude that exact use case 😢

Is there an alternative way of performing this capture WITHOUT capturing the entire computers audio through the "screen" capturer?

https://github.com/atom/electron/blob/master/docs/api/desktop-capturer.md

Also open to any JS methods of capturing a stream from an Audio element

@MarshallOfSound

This comment has been minimized.

Copy link
Member Author

@MarshallOfSound MarshallOfSound commented Mar 13, 2016

Worth noting the Chrome API I believe this was supposed to mirror / implement supports audio streams.

https://developer.chrome.com/extensions/tabCapture

@zcbenz

This comment has been minimized.

Copy link
Member

@zcbenz zcbenz commented Mar 13, 2016

I don't think our current API set is capable of this.

@zcbenz zcbenz changed the title How to capture audio through the desktopCapture API Capture media for specified WebContents Mar 13, 2016
@mgmorcos

This comment has been minimized.

Copy link

@mgmorcos mgmorcos commented Jul 4, 2016

+1

@demian85

This comment has been minimized.

Copy link

@demian85 demian85 commented Mar 7, 2017

Not sure why #7255 is closed since example in docs is still not working and thrown error does not provide info whatsoever.
From the docs:

To capture both audio and video from the entire desktop the constraints passed to navigator.webkitGetUserMedia must include chromeMediaSource: 'screen', and audio: true, but should not include a chromeMediaSourceId constraint.

However, at least on Mac OSX, that throws NavigatorUserMediaError with no message.

Also, the constraints spec has changed and mandatory property is not needed anymore.
webkitGetUserMedia has also changed to mediaDevices.getUserMedia()

@ajbdev

This comment has been minimized.

Copy link

@ajbdev ajbdev commented Aug 17, 2017

Will there be any attempt to fix this?

Just out of curiosity, what is the blocking reason for this?

I can't figure out how to capture audio at all on Mac. On Windows I can capture audio from every possible input only (mic, apps, etc).

@mhashmi

This comment has been minimized.

Copy link

@mhashmi mhashmi commented Sep 14, 2017

Echoing what others are saying, I just tried audio / video capture on Mac (Sierra), Linux (Debian Jessie) and Windows 10 using the constraints specified in the desktopCapture API. As ajbdev noted, on Windows, the audio capture works correctly, however, on Linux and Mac we simply get an error (or more specifically, the error callback with no error defined).

I have thus far tried with Electron 1.7.6 and 1.6.5[1] with similar results. Is there any version we can switch to where audio capture works? Is it perhaps possible to capture audio and video separately and add the audio track to the video stream?

[1] Electron v1.6.5 does not throw an error on the constraints[2] outlined in the desktopCapture API, however, we get a audio sync state error in the terminal when the video stream is initialised.

[2] Passing constraints:

let constraints = {
  audio: {
    mandatory: {
      chromeMediaSource: 'desktop'
    }
  },
  video: {
    mandatory: {
      chromeMediaSource: 'desktop'
    }
  }
} 
@CapOM

This comment has been minimized.

Copy link
Contributor

@CapOM CapOM commented Feb 21, 2018

Have you tried chrome.desktopCapture.chooseDesktopMedia ? (doc https://developer.chrome.com/extensions/desktopCapture)

@icecoll

This comment has been minimized.

Copy link

@icecoll icecoll commented Apr 11, 2018

  • electron v1.8.4
  • Ubuntu 16.04
function startRecord() {
    electron.desktopCapturer.getSources({types: ['window', 'screen']}, (error, sources) => {
      if (error) throw error
      for (let i = 0; i < sources.length; ++i) {
        if (sources[i].name === "foo") {
          navigator.mediaDevices.getUserMedia({
            audio: false,
            video: {
              mandatory: {
                chromeMediaSource: 'desktop',
                chromeMediaSourceId: sources[i].id,
                minWidth: 1280,
                maxWidth: 1280,
                minHeight: 720,
                maxHeight: 720
              }
            }
          })
            .then((stream) => handleStream(stream))
            .catch((e) => handleError(e))
          return
        }
      }
    });
}

function handleStream(stream) {
    navigator.mediaDevices.getUserMedia({audio: true, video: false}).then(function(mediaStream){
      var audioTracks = mediaStream.getAudioTracks();
      //add video and audio sound
      var medias = $("audio,video");
      for (var i = 0; i < medias.length; i++) {
        var tmpStream = medias[i].captureStream();  // mainWindow = new BrowserWindow({webPreferences: {experimentalFeatures: true} })
        if(tmpStream) {
          var tmpTrack = tmpStream.getAudioTracks()[0];
          audioTracks.push(tmpTrack);
        }
      }

      // mix audio tracks
      if(audioTracks.length > 0){
        var mixAudioTrack = mixTracks(audioTracks);
        stream.addTrack(mixAudioTrack);
      }

      stream.addTrack(audioTrack);
      recorder = new MediaRecorder(stream);
      recorder.ondataavailable = function(event) {
        // deal with your stream
      };
      recorder.start(1000);
    }).catch(function(err) {
      //console.log("handle stream error");
    })
}

  function mixTracks(tracks) {
    var ac = new AudioContext();
    var dest = ac.createMediaStreamDestination();
    for(var i=0;i<tracks.length;i++) {
      const source = ac.createMediaStreamSource(new MediaStream([tracks[i]]));
      source.connect(dest);
    }
    return dest.stream.getTracks()[0];
  }

It works! reference

@mhashmi

This comment has been minimized.

Copy link

@mhashmi mhashmi commented Apr 11, 2018

@icecoll That worked, many thanks!

edit: I tested your code in OS X as well as our app. Works perfectly.

@mhashmi

This comment has been minimized.

Copy link

@mhashmi mhashmi commented Apr 12, 2018

@icecoll the capture is unfortunately from the microphone. I confirmed too quickly as I had headphones attached to my mac which were lying next to where the mac's microphone is, hence the confusion.

Others in the bug report I opened (#10515) pointed this out.

@icecoll

This comment has been minimized.

Copy link

@icecoll icecoll commented May 18, 2018

@mhashmi I think getUserMedia can only get the microphone's sound but not the speaker and headphone's, if you want to get the output device sound, you must try other way. If you just want to get sound from <video>/<audio> tags, I think you can try to use getAudioTracks method, and add the audio tracks to your stream.

@gio1217

This comment has been minimized.

Copy link

@gio1217 gio1217 commented May 28, 2018

same issue

@bhavinshiroya1994

This comment has been minimized.

Copy link

@bhavinshiroya1994 bhavinshiroya1994 commented May 29, 2018

@icecoll same problem please solve this issue.

@peererror

This comment has been minimized.

Copy link

@peererror peererror commented May 30, 2018

is any way to get speaker audio from webrtc or apps audio

@peererror

This comment has been minimized.

Copy link

@peererror peererror commented Jun 10, 2018

is there a way to make work chrome.tabCapture or any other thing I want to capture the audio output of my electron app .@MarshallOfSound were you able to solve your problem

@m-onz

This comment has been minimized.

Copy link

@m-onz m-onz commented Jun 12, 2018

Same issues... I have not got any audio working on osx or linux

@AmarBouchibane

This comment has been minimized.

Copy link

@AmarBouchibane AmarBouchibane commented Oct 12, 2018

@icecoll the capture is unfortunately from the microphone. I confirmed too quickly as I had headphones attached to my mac which were lying next to where the mac's microphone is, hence the confusion.

I ran into the same mistake. I hope this will be fixed someday.

@danmo

This comment has been minimized.

Copy link

@danmo danmo commented Feb 25, 2019

2019 - no luck, hope is still strong on this one.

@4eveRS

This comment has been minimized.

Copy link

@4eveRS 4eveRS commented Jun 26, 2019

Any update on this enhancement request? looking forward for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.