chrome.desktopCapture API

[max-mapper]

Currently in Chromium there is only support for capturing the entire desktop (using getUserMedia with chromeMediaSource: "screen". I looked at the Chromium issue tracker and I can't find anything on the roadmap for better sharing granularity (e.g. to only share 1 app window instead of the entire screen). In Chrome there is a much nicer desktopCapture API that implements this really nice popup UI as well as per-OS-window or per-browser-tab or entire screen sharing.

This is probably a longshot, but how hard would it be to add the desktopCapture Chrome Platform API to Atom Shell? I can't even find the google repo where desktopCapture is implemented...

[anaisbetts]

This would be 🆒, looks like it starts in https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/media/desktop_media_picker.h&sq=package:chromium&rcl=1428421466 and https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/extensions/api/desktop_capture/desktop_capture_api.cc&sq=package:chromium

[whistlegraph]

+1 especially https://developer.chrome.com/extensions/tabCapture

[petethepig]

+1

[Quramy]

+1

[icetraxx]

+1

[hotrush]

👍

[octalmage]

Would love to see this implemented!

[tommoor]

If anyone is interested in taking this on, we'd be willing to sponsor someone to work on this, get in touch with me if that sounds like an interesting challenge 😄

There is a lot of reference code in nw.js as they have already implemented this feature, albeit with some questionable api design choices.

[rogeru]

+1

[johrue]

+1

[hokein]

I will take this issue. But before implementation, some details of the API design are needed to be disscussed. There are two options we can provide in Electron as far as I can see:

• Option1: Align with Chromoium upstream way by implementing a win.mainContents.chooseMediaList API. The API will prompt a Chromium picker dialog to allow user choose which window/screen they want to share.
  The API usage is illustrated as follows:

mainWindow.webContents.chooseDesktopMedia(["screen", "window"], function(error, desktopId) {
  if (error) {
    console.log('Desktop Capture access rejected.');
    return;
  }
  navigator.webkitGetUserMedia({
    audio: false,
    video: {
      mandatory: {
        chromeMediaSource: 'desktop',
        chromeMediaSourceId: desktopId,
        minWidth: 1280,
        maxWidth: 1280,
        minHeight: 720,
        maxHeight: 720
      }
    }
  }, gotStream, getUserMediaError);
});

• Option2: We don't implement the Chromium picker dialog in Electron side by providing a more details information for each captured window/screen. So the API is like:

var screenCapture = win.webContents.getMediaList(["window", "screen"]);

screenCapture.on('source-add', function(item) {
 // item.name: the name of captured screen/window shown to the user
 // item.image: the thumbnail NativeImage of captured screen/window.
 // item.id: the id of captured screen/window used in navigator.webkitGetUserMedia 
 if (item.name == 'Electron') {
   navigator.webkitGetUserMedia({
     audio: false,
     video: {
       mandatory: {
         chromeMediaSource: 'desktop',
         chromeMediaSourceId: item.id,
         minWidth: 1280,
         maxWidth: 1280,
         minHeight: 720,
         maxHeight: 720
       }
     }
   }, gotStream, getUserMediaError);
 }
});

screenCapture.on('source-change', function(item) {});
screenCapture.on('source-remove', function(item) {});

Compared to option 1, option 2 provides the possibility of custumization to users of Electron(They can do whaterver they need, e.g. implement a customized dialog, and select a fixed window in code). On the other side, user are benefited from the nice Chromium picker dialog in option 1, but they also loose customization in cost.

We would like to hear the feedbacks from users. Suggestions are welcomed 😄

[whistlegraph]

I think most Electron based apps visually 'pretend' not to be a browser,
while simultaneously taking advantage of all the features browsers can
offer. My suggestion is to start with option 2 because If you only
implement option 1, some Electron user will want to create a custom picker
dialog that fits their application's design and this ticket will re-open
soon enough.

rey.sc

On Tue, Sep 29, 2015 at 11:27 PM, Haojian Wu notifications@github.com
wrote:

I will take this issue. But before implementation, some details of the API
design are needed to be disscussed. There are two options we can provide in
Electron as far as I can see:

• Option1: Align with Chromoium upstream way by implementing a
  win.mainContents.chooseMediaList API. The API will prompt a Chromium
  picker dialog to allow user choose which window/screen they want to share.
  The API usage is illustrated as follows:

mainWindow.webContents.chooseDesktopMedia(["screen", "window"], function(error, desktopId) {
if (error) {
console.log('Desktop Capture access rejected.');
return;
}
navigator.webkitGetUserMedia({
audio: false,
video: {
mandatory: {
chromeMediaSource: 'desktop',
chromeMediaSourceId: desktopId,
minWidth: 1280,
maxWidth: 1280,
minHeight: 720,
maxHeight: 720
}
}
}, gotStream, getUserMediaError);
});

• Option2: We don't implement the Chromium picker dialog in Electron
  side by providing a more details information for each captured
  window/screen. So the API is like:

var screenCapture = win.webContents.getMediaList(["window", "screen"]);

screenCapture.on('source-add', function(item) {
// item.name: the name of captured screen/window shown to the user
// item.image: the thumbnail NativeImage of captured screen/window.
// item.id: the id of captured screen/window used in navigator.webkitGetUserMedia
if (item.name == 'Electron') {
navigator.webkitGetUserMedia({
audio: false,
video: {
mandatory: {
chromeMediaSource: 'desktop',
chromeMediaSourceId: item.id,
minWidth: 1280,
maxWidth: 1280,
minHeight: 720,
maxHeight: 720
}
}
}, gotStream, getUserMediaError);
}
});

screenCapture.on('source-change', function(item) {});
screenCapture.on('source-remove', function(item) {});

Compared to option 1, option 2 provides the possibility of custumization
to users of Electron(They can do whaterver they need, e.g. implement a
customized dialog, and select a fixed window in code). On the other side,
user are benefited from the nice Chromium picker dialog in option 1, but
they also loose customization in cost.

We would like to hear the feedbacks from users. Suggestions are welcomed [image:
😄]

—
Reply to this email directly or view it on GitHub
#1380 (comment).

[anaisbetts]

Option 2 makes more sense to me, you can always use the information in a JS-based API to build that picker

[tommoor]

Another advantage of Option 2 is that you could add cleverness, like adding a shortcut to share the same screen as you previously shared or defaulting to it. The implementation could end up even nicer than Chrome's.

Gets my vote 👍

[octalmage]

All about option 2 here. If it's implemented in a way that the picker from option 1 could be created in userland, it's a win-win.

[RDLemos]

+1

[adileo]

+1

[mzahor]

+1

[sidneys]

+1

[Morantron]

+1

[c0b41]

+1

[apurv]

+1

[Raj123456788]

Hey Guys,
I am unable to get desktop picker dialog for available sources. I am newbie can someone guide me what am I missing?

[jlord]

Hi @Raj123456788 — Can you open a new issue describing your problem, the operating system you're using and version of Electron? If you can also provide code that can be used to reproduce your issue that would be spectacular. Thanks, that will help us get to the root of your issue ✨

[Raj123456788]

Done!!

[Raj123456788]

Can anyone please answer this question: #4432

[iclems]

Now that W3C, Chrome, Firefox are converging on using getDisplayMedia as a standard, could be worth implementing a native dialog shimming that on top of this existing module no?

[iclems]

FYI when Electron updates to Chrome 70, we should get getdisplaymedia automatically when using experimental features.

[CapOM]

See #16513