Support Electron and other content embedders

[aslushnikov]

What's a "content embedder"?

Chromium codebase is organized into layers:

• //blink - HTML rendering engine
• //content - an API for browser implementers (think of it as a library that you'd use if you were to write your own browser). Things like process model are handled here; uses //blink.
• //chrome - Chromium implementation (uses //content).

Content-embedders are all the products that are based on //content layer. Chromium, Chrome Headless, ChromeCast are all different //content embedders.

Electron is a //content embedder as well.

DevTools Protocol and content embedders

Majority of DevTools API is implemented in //blink and //content. However, certain methods are supposed to be implemented by embedders. DevTools team makes sure all the methods needed for Puppeteer operation are supported by both Chromium and Headless.

Electron is, however, missing out.

What exactly goes wrong?

• Since version v1.5.0, when Puppeteer connects to a browser it requests all existing browser
  contexts with the Target.getBrowserContexts() protocol method.
• Method Target.getBrowserContexts() is supposed to be implemented by content embedders.
• Electron doesn't implement the method, so Puppeteer connect to electron fails (pptr connect with electron failed #3793).

What can be done?

Option 1: Defer to Embedders.

Ideally, //content-embedders implement all necessary DevTools protocol methods on their end. This way they'll guarantee 100% compatibility with Puppeteer API. This, however, is a major undertaking for embedders; not sure if there's any interest in this.

Option 2: support clicking/typing/evaluation

Alternatively, we can aim for a "good-enough" compatibility with //content embedders. For example, we can make sure that things like clicking, navigation, typing and evaluation work fine with electron app (and add these electron tests to our CI). These should be enough to drive many default testing and automation scenarios.

🚀 Vote!

• If there's an interest in Option 1, please file a bug to the repository with the content embedder you'd like to be supported and ask them to implement the Target.getBrowserContexts() method. Plz cross-post the link here.
• If there's an interest in Option 2, please 👍 this issue and share your usecase.

[aslushnikov]

Option 2 seems to be good-enough for me to address the e2e tests of electron apps.

[JoelEinbinder]

Do we have a list of all the protocol methods that are embedder-implemented which puppeteer uses?

[aslushnikov]

@JoelEinbinder it's more or less everything that's implemented in //headless (minus "headlessexperimental" domain).

Most notable are browser contexts and ‘browser.close’.

[allavoie]

@aslushnikov: As option 2 might be the target, where will we find a test suite to base are own integration with this version ?

[aslushnikov]

where will we find a test suite to base are own integration with this version ?

@allavoie not sure I follow. Can you please elaborate what test suite would you need?

[allavoie]

@aslushnikov: I am referring to the tests you mentioned in your initial comment:

(and add these electron tests to our CI)

[MarshallOfSound]

For some added context / confirmation here the methods that embedders would need to implement are just these ones in DevtoolsManagerDelegate?: https://cs.chromium.org/chromium/src/content/public/browser/devtools_manager_delegate.h?l=22-94

Electron currently only implements a handful of these https://github.com/electron/electron/blob/master/atom/browser/ui/devtools_manager_delegate.h#L25-L34

Implementing all of them would be a decent chunk of work and probably a maintenance burden especially when not many folks on Electron are heavily invested in puppeteer or implementing the missing bits of this protocol.

If it's just getBrowserContexts that y'all need implemented we can probably do that but I don't know what surface area of that delegate puppeteer requires implemented to work 100% correctly.

If someone can point me at a conclusive list of APIs that Electron needs to implement for puppeteer to work I'm happy to look into it.

[tombenezra]

any updates or workaround? downgrading to 1.4.0 didn't solve the issue as it uses other DevTools protocol methods

[allavoie]

This combination seems to work:
"electron": "5",
"puppeteer-core": "1.15.0"

Refer to: https://github.com/peterdanis/electron-puppeteer-demo,

I had to do a small change to make it work, but the idea is there. I was able to generate a png screenshot based on this repo.
I wonder if there might me another dependency I am forgetting to mention. Please feel free to ask more info on my setup, as I do not know what more to add here.

[aslushnikov]

If someone can point me at a conclusive list of APIs that Electron needs to implement for puppeteer to work I'm happy to look into it.

@MarshallOfSound I'm happy to help here!

If it's just getBrowserContexts that y'all need implemented we can probably do that but I don't know what surface area of that delegate puppeteer requires implemented to work 100% correctly.

For a reasonable testing story, I think it's actually just a single method that is missing:

• Browser.close - this should close the electron instance, ignoring all the beforeunload hooks.

With this, it should be possible to use most of Puppeteer API's to test Electron apps. I think this will satisfy the majority of the usecases today.

[allavoie]

@aslushnikov, @MarshallOfSound , also ready to help, if possible.

[blohamen]

Hi, I hope I have a relevant problem. The problem is that we can't respond to intercepted requests correctly using puppeteer in electron. I have a code snippet

 await chartPage.setRequestInterception(true);
  await chartPage.on('request', (req) => {
    if (req.url().includes('TracsProxy')) {
      console.log(req);
      req.respond({
        status: 200,
        contentType: 'application/json',
        body: JSON.stringify(tracsProxyMock),
      });
    } else {
      req.continue();
    }
  });

I'm using puppeteer-core: 2.1.1
and electron 8.0.2