Start electron without a binary (i.e. electron main.js)

[tanin47]

I'd like to avoid building the full binary with electron-builder. I wonder if I can use wdio with electron main.js instead. This would be really helpful. Thank you.

Edit: to add a bit more info, I was trying to make run.sh which contains electron main.js and simulates a binary. But it didn't work.

[christian-bromann]

We could definitely do that. I didn't know you can start Electron applications directly like this. We could introduce an main or entryPoint property to the service options and if set, directly start the application through the provided file.

Wdyt @goosewobbler ?

[tanin47]

To add a benefit, it is much faster to start an electron app that way. Great for the code-test loop.

Thank you so much.

[goosewobbler]

Once this is supported by the service we should have another example / e2e directory for this too, something like example-no-binary or example-script?

[goosewobbler]

I prototyped this in #345 but I am currently unable to test locally as running Electron like this produces dyld[63749]: Library not loaded: @rpath/Electron Framework.framework/Electron Framework errors. Nothing I've tried to fix this works on my Mac atm

EDIT: found the culprit and workaround: pnpm/pnpm#7253

Updated the PR but obviously need a different approach...

[tanin47]

@goosewobbler I tried it out by injecting:

appBinaryPath = './node_modules/.bin/electron';
appArgs = ['--app=./dist/dev/main.js'];

to launcher.js.

It started the electron app but app.on('ready', ...) is not fired. It actually works correctly if I trigger the ready event manually using setTimeout(...).

I run the CLI manually with: ./node_modules/.bin/electron --app=./dist/dev/main.js. It works just fine (i.e. the ready event is fired).

2 questions:

• Do you know what the difference is when running the electron app through wdio / chromedriver?
• Is there a way to run chromedriver with electron app through CLI? I cannot figure out how to do it.

[tanin47]

Nvm. It actually works! I registered the ready event too late (many weird bugs in my codebase lolcry). When switching to app.whenReady().then(...), it works just fine.

One issue that I encounter is that the app is not closed after the test finishes. I can invoke app.quit() explicitly, but I didn't need to do it before. I wonder if you know how to get the app to quit after the test is run.

[goosewobbler]

Interesting, I can also get the app to load manually now - also setting the appBinaryPath and appArgs as above - but when run with WDIO the specs just hang. @christian-bromann might have some ideas about how to proceed.

[tanin47]

Mine didn't hang. The tests work fine (except the minor inconvenience mentioned above). Interestingly I have to use the prefix --app. A bare bone argument without --app doesn't work as it just starts the electron without loading any file.

[goosewobbler]

I've updated the PR for that, still hanging, and also with app.whenReady(). I wonder if there are some other chromium args we need to use...

[goosewobbler]

@tanin47 Do you have any available code for this? It would be useful to see what you did to enable this without the app hanging.

[tanin47]

I'm working on https://superintendent.app. The code is highly complex. It's difficult to share in a productive way.

I use webpack to build the code (with the watch option); This means the app is a single JS file. My issue is that the app doesn't close when the test finishes. But generally the tests work. Have written 10s of tests so far. Very productive.

Can you share how to run the test in this PR? https://github.com/webdriverio-community/wdio-electron-service/pull/345/files -- I have some time to help take a look to see whether there is a difference between my app and your test app.

Edit: nvm found it: test:integration:no-binary

[tanin47]

I am unable to run the test. @goosewobbler I wonder if you know how to unblock it. Here's the error:

Other tests (e.g. pnpm test:integration:esm, pnpm test:integration:cjs, pnpm test:integration:electron-builder) run successfully

[goosewobbler]

@tanin47 Thanks for taking a look - I'm stuck with the same error. Seems like a resolution issue for electron/main in the default_app.asar/main.js of the electron binary. If you log out the appBinaryPath and appArgs and run it manually, e.g. ~/Workspace/wdio-electron-service/example-no-binary/node_modules/.bin/electron --app=./example-no-binary/dist/main.bundle.js the app loads fine.

I also tried removing all other args (apart from app) and a few other things including switching the example-no-binary to CJS. No success yet...

[tanin47]

@goosewobbler Thank you!

There's something about Electron 28.x.x. I removed all args from launcher.js as shown below:

appBinaryPath = './node_modules/.bin/electron';
appArgs = [];

It should have simply started Electron, but it still fails with the default_app.asar/main.js error.

Interestingly Electron 19.x.x doesn't have this issue.

I also try passing --version. It again works as expected on v19, but v28 still fails with the default_app.asar/main.js error.

I wonder if appArgs even passes to Electron.

Can you try running ~/Workspace/wdio-electron-service/example-no-binary/node_modules/.bin/electron without any arg through Terminal? Does it work?

[goosewobbler]

I can confirm ~/Workspace/wdio-electron-service/example-no-binary/node_modules/.bin/electron in terminal works, it just opens the default Electron start window. And I had the same result running WDIO without the appArgs, still fails with the same error. These two results are with Electron v28.

For some reason downgrading Electron below v26 results in WDIO not running with 404s for Chromedriver thrown - I'll raise a separate issue for this. v26 and above runs the tests but fails with the default_app.asar/main.js error as expected. How were you able to test v19?

[tanin47]

For v19, it is on my project, not wdio-electron-service. v19 doesn't support esm nor cjs, so it doesn't work in the wdio-electron-service repo. I wonder if we should add another folder that doesn't require esm/cjs, which would make it testable with older electron versions.

[goosewobbler]

Electron has always supported CJS, no? ESM support was introduced in v28...

[tanin47]

Oh I didn't know that. I tested it. It works as expected when appArgs is [] (opening the default electron window) or ['--version'] (open and close immediately), but it doesn't work when passing ['--app=./dist/main.bundle.js']; it just opens Electron with no window. No error message. This looks like a JS error, not an electron error.

As a side note, I feel like it might be beneficial to have a simple test js that works across all electron versions and doesn't do anything else (e.g. like preloading).

For some reason downgrading Electron below v26 results in WDIO not running with 404s for Chromedriver thrown

v19 is old and you will need to download the chrome driver manually.

For electron v19, you will have to download "ChromeDriver 102.0.5005.61" from https://chromedriver.chromium.org/downloads. You can put it somewhere and set the wdio conf as follows:

exports.config = {
  services: ['electron'],
  capabilities: [
    {
      'browserName': 'electron',
      'wdio:electronServiceOptions': {
        // appArgs: ['foo', 'bar=baz'],
        restoreMocks: true,
      },
      'wdio:chromedriverOptions': {
        binary: '/Users/tanin/Downloads/chromedriver', // your path to the chrome driver.
      }
    } as WebdriverIO.Capabilities,
  ],

Please don't forget to bypass the security on Mac since this is a binary downloaded from internet.

Also, to switch Electron to v19, we can change package.json to point Electron to ^19.0.0.

I hope this is helpful.

[goosewobbler]

@tanin47 thanks for the investigation, let us know if you have any new insight or get any further with the issue blocking this feature. For now I think I will finish the last pieces of the mocking work (#368) and start to look at #412. I raised #422 to cover improving the UX around using e.g. Electron 19.

[tanin47]

Thank you. I'm not blocked since I can modify launcher.js directly. I think we have a good hypothesis that v19 might work. Solving #422 sounds like a good next step since it is inherently good and will help validate this hypothesis.

[goosewobbler]

I took another look at this and got rid of the default_app.asar/main.js error - the culprit was the workaround for a PNPM issue with Electron which has since been resolved. I updated PNPM and removed the workaround, however now instead of an error the app just times out and is respawned infinitely:

2024-04-10T14:59:41.231Z WARN webdriver: Request timed out! Consider increasing the "connectionRetryTimeout" option.

This happens regardless of which Electron version is used, I am testing with 27 / 28 / 29 and 19.

You can see my working in the sm/electron-script-exec-test branch, my local had different commits and it was a long time since I last looked at this so rather than risk losing anything I created a new branch to play around in.

I also logged out the args that are being applied to the binary, listed here.

[tanin47]

Thank you. I'm updating superintendent.app from version 19 to 29, and now my tests are not working. The error is default_app.asar/main.js:

I'll check out your branch and report back.

[tanin47]

Actually, my error is different. It says ENOTDIR and that the file doesn't exist.

Starting ./node_modules/.bin/electron --app=./dist/dev/main.js works normally. This is perplexing.

Edit: debug a bit more. I think there might be something wrong with the chromedriver itself.

We can see its log by adding the below to launcher.js:

cap['wdio:chromedriverOptions']['verbose'] = true
cap['wdio:chromedriverOptions']['log-path'] = '/tmp/chromedriver.log'

In the log, it prints a valid command: [1713337369.701][INFO]: Launching chrome: ./node_modules/.bin/electron --allow-pre-commit-input --app=./dist/dev/main.js --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --enable-automation --enable-logging --log-level=0 --no-first-run --no-service-autorun --password-store=basic --remote-debugging-port=0 --test-type=webdriver --use-mock-keychain --user-data-dir=/var/folders/xf/_dg8lbj13rx87n5_v9p3ywn00000gn/T/.org.chromium.Chromium.va2mBE data:

However Launching chrome: is constructed (ref). It's not what is actually used, which is base::Process process = base::LaunchProcess(command, options);

The printed command confirms that everything is sent correctly to chromedriver. This must means there's a bug in launching the process.

[tanin47]

I've digged a little bit deeper and am not quite sure whether the bug lies in chromedriver.

I ran chromedriver manually with: ./chromedriver --port=49850 --binary=./chromedriver --verbose --log-path=/tmp/chromedriver.log --allowed-origins=* --allowed-ips=0.0.0.0

Then, I fired a request from postman to: http://0.0.0.0:49850/session with the body:

{
    "capabilities": {
        "alwaysMatch": {
            "browserName": "chrome",
            "wdio:electronServiceOptions": {},
            "wdio:chromedriverOptions": {
                "binary": "./chromedriver",
                "verbose": true,
                "log-path": "/tmp/chromedriver.log",
                "allowedOrigins": [
                    "*"
                ],
                "allowedIps": [
                    "0.0.0.0"
                ]
            },
            "goog:chromeOptions": {
                "binary": "./node_modules/.bin/electron",
                "windowTypes": [
                    "app",
                    "webview"
                ],
                "args": [
                    "--app=./dist/dev/main.js"
                ]
            },
            "browserVersion": "122.0.6261.156"
        },
        "firstMatch": [{}]
    },
    "desiredCapabilities": {
        "browserName": "chrome",
        "wdio:electronServiceOptions": {},
        "wdio:chromedriverOptions": {
            "binary": "./chromedriver",
            "verbose": true,
            "log-path": "/tmp/chromedriver.log",
            "allowedOrigins": [
                "*"
            ],
            "allowedIps": [
                "0.0.0.0"
            ]
        },
        "goog:chromeOptions": {
            "binary": "./node_modules/.bin/electron",
            "windowTypes": [
                "app",
                "webview"
            ],
            "args": [
                "--app=./dist/dev/main.js"
            ]
        },
        "browserVersion": "122.0.6261.156"
    }
}

and it works!

But if the chromedriver is started from node_modules/@wdio/utils/build/node/startWebDriver.js, then it doesn't work.

I wonder if there's some sort of security issues instead that might prevent chromedriver from functioning correctly when being spawn inside node env.

[tanin47]

I am unblocked. I can start the chromedriver separately and set the port in wdio.conf.ts in order to avoid wdio spawning chromedriver. It's odd that this works.

[goosewobbler]

@tanin47 That's some great debug there - thanks for your efforts, I haven't had a lot of time to dedicate to this. So you're starting Chromedriver outside of Node and it works...

@christian-bromann any ideas here? Perhaps the process from startWebDriver isn't being spawned with the correct options?

https://github.com/webdriverio/webdriverio/blob/main/packages/wdio-utils/src/node/startWebDriver.ts#L86
https://nodejs.org/api/child_process.html#child_processspawncommand-args-options

[christian-bromann]

any ideas here? Perhaps the process from startWebDriver isn't being spawned with the correct options?

Note sure, @tanin47 do you have any logs to share here?

[tanin47]

Let me get the log for you.

I wanted to share a bit more:

• For cp.spawn(chromedriverExcecuteablePath, driverParams) on this line, if we swap it with cp.execFileSync(...); it'll block. Then, I'd fire an http request manually to initiate a session, and it would not work.
• Adding {shell: true} doesn't make it work.
• I started the chromedriver in the node console using execFileSync with and without {shell: true}, and it works just fine. (so this problem is not really specific to node???)
• Please note that I use a manually downloaded chromedriver, so all of these definitely use the same chromedriver binary.

There's something specific about starting chromedriver within wdio. I suspect it is either some weird env variables or security issues. I intend to debug more today.

[tanin47]

Nvm. I've figured out the culprit. It's the env variable: NODE_OPTIONS=' --loader ts-node/esm/transpile-only --no-warnings'

You can start 2 chromedrivers from terminal and try to create a session:

• NODE_OPTIONS=' --loader ts-node/esm/transpile-only --no-warnings' ./chromedriver --port=49427 --binary=./chromedriver --allowed-origins=* --allowed-ips=0.0.0.0 -- this will fail with the error above.
• ./chromedriver --port=49428 --binary=./chromedriver --allowed-origins=* --allowed-ips=0.0.0.0 -- this will succeed.

It seems NODE_OPTIONS is used by newer chromedrivers or newer electrons or both.

In other words, if you modify the line in startWebDriver.js to: driverProcess = cp.spawn(chromedriverExcecuteablePath, driverParams, { env: { ...process.env, NODE_OPTIONS: null } });, it works!

[tanin47]

Nvm again. It is used by Electron. We can reproduce the error by running NODE_OPTIONS=' --loader ts-node/esm/transpile-only --no-warnings' ./node_modules/.bin/electron

[goosewobbler]

@tanin47 excellent detective work again! I had time to try { detached: true } on the CD process spawn, no change, glad you took it a few steps further. So ts-node is the culprit. I verified that the above reproduction fails with the asar main.js error again. Dropping the --loader switch but specifying some other options or an empty string runs the app with a specified --app= entrypoint.

This is where the NODE_OPTIONS is being set: https://github.com/webdriverio/webdriverio/blob/main/packages/wdio-local-runner/src/worker.ts#L129

@christian-bromann is ts-node going to be replaced with tsx?

If we use tsx, e.g. NODE_OPTIONS=' --import tsx/esm' ./node_modules/.bin/electron the app also runs.

[christian-bromann]

is ts-node going to be replaced with tsx?

It seems like tsx has a higher development velocity and seems like a modern version of ts-node which is why I created the issue. If this is something that would also solve problems here, that is good feedback. That said, no one has picked it up yet.

[goosewobbler]

That makes sense. I might prototype it if I get time...

[tanin47]

Thank you so much folks. On a good note, wdio has been working well for me. I have 20+ tests and can run it on Windows, Mac, and Linux.

On Windows, there's a different issue. I'm not sure if it is because, on Windows, ./node_modules/.bin/electron is not a real binary but a wrapper that runs node electron.js. But I don't use Windows to develop, and a compiled Electron binary works fine with wdio, so it's fine

As a side note, it seems there are differences between ./node_modules/.bin/electron and a compiled Electron app. ./node_modules/.bin/electron seems susceptible various configurations from env variables but a compiled Electron app isn't. Reusing these node envs might conflict with what Electron uses.

[goosewobbler]

On Windows, there's a different issue. I'm not sure if it is because, on Windows, ./node_modules/.bin/electron is not a real binary but a wrapper that runs node electron.js. But I don't use Windows to develop, and a compiled Electron binary works fine with wdio, so it's fine

The Windows case is a bit different, and yes the wrapper might have an impact. I think we restrict this issue to the Mac & Linux cases and follow up as necessary.

As a side note, it seems there are differences between ./node_modules/.bin/electron and a compiled Electron app. ./node_modules/.bin/electron seems susceptible various configurations from env variables but a compiled Electron app isn't. Reusing these node envs might conflict with what Electron uses.

I see. Can this be distilled into an issue or is it just something to be aware of?

I finally got this working after roughly replacing ts-node in WDIO with tsx locally, will update the PR here. The PR for WDIO will take more time, ts-node is referenced in a lot of places.

[tanin47]

I see. Can this be distilled into an issue or is it just something to be aware of?

Both WDIO and Electron utilizes Node and looks at the same set of envs. The conflicts are bound to happen. I'm not sure if it is possible to filter out all Node-related envs before starting a chrome driver (or is that even a right solution?).

Thank you so much again!

EDIT: Electron doesn't generally accept node config. They only accept NODE_OPTIONS. In packaged app, it is disallowed. (ref).

To answer the question, we can open a feature request to WDIO to support configuring NODE_OPTIONS for chromedriver. Today the current NODE_OPTIONS (which is intended for WDIO / npm / build env itself) is accidentally carried over to chromedriver / Electron, and that is problematic.

[goosewobbler]

To answer the question, we can open a feature request to WDIO to support configuring NODE_OPTIONS for chromedriver. Today the current NODE_OPTIONS (which is intended for WDIO / npm / build env itself) is accidentally carried over to chromedriver / Electron, and that is problematic.

Ok, well this is definitely a discussion for the WDIO issue - can you create one and link it here?

I got somewhere in replacing ts-node with tsx: webdriverio/webdriverio#12752

[goosewobbler]

WDIO no longer uses ts-node, we can move forward with this when v9 is released.