Add support for ESM modules and async plugin initializers.

[ShogunPanda]

Add support for ESM modules and async plugin initializers

Add support for ESM modules and async plugin initializers.

♻️ Current situation

Today, if you try to load a package which type=module in package.json it throws an error as it tries to use require() on a ESM module.

💡 Proposed solution

This PR introduces detection of ESM module and if detect it uses await import().

Implications

By using await import, by definition the entire plugin loading phase has become asynchronous. All affected code places have been refactored to be async function and to await for plugin loading and initialization.

Almost for free, HomeBridge plugins can now export a async function as default and this will be awaited. This means that now plugin can do async initialization.

Reviewer Nudging

Plugin and PluginManager code.

[Supereg]

Thanks for your PR.
I'm not sure about the support for async functions exported by a homebridge plugin, everything else looks good 👍
Have you tested this with a ESM module?

[ShogunPanda]

Yup, I tested with a custom module and it works awesomely. The only thing I haven't tested is child bridges.

[ShogunPanda]

Hello! Any chance for this to be merged?

[Fish2]

any news of pulling this in ?

[dgreif]

Hey @ShogunPanda, this looks great! Sorry it's been sitting for so long! If you don't mind addressing at least the import comment, then I'd be happy to spin this up locally and verify it's working against an ESM version of one of my plugins.

[ShogunPanda]

@dgreif As all comments are resolved, I just rebased my PR against the latest master, so this should be good to go.

[oznu]

Approved for beta branch.

Beta v1.3.6-beta.1

[oznu]

https://github.com/homebridge/homebridge/wiki/Homebridge-Beta-Testing

[bwp91]

On line 60 in plugin.ts, I believe this needs to be changed from

    this.main = packageJSON.main || "./index.js";

to something like

    this.main = packageJSON.main || packageJSON.exports || "./index.js";

As it seems an ESM package.json example "exports": "./lib/index.js", the path is not being respected.

Edit, maybe not. I didn't realise the "main" field was still recommended for a ES module too.

[ShogunPanda]

Yup, I think so. Probably I would have been more restrictive to something like:

this.main = packageJSON.main || (typeof packageJSON.exports === 'string' ? packageJSON.exports : null) || "./index.js";

But for now seems to be fine. Let's see if this becomes an issue.

[Supereg]

We have most likely an Issue related to this PR. Could one please investigate: #3013 (comment)?

EDIT: I think the issue is the migration of the for loop in the pluginManager.ts. Inside the plugin init error handling was a return (which worked previously) but now skips the whole plugin init iteration. Will prepare a fix, once the author confirms by suspicions.

[seydx]

Hey guys, does anyone else have the following problem on Windows?

Error: Only file and data URLs are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'c:'
    at new NodeError (node:internal/errors:371:5)
    at defaultResolve (node:internal/modules/esm/resolve:1016:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:422:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:222:40)
    at ESMLoader.import (node:internal/modules/esm/loader:276:22)
    at importModuleDynamically (node:internal/modules/cjs/loader:1041:29)
    at importModuleDynamicallyWrapper (node:internal/vm/module:437:21)
    at importModuleDynamically (node:vm:381:46)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:35:14)
    at eval (eval at <anonymous> (C:\Users\Long\AppData\Roaming\npm\node_modules\homebridge\src\plugin.ts:24:24), <anonymous>:3:1)

homebridge/src/plugin.ts

Line 24 in ef242e9

const _importDynamic = new Function("modulePath", "return import(modulePath)");

[ShogunPanda]

Most likely it will happen to everybody on Windows.
I don't have a Windows PC and only developed on MacOS or Linux.

[seydx]

Most likely it will happen to everybody on Windows.
I don't have a Windows PC and only developed on MacOS or Linux.

Will post a PR later with a fix

edit: see #3070