Loading native Node module fails with "stream did not contain valid UTF-8"

[ealib]

Deno

• deno 1.29.1 (release, x86_64-pc-windows-msvc)
• v8 10.9.194.5
• typescript 4.9.4

Output

error: Unable to load C:\Users\UserName\AppData\Local\deno\npm\registry.npmjs.org\node-mdaemon-api\22.0.3-alpha.19\node-mdaemon-api.node imported from file:///C:/tmp/test-native.ts

Caused by:
    stream did not contain valid UTF-8

How to reproduce

Command

deno run --unstable --allow-all test-native.ts

Script test-native.ts is

import { getModuleInfo } from 'npm:node-mdaemon-api'; 

const moduleInfo = getModuleInfo();

console.dir(moduleInfo);

[ealib]

For comparison, this script

'use strict'

const md = require('node-mdaemon-api');

const moduleInfo = md.getModuleInfo();

console.dir(moduleInfo);

works correctly under Node v18.13.0.

[bartlomieju]

Node.js refuses to load this file:

node:internal/errors:484
    ErrorCaptureStackTrace(err);
    ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".node" for /Users/ib/dev/test_napi/asdf/asdf.darwin-arm64.node
    at new NodeError (node:internal/errors:393:5)
    at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:75:9)
    at defaultGetFormat (node:internal/modules/esm/get_format:114:38)
    at defaultLoad (node:internal/modules/esm/load:81:20)
    at nextLoad (node:internal/modules/esm/loader:161:28)
    at ESMLoader.load (node:internal/modules/esm/loader:594:26)
    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:446:22)
    at new ModuleJob (node:internal/modules/esm/module_job:64:26)
    at #createModuleJob (node:internal/modules/esm/loader:469:17)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:423:34) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

From what I gather native extensions must always be require()d. We need to improve error message here.

[ealib]

Thank you @bartlomieju for taking the time to verify this behaviour. Unfortunately I forgot to point out that the native node-mdaemon-api module (I am the author) only exists for Windows x64, not for Mac.

I created a repository with these scripts by specifying exactly the requirements in the package.json and, in particular, "type": "module", to force Node to operate in ESM mode, thus explicitly requiring native modules to be loaded with require, as you observed.

[bartlomieju]

Thank you @bartlomieju for taking the time to verify this behaviour. Unfortunately I forgot to point out that the native node-mdaemon-api module (I am the author) only exists for Windows x64, not for Mac.

No worries, I figured that out and created a simple project that does the same thing but can be compiled on ARM.

I created a repository with these scripts by specifying exactly the requirements in the package.json and, in particular, "type": "module", to force Node to operate in ESM mode, thus explicitly requiring native modules to be loaded with require, as you observed.

Sure, I'll try to look into fixing this problem next week - we should at least have a nice error saying what's wrong here. Side note: it's surprising that there's still no way to import Node API modules via ESM.

[ealib]

With

C:\>deno --version
deno 1.33.3 (release, x86_64-pc-windows-msvc)
v8 11.4.183.2
typescript 5.0.4

the error message has changed

error: Unable to load C:\Users\UserName\AppData\Local\deno\npm\registry.npmjs.org\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api\node-mdaemon-api.node imported from file:///C:/tmp/test-native.ts

Caused by:
    invalid utf-8 sequence of 1 bytes from index 2

[ealib]

As @loynoir suggested in #19130 , I tried to directly import the native binary module, but Deno fails with the same error:

C:\Users\UserName\deno-issue-17246>deno repl -A
Deno 1.33.4
exit using ctrl+d, ctrl+c, or close()
> const md = await import('npm:node-mdaemon-api@23.0.0-alpha.21/node-mdaemon-api.node');
Uncaught TypeError: Unable to load C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api\node-mdaemon-api.node
  Caused by:
    invalid utf-8 sequence of 1 bytes from index 2
    at async <anonymous>:2:13
>