Importing module on filesystem from data URL throws TypeError: Invalid URL / ERR_UNSUPPORTED_RESOLVE_REQUEST: Invalid relative URL or base scheme is not hierarchical.

[benjamingwynn]

Version

v20.11.1

Platform

Linux devbox.home.arpa 6.1.0-16-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.67-1 (2023-12-12) x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Creating an index.mjs file with the following contents and running it with node index.mjs causes nodejs to be unable to resolve the foo module when running in base-64 land.

import fsp from 'fs/promises';

// create `foo` module
await fsp.mkdir('./node_modules/foo', { recursive: true });
await fsp.writeFile(
    './node_modules/foo/package.json',
    JSON.stringify({
        name: 'foo',
        exports: {
            '.': './index.mjs',
        },
    })
);

// build base64 url just importing `foo`
const importName = `foo`;
const unencoded = `await import("${importName}")`;
const encoded = Buffer.from(unencoded).toString('base64');
const url = `data:text/javascript;base64,${encoded}`;

// import that base 64 url
await import(url);

This seems to be because to resolve foo we try to load package.json's, but it seems like we're trying to do it from the data URL, which can't work.

How often does it reproduce? Is there a required condition?

You should run this code somewhere where it's safe to create a new node_modules.

Running the code always produces the same result on macOS and Linux. Untested on Windows.

What is the expected behavior? Why is that the expected behavior?

Either:

• nodejs should error with a clear description of the problem, rather than throw an internal error from URL
• (or) nodejs should allow importing from the data URL by using the context of the base64 module's parent for imports called from the base64 module

I think either this shouldn't try to do this and should error explicitly that it's not supported, or it should work if it's within scope/spec. I'd argue an internal error saying the URL is invalid (when both package.json and the data URL are valid) is confusing here for most nodejs developers.

What do you see instead?

node:internal/url:775
    this.#updateContext(bindingUrl.parse(input, base));
                                   ^

TypeError: Invalid URL
    at new URL (node:internal/url:775:36)
    at getPackageScopeConfig (node:internal/modules/esm/package_config:29:24)
    at packageResolve (node:internal/modules/esm/resolve:807:25)
    at moduleResolve (node:internal/modules/esm/resolve:910:20)
    at defaultResolve (node:internal/modules/esm/resolve:1130:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:396:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:365:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:240:38)
    at ModuleLoader.import (node:internal/modules/esm/loader:328:34)
    at importModuleDynamically (node:internal/modules/esm/translators:158:35) {
  code: 'ERR_INVALID_URL',
  input: './package.json',
  base: 'data:text/javascript;base64,YXdhaXQgaW1wb3J0KCJmb28iKQ=='
}

Node.js v20.11.1

Additional information

Potentially related to #51444 and #38714

Potentially related to subsystems:

@nodejs/loaders @nodejs/modules

Workaround: https://github.com/benjamingwynn/data-import-fix

[Pomax]

Note that Node v21 has slightly better information in its error output:

TypeError [ERR_UNSUPPORTED_RESOLVE_REQUEST]: Failed to resolve module specifier "canvas" from "data:text/javascript;base64:...[snip]...: Invalid relative URL or base scheme is not hierarchical.
    at moduleResolve (node:internal/modules/esm/resolve:914:23)
    at defaultResolve (node:internal/modules/esm/resolve:1148:11)
    ... 4 lines matching cause stack trace ...
    at link (node:internal/modules/esm/module_job:86:36) {
  code: 'ERR_UNSUPPORTED_RESOLVE_REQUEST',
  cause: TypeError: Invalid URL
      at new URL (node:internal/url:804:36)
      at moduleResolve (node:internal/modules/esm/resolve:911:18)
      at defaultResolve (node:internal/modules/esm/resolve:1148:11)
      at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:390:12)
      at ModuleLoader.resolve (node:internal/modules/esm/loader:359:25)
      at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:234:38)
      at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:87:39)
      at link (node:internal/modules/esm/module_job:86:36) {
    code: 'ERR_INVALID_URL',
    input: 'canvas'
  }
}

That said, I'm seeing the same error on Node 18 so probably not a regression, but still definitely a bug.

Also note that this cascades: even if you rewrite your own import to a file:/// location, Node will simply crash downstream because any subsequent imports/requires also try to look for package.json relative to the data uri instead of... you know... using the one that Node already loaded =)

As temporary workaround, I'm currently writing what should be a data-uri to a temporary .js, file and then running an exec("node tempfile.js", {stdio: "inherit"}) on that. That "works", but it's super cludgey.

[benjamingwynn]

@Pomax thanks for letting me know about node 21, I've updated the title to reflect what node 21 prints.

I also tried writing a workaround for this using the new module extension API. Could you test it out for your case and see if it's a valid solution?

https://www.npmjs.com/package/data-import-fix
https://github.com/benjamingwynn/data-import-fix

[anonrig]

cc @GeoffreyBooth any suggestions on how to fix this?

[benjamingwynn]

@anonrig as a temporary workaround, my solution above seems to work well for me

[daniel-nagy]

Another workaround is to use createRequire within the data URI. e.g.

const module = /* JavaScript */ `
  import { createRequire } from "node:module";
  const filename = "${import.meta.url}";
  const require = createRequire(filename);
  require("foo");
`;

import(`data:text/javascript,${module}`);

[Pomax]

A nice find! although a proper fix in Node is still very much required of course.