feat(compartment-mapper): Import and export subpath patterns by kriskowal · Pull Request #3048 · endojs/endo

kriskowal · 2026-01-09T07:01:45Z

Closes: #2897

Description

Adds support for Node.js subpath pattern replacement in package.json exports and imports fields to the compartment-mapper. This enables patterns like:

{
  "exports": {
    "./features/*.js": "./src/features/*.js"
  },
  "imports": {
    "#internal/*.js": "./lib/*.js"
  }
}

The implementation matches Node.js semantics where * is a string replacement token that matches any substring, including across / path separators. Patterns are matched by specificity: the pattern with the longest matching prefix wins, and exact entries always take precedence over wildcard patterns.

Security Considerations

Pattern-matched module imports go through the same policy enforcement (enforcePolicyByModule) as other module resolutions. Cross-compartment patterns (from dependency exports) resolve within the dependency's compartment, maintaining compartment isolation. The prefix/suffix string matching approach avoids regex-based matching, eliminating potential ReDoS concerns.

Scaling Considerations

Pattern matching is O(p) where p is the number of wildcard patterns, sorted by prefix length for specificity. Exact entries are checked first via a Map lookup. Results are cached via write-back to moduleDescriptors, so each specifier is matched at most once.

Documentation Considerations

Users can now use wildcard patterns in their package.json exports and imports fields when using compartment-mapper. The patterns follow Node.js subpath pattern semantics:

* matches any substring, including across / separators
Exact entries take precedence over wildcard patterns
Longer prefix wins when multiple patterns match
Null targets (null values) exclude subpaths from resolution
Conditional pattern values are resolved through the same condition-matching rules as non-pattern exports
Globstar (**) patterns are silently ignored (matching Node.js behavior)
Array fallback values are deliberately not supported (see design doc for rationale)
Pattern and replacement must have matching wildcard counts

Testing Considerations

13 unit tests cover the pattern matcher: exact match, wildcard matching, cross-/ matching, specificity ordering, #-imports patterns, null-target exclusion, error cases (globstar rejection, wildcard count mismatch), and various input formats (tuples, PatternDescriptor array, record object)
Scaffold integration tests verify the full pipeline through loadLocation, importLocation, mapNodeModules, makeArchive, parseArchive, writeArchive, loadArchive, importArchive
Node.js parity tests run the same fixtures under plain Node.js, ensuring behavioral equivalence by construction. Assertions are shared via _subpath-patterns-assertions.js.
User-specified conditions tested via ses-ava with --conditions=blue-moon to verify the correct branch is selected by both Node.js and the Compartment Mapper
Policy enforcement tested: pattern-matched imports allowed when package is permitted, rejected when not
Archive stripping tested: inspects compartment-map.json in archive and asserts no compartment has a patterns property
Tests verify that patterns work correctly through archiving (patterns are expanded to concrete module entries)

Compatibility Considerations

This is a purely additive feature. Existing packages without wildcard patterns are unaffected. The patterns field is only added to compartment descriptors when patterns exist (using conditional spread), so existing snapshot tests pass without modification.

Upgrade Considerations

No breaking changes. Packages can start using wildcard patterns in exports/imports immediately after upgrading to a version with this feature.

boneskull

No real concerns here. Not even nits, really

boneskull · 2026-01-09T19:34:59Z

packages/compartment-mapper/src/types/node-modules.ts

+   * Package imports field for self-referencing subpath patterns.
+   * Keys must start with '#'.
+   */
+  imports?: unknown;


(suggestion)

Steal this code for the type defs of exports and imports: https://github.com/sindresorhus/type-fest/blob/850b33c4dd292e0ff8cff039ee167d69be324fce/source/package-json.d.ts#L227-L248

boneskull · 2026-01-09T19:36:15Z

packages/compartment-mapper/src/types/pattern-replacement.ts

+export type SubpathMapping =
+  | Array<[pattern: string, replacement: string]>
+  | Record<string, string>;


👍 for named tuple elements

boneskull · 2026-01-09T19:36:50Z

packages/compartment-mapper/src/infer-exports.js

+ * The imports field provides self-referencing subpath patterns that
+ * can be used to create private internal mappings.
+ *
+ * @param {string} _name - the name of the package (unused, but kept for consistency)


Consistency with what?

boneskull · 2026-01-09T19:37:17Z

packages/compartment-mapper/src/infer-exports.js

+  }
+  for (const [key, value] of entries(imports)) {
+    // imports keys must start with '#'
+    if (!key.startsWith('#')) {


(suggestion) warn

boneskull · 2026-01-09T19:38:10Z

packages/compartment-mapper/src/infer-exports.js

+ * @param {string} value
+ * @returns {boolean}
+ */
+const hasWildcard = (key, value) => key.includes('*') || value.includes('*');


Isn't it also important to know which of these two values contains the wildcard(s)?

boneskull · 2026-01-09T20:22:11Z

packages/compartment-mapper/src/pattern-replacement.js

+ * @param {string} segment
+ * @returns {boolean}
+ */
+const hasWildcard = segment => segment.includes(WILDCARD);


(could) Refactor and share with infer-exports.js

boneskull · 2026-01-09T20:23:03Z

packages/compartment-mapper/src/pattern-replacement.js

+    return patternSegment === specifierSegment ? '' : null;
+  }
+
+  const wildcardIndex = patternSegment.indexOf(WILDCARD);


(could) if we're able to assert here, I'd probably want to assert that wildcardIndex is a non-negative integer

boneskull · 2026-01-09T20:24:05Z

packages/compartment-mapper/src/pattern-replacement.js

+        `Globstar (**) patterns are not supported in pattern: "${pattern}"`,
+      );
+    }
+    if (replacement.includes(GLOBSTAR)) {


(could) hasGlobstar()

boneskull · 2026-01-09T20:25:35Z

packages/compartment-mapper/test/pattern-replacement.test.js

+  t.is(node.value, null);
+  t.deepEqual(Object.keys(node.children), []);


(could) refactor to t.like(node, {value: null, children: []}) assuming it works the way I think it should

boneskull · 2026-01-09T20:33:46Z

packages/compartment-mapper/test/pattern-replacement.test.js

+});
+
+test('assertMatchingWildcardCount - throws for mismatched counts', t => {
+  const error = t.throws(() => assertMatchingWildcardCount('./*/a/*', './*'));


(complaining) t.throws() is weaksauce.

expect( () => assertMatchingWildcardCount('./*/a/*', './*'), 'to throw', { message: expect.it( 'to match', /wildcard count mismatch/i, 'and', 'to match', /2/, 'and', 'to match', /1/ ) } );

(imperative) feel the power of BUPKIS

changeset-bot · 2026-03-22T06:39:21Z

🦋 Changeset detected

Latest commit: e9d5c73

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 7 packages

Name	Type
@endo/compartment-mapper	Minor
@endo/bundle-source	Patch
@endo/check-bundle	Patch
@endo/daemon	Patch
@endo/import-bundle	Patch
@endo/test262-runner	Patch
@endo/cli	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

boneskull

See suggested fix to the conditional in interpretImports. Once this is addressed I'll approve.

Otherwise, just a bunch of questions and suggestions.

boneskull · 2026-03-27T20:38:49Z

packages/compartment-mapper/designs/subpath-pattern-replacement.md

+Node.js allows array values in exports as fallback lists, where each
+entry is tried in order and the first file that exists on disk is used.
+Pattern resolution in the compartment-mapper is a pure string operation
+with no filesystem access.
+Array fallbacks would require threading read powers through the pattern
+matcher and changing the `SubpathReplacer` signature.
+Node.js documentation discourages array fallbacks.
+If a pattern value is an array, `interpretExports` yields all elements
+as separate entries and the first match wins without fallback probing.


This seems to imply that we don't have 1:1 parity with Node.js, is this correct?

boneskull · 2026-03-27T20:39:32Z

packages/compartment-mapper/src/types/compartment-map-schema.ts

+   * `*` matches any substring including `/` (Node.js semantics).
+   * Stripped during digest/archiving - expanded patterns become concrete module entries.
+   */
+  patterns?: PatternDescriptor[];


Suggested change

patterns?: PatternDescriptor[];

patterns?: Array<PatternDescriptor>;

I'm pretty sure we're almost exclusively using this form.

boneskull · 2026-03-27T20:41:16Z

packages/compartment-mapper/src/types/node-modules.ts

+ * maps to an {@link Exports} value.
+ */
+export type ExportConditions = {
+  // eslint-disable-next-line no-use-before-define


We should disable this rule entirely in **/*.ts as ESLint doesn't understand type scoping. TS sources should use tseslint's implementation instead.

This and the next two types might also want an attribution to type-fest, since it appears identical 😄

boneskull · 2026-03-27T20:45:21Z

packages/compartment-mapper/src/types/compartment-map-schema.ts

  path: never;
  retained: never;
  scopes: never;
+  patterns: never;


This appears redundant, since this field does not appear in CompartmentMapDescriptor. Its presence in a DigestedCompartmentDescriptor should cause an error, even without the above addition.

Suggested change

patterns: never;

boneskull · 2026-03-27T20:47:32Z

packages/compartment-mapper/src/types/pattern-replacement.ts

+   * When absent, the pattern resolves within the owning compartment.
+   * Set when propagating export patterns from a dependency package.
+   */
+  compartment?: string;


I could have sworn we had exactOptionalPropertyTypes enabled. Maybe not.

boneskull · 2026-03-27T21:01:41Z

packages/compartment-mapper/src/pattern-replacement.js

+      const wildcardIndex = pattern.indexOf('*');
+      if (wildcardIndex === -1) {
+        exactEntries.set(pattern, { replacement: null, compartment });
+      } else {
+        const prefix = pattern.slice(0, wildcardIndex);
+        const suffix = pattern.slice(wildcardIndex + 1);
+        wildcardEntries.push({
+          prefix,
+          suffix,
+          replacementPrefix: null,
+          replacementSuffix: null,
+          compartment,
+        });
+      }


(could) split out into function, and L138-L150 as well

boneskull · 2026-03-27T21:03:14Z

packages/compartment-mapper/src/pattern-replacement.js

+
+  // Sort wildcard entries by prefix length descending for specificity.
+  // Node.js selects the pattern with the longest matching prefix.
+  wildcardEntries.sort((a, b) => b.prefix.length - a.prefix.length);


I'm gonna guess wildcardEntries doesn't get big enough to start worrying about incurring another loop over it.

boneskull · 2026-03-27T21:06:02Z

packages/compartment-mapper/src/pattern-replacement.js

+          entry.prefix.length,
+          specifier.length - entry.suffix.length,
+        );
+        const result = `${entry.replacementPrefix}${captured}${entry.replacementSuffix}`;


do we have some validation code that ensures there isn't anything fishy in exports/imports like /etc/pwd

boneskull · 2026-03-27T21:07:42Z

packages/compartment-mapper/test/subpath-patterns-node-parity.test.js

+  await t.throwsAsync(
+    () => import(new URL('app/multi-star-import.js', fixtureDir).href),
+    {
+      code: 'ERR_PACKAGE_PATH_NOT_EXPORTED',


does this actually work?? I didn't think AVA knew about code.

boneskull · 2026-03-27T21:08:27Z

CONTRIBUTING.md

+- Prefer `/** @import */` over dynamic `import()` in JSDoc type annotations.
+  Use a top-level `/** @import {Foo} from 'bar' */` comment instead of inline
+  `{import('bar').Foo}` in `@param`, `@type`, or `@returns` tags.


kriskowal requested a review from boneskull January 9, 2026 07:01

boneskull approved these changes Jan 9, 2026

View reviewed changes

kriskowal force-pushed the kriskowal-subpath-pattern-replacement branch from e72403c to fca6c85 Compare March 22, 2026 06:39

kriskowal force-pushed the kriskowal-subpath-pattern-replacement branch 3 times, most recently from df71dc8 to 122ea31 Compare March 23, 2026 04:35

kriskowal requested a review from turadg March 23, 2026 04:36

kriskowal marked this pull request as ready for review March 23, 2026 04:37

kriskowal force-pushed the kriskowal-subpath-pattern-replacement branch from 122ea31 to 8f8db1e Compare March 23, 2026 04:44

feat(compartment-mapper): Import and export subpath patterns

e9d5c73

kriskowal force-pushed the kriskowal-subpath-pattern-replacement branch from 8f8db1e to e9d5c73 Compare March 23, 2026 04:51

boneskull added the enhancement New feature or request label Mar 24, 2026

boneskull requested changes Mar 27, 2026

View reviewed changes

		t.is(node.value, null);
		t.deepEqual(Object.keys(node.children), []);

	patterns?: PatternDescriptor[];
	patterns?: Array<PatternDescriptor>;

Conversation

kriskowal commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Security Considerations

Scaling Considerations

Documentation Considerations

Testing Considerations

Compatibility Considerations

Upgrade Considerations

Uh oh!

boneskull left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

changeset-bot bot commented Mar 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

boneskull left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kriskowal commented Jan 9, 2026 •

edited

Loading

changeset-bot bot commented Mar 22, 2026 •

edited

Loading