feat(metro-config): add debug id support #25935

EvanBacon · 2023-12-14T05:59:02Z

Why

Injecting the debugId is a complex operation that doesn't fit nicely to any established plugin systems due to the way it requires a content hash, followed by a conversion to hermes bytecode. Other modern bundlers, specifically bun, also provide the debugId annotation, even though it seems to be primarily leveraged by the creators of the semantic––Sentry.

By adding this upstream, we can expose simple hooks for sentry to inject virtual runtime modules that access the debugId while leaving the more complex operation to expo/metro-config (which has robust testing).

How

Add two implementations of the debugId feature:

multi-asset serializer: this is used in npx expo export
legacy serializer: used in npx expo export:embed

Both follow a similar semantic of creating a debugId before the final bundle is processed. Neither serializer injects any debug Id info when inline source maps are enabled or when source maps are disabled.

In the multi-asset serializer, we inject the debug Id in the hermes source maps too. This is currently unused but will be leveraged for React Server Components in the future.

We may need a follow up PR like #25859 to expose hooks for sentry to inject runtime modules.

Test Plan

Only tested against the internal suite, this may have inconsistencies against various production native builds but that seems unlikely given the heavy restrictions we've placed on modifying the metro serializer.

Co-authored-by: Expo Bot <34669131+expo-bot@users.noreply.github.com>

krystofwoldrich · 2023-12-14T10:43:13Z

docs/pages/versions/unversioned/config/metro.mdx

+
+The `debugId` is a deterministic hash of the bundle's contents without the external bundle splitting references. This is effectively the same value used to create a chunks filename, but formatted as a UUID, e.g. `431b98e2-c997-4975-a3d9-2987710abd44`.
+
+`@expo/metro-config` injects `debugId` during `npx expo export` and `npx expo export:embed`. Any additional optimization steps in `npx expo export:embed` like Hermes bytecode generation will need to have the `debugId` injected manually.


The description reads well and matches the expected behavior/the current Debug IDs behavior with Sentry Serializer in bare RN apps.

Sentry currently has these guidelines https://docs.sentry.io/platforms/react-native/sourcemaps/#manual-upload-with-hermes for bare RN. We will add a similar for npx expo export:embed.

krystofwoldrich · 2023-12-14T10:46:49Z

packages/@expo/metro-config/src/serializer/__tests__/withExpoSerializers.test.ts

        sourceMaps: false,
      },
    });
-    expect(typeof str).toBe('string');
+    expect(typeof bundle).toBe('object');
    // Ensure the module is run.
-    expect(str).toMatch(/TEST_RUN_MODULE\("\/app\/index\.js"\);/);
-    expect(str).toMatch(/expo-mock\/async-require/);
+    expect(bundle.code).toMatch(/TEST_RUN_MODULE\("\/app\/index\.js"\);/);
+    expect(bundle.code).toMatch(/expo-mock\/async-require/);
+    expect(bundle.map).toMatch(/debugId/);


This is unexpected to me, why is the source map with debug ID present when the sourceMaps: false option is set?

Is this the export:embed scenario?

This is an issue with how export:embed works, the more recent versions don't perform additional work.

packages/@expo/metro-config/src/serializer/serializeChunks.ts

krystofwoldrich · 2023-12-14T11:20:58Z

packages/@expo/metro-config/src/serializer/serializeChunks.ts

+      // NOTE: Sentry does this, but bun does not.
+      // sourceMapObject.debug_id = debugId;


Just confirming only one attribute debugId will work.

Sentry Serializer includes both for legacy reasons, which can be omitted when implementation this feature now.

krystofwoldrich · 2023-12-14T13:12:10Z

packages/@expo/metro-config/src/serializer/withExpoSerializers.ts

+    if (config.serializer?.customSerializer) {
+      const bundle = await config.serializer?.customSerializer(
+        entryPoint,
+        preModules,
+        graph,
+        options
+      );
+      if (typeof bundle === 'string') {
+        bundleCode = bundle;
+      } else {
+        bundleCode = bundle.code;
+        bundleMap = bundle.map;
+      }
+    } else {


The default serializer is not created if there is an existing custom serializer. -> https://github.com/expo/expo/pull/25935/files#diff-a07d4017951693f251b9c918dec8b9f2a2d2896f77d212181e6d98444050eeceR172

The custom serializer might not generate a debug ID we should not try to fill in the map and generate the ID for it. As it might be loose not matching any bundle.

I'm proposing this: @evanbacon/metro-config/debug-id-support...krystofwoldrich:expo:kw-add-debug-id-to-default-serializer

Seems fine, but the sentry plugin would possibly not use this as it would only apply to one possible export path, export:embed. If we expose a more limited function like modifyDeps(preModules, { debugId }) then it will work in both code paths.

docs/pages/versions/unversioned/config/metro.mdx

docs/pages/versions/v50.0.0/config/metro.mdx

Co-authored-by: Aman Mittal <amandeepmittal@live.com>

# Why Expose experimental plugins to run before the serialization of each code asset. ```ts unstable_beforeAssetSerializationPlugins?: ((serializationInput: { graph: ReadOnlyGraph<MixedOutput>; premodules: Module[]; debugId?: string; }) => Module[])[]; ``` ### Implementation notes These are different from the serialization processors which run on the entirety of the code before it is potentially bundle split. Once the code is split into their respective chunks, we run the plugins before each chunk is serialized. ### Intended use These plugins are intended to expose the premodules and the debugId introduced from #25935 ```js // metro.config.js const { getDefaultConfig } = require("expo/metro-config"); function unstable_beforeAssetSerializationPlugin({graph, premodules, debugId}: { graph: ReadOnlyGraph<MixedOutput>; premodules: Module[]; debugId?: string; }) : Module[] { const modifiedPremodules = modifyYourModules(premodules, debugId); return modifiedPremodules; } module.exports = getDefaultConfig(projectRoot, {unstable_beforeAssetSerializationPlugins: [unstable_beforeAssetSerializationPlugin]}); ``` --------- Co-authored-by: Brent Vatne <brentvatne@gmail.com>

# Why Injecting the debugId is a complex operation that doesn't fit nicely to any established plugin systems due to the way it requires a content hash, followed by a conversion to hermes bytecode. Other modern bundlers, specifically bun, also provide the `debugId` annotation, even though it seems to be primarily leveraged by the creators of the semantic––Sentry. By adding this upstream, we can expose simple hooks for sentry to inject virtual runtime modules that access the debugId while leaving the more complex operation to expo/metro-config (which has robust testing).  # How Add two implementations of the debugId feature: 1. multi-asset serializer: this is used in `npx expo export` 2. legacy serializer: used in `npx expo export:embed` Both follow a similar semantic of creating a debugId before the final bundle is processed. Neither serializer injects any debug Id info when inline source maps are enabled or when source maps are disabled. In the multi-asset serializer, we inject the debug Id in the hermes source maps too. This is currently unused but will be leveraged for React Server Components in the future. We may need a follow up PR like expo#25859 to expose hooks for sentry to inject runtime modules.  # Test Plan Only tested against the internal suite, this may have inconsistencies against various production native builds but that seems unlikely given the heavy restrictions we've placed on modifying the metro serializer. --------- Co-authored-by: Expo Bot <34669131+expo-bot@users.noreply.github.com> Co-authored-by: Aman Mittal <amandeepmittal@live.com>

# Why Expose experimental plugins to run before the serialization of each code asset. ```ts unstable_beforeAssetSerializationPlugins?: ((serializationInput: { graph: ReadOnlyGraph<MixedOutput>; premodules: Module[]; debugId?: string; }) => Module[])[]; ``` ### Implementation notes These are different from the serialization processors which run on the entirety of the code before it is potentially bundle split. Once the code is split into their respective chunks, we run the plugins before each chunk is serialized. ### Intended use These plugins are intended to expose the premodules and the debugId introduced from expo#25935 ```js // metro.config.js const { getDefaultConfig } = require("expo/metro-config"); function unstable_beforeAssetSerializationPlugin({graph, premodules, debugId}: { graph: ReadOnlyGraph<MixedOutput>; premodules: Module[]; debugId?: string; }) : Module[] { const modifiedPremodules = modifyYourModules(premodules, debugId); return modifiedPremodules; } module.exports = getDefaultConfig(projectRoot, {unstable_beforeAssetSerializationPlugins: [unstable_beforeAssetSerializationPlugin]}); ``` --------- Co-authored-by: Brent Vatne <brentvatne@gmail.com>

EvanBacon added 6 commits December 13, 2023 15:13

Add debugId annotations automatically to multi-asset exports

9298839

use same format as sentry

eb964da

wip -- add support to export:embed

bd68674

wip -- export:embed support

b626083

add tests for legacy

b25f8b4

update

1e8d759

EvanBacon requested review from Simek, amandeepmittal, byCedric and marklawlor as code owners December 14, 2023 05:59

EvanBacon requested review from quinlanj and removed request for Simek and marklawlor December 14, 2023 05:59

expo-bot added the bot: suggestions ExpoBot has some suggestions label Dec 14, 2023

github-actions bot added the bot: fingerprint changed label Dec 14, 2023

EvanBacon and others added 2 commits December 14, 2023 00:01

update docs

8afce68

Update packages/@expo/metro-config/CHANGELOG.md

aec9fa3

Co-authored-by: Expo Bot <34669131+expo-bot@users.noreply.github.com>

expo-bot added bot: passed checks ExpoBot has nothing to complain about and removed bot: suggestions ExpoBot has some suggestions labels Dec 14, 2023

krystofwoldrich reviewed Dec 14, 2023

View reviewed changes

packages/@expo/metro-config/src/serializer/serializeChunks.ts Show resolved Hide resolved

krystofwoldrich reviewed Dec 14, 2023

View reviewed changes

packages/@expo/metro-config/src/serializer/serializeChunks.ts Show resolved Hide resolved

krystofwoldrich reviewed Dec 14, 2023

View reviewed changes

quinlanj approved these changes Dec 14, 2023

View reviewed changes

amandeepmittal reviewed Dec 14, 2023

View reviewed changes

EvanBacon and others added 3 commits December 14, 2023 20:15

Apply suggestions from code review

742bfb1

Co-authored-by: Aman Mittal <amandeepmittal@live.com>

Merge branch 'main' into @evanbacon/metro-config/debug-id-support

0740fe5

updated maps

e32fad3

github-actions bot added bot: fingerprint compatible and removed bot: fingerprint changed labels Dec 15, 2023

update

61dfe7c

EvanBacon merged commit 50a4797 into main Dec 15, 2023
20 checks passed

EvanBacon deleted the @evanbacon/metro-config/debug-id-support branch December 15, 2023 03:20

brentvatne added the published Changes from the PR have been published to npm label Dec 15, 2023

quinlanj mentioned this pull request Dec 16, 2023

[metro-config] experimental pre-serialization plugins #25859

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(metro-config): add debug id support #25935

feat(metro-config): add debug id support #25935

EvanBacon commented Dec 14, 2023

krystofwoldrich Dec 14, 2023

krystofwoldrich Dec 14, 2023

krystofwoldrich Dec 14, 2023

EvanBacon Dec 15, 2023

krystofwoldrich Dec 14, 2023

krystofwoldrich Dec 14, 2023

EvanBacon Dec 15, 2023


		The `debugId` is a deterministic hash of the bundle's contents without the external bundle splitting references. This is effectively the same value used to create a chunks filename, but formatted as a UUID, e.g. `431b98e2-c997-4975-a3d9-2987710abd44`.

		`@expo/metro-config` injects `debugId` during `npx expo export` and `npx expo export:embed`. Any additional optimization steps in `npx expo export:embed` like Hermes bytecode generation will need to have the `debugId` injected manually.

		// NOTE: Sentry does this, but bun does not.
		// sourceMapObject.debug_id = debugId;

feat(metro-config): add debug id support #25935

feat(metro-config): add debug id support #25935

Conversation

EvanBacon commented Dec 14, 2023

Why

How

Test Plan

krystofwoldrich Dec 14, 2023

Choose a reason for hiding this comment

krystofwoldrich Dec 14, 2023

Choose a reason for hiding this comment

krystofwoldrich Dec 14, 2023

Choose a reason for hiding this comment

EvanBacon Dec 15, 2023

Choose a reason for hiding this comment

krystofwoldrich Dec 14, 2023

Choose a reason for hiding this comment

krystofwoldrich Dec 14, 2023

Choose a reason for hiding this comment

EvanBacon Dec 15, 2023

Choose a reason for hiding this comment