Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Embed source files in source map file #22189

Closed
verhovsky opened this issue Jul 4, 2024 · 6 comments · Fixed by #23741 · May be fixed by #22190
Closed

Embed source files in source map file #22189

verhovsky opened this issue Jul 4, 2024 · 6 comments · Fixed by #23741 · May be fixed by #22190

Comments

@verhovsky
Copy link

verhovsky commented Jul 4, 2024

I want to generate a source map when compiling my C code that includes all the source files in the source map file itself instead of having to serve all the source code files alongside the source map file. This can be done by adding a sourcesContent key to the source map file. Otherwise I would have to upload 93 C files to npm with my JavaScript library.

wasm-sourcemap.py includes support for doing this already with the --sources argument

parser.add_argument('-s', '--sources', action='store_true', help='read and embed source files from file system into source map')

but there's no way to pass that argument from the emcc command

emscripten/tools/building.py

Lines 1107 to 1111 in b3c2567

sourcemap_cmd = [sys.executable, '-E', path_from_root('tools/wasm-sourcemap.py'),
wasm_file,
'--dwarfdump=' + LLVM_DWARFDUMP,
'-o', map_file,
'--basepath=' + base_path]

This could be a -ginline-source-map option or -gsource-map -s INLINE_SOURCE_MAP, which would match TypeScript's --inlineSourceMap. Though I expected this to be the default behavior. Serving C files over HTTP is weird.

My use case is that I have an npm package compiled with Emscripten and my source map requires files from ../../../../../../../opt/homebrew/Cellar/emscripten/3.1.61/libexec/system/lib/ and I have no idea how I could put that path in my npm module tree-sitter/tree-sitter#3381

@sbc100
Copy link
Collaborator

sbc100 commented Jul 6, 2024

I think the normal way that folks deal with this issue it to use path substitution and server the sources form localhost: https://developer.chrome.com/blog/wasm-debugging-2020.

Embedding the all of the source files in the source map seems like it could lead to absolutely enourmous source maps so I'm not sure we would want to make that the default.

@dschuff @pfaffe @RReverser WDYT?

@verhovsky
Copy link
Author

I am compiling a JavaScript library for npm. I want the source map to also work for downstream users when someone develops a website using the library. Users of the library probably won't bother to copy files out of node_modules/ to serve them for their source maps and they definitely won't bother with writing paths out in Chrome settings. They'll just use the library without source maps, so the choice is really between absolutely enormous source maps or no source maps.

As a datapoint, the sourcesContent for me would be 800KB, less than the 1MB wasm file.

$ cat $(jq -r '.sources[]' tree-sitter.wasm.map) | wc -c
847334
$ wc -c tree-sitter.wasm
1055249 tree-sitter.wasm

@sbc100
Copy link
Collaborator

sbc100 commented Jul 8, 2024

(Sorry, the wasm-debugging link I sent above was for DWARF debugging. It probably not relevant to source map debugging)

@pfaffe
Copy link
Collaborator

pfaffe commented Jul 8, 2024

The request makes sense in general, but don't make it the default. As default, it could cause quite the surprise for any user who doesn't want to ship their sources.

@verhovsky
Copy link
Author

verhovsky commented Jul 8, 2024

I meant that putting all the sources in sourcesContent should be the default when using -gsource-map. It shouldn't be surprising that a source map includes the source, at a minimum it already contains line/character numbers which gives you some hint about the source.

@dschuff
Copy link
Member

dschuff commented Aug 8, 2024

I think it's a good idea to support this, but I also think it's best not to have it on by default. Not just for the surprise in possibly exposing the sources, but also that it would make the source map much bigger, and many common use cases (e.g. stack trace symbolization) don't require the sources.

@sbc100 sbc100 closed this as completed in c70b9ff Mar 13, 2025
JoeOsborn pushed a commit to JoeOsborn/emscripten that referenced this issue Mar 14, 2025
Provide source map settings to emcc (resolves emscripten-core#23686, resolves emscripten-core#22189) :
- for embedding the sources content in the source map :
`-gsource-map=inline`.
- for applying path prefix substitution :
`-sSOURCE_MAP_PREFIXES=["<old>=<new>"]`.

Update documentation accordingly. 

Fix source file resolver : 
- Always fall back to the given filepath if prefix not provided or
doesn't match.
- Fix source content loading when no `--load-prefix` is given/needed.
- Fix relative path in "sources" field when`--prefix` is given but
doesn't match the given file.
- Cache filepaths with no/unmatched prefix as well.
- Resolve deterministic prefix when loading source content (related:
emscripten-core#20779).
- Don't emit relative paths for sources with a deterministic prefix.

Improve existing test for wasm-sourcemap.py :
- Parameterize `test_wasm_sourcemap()` and do proper checks according to
the combinations of options given.
- Fix regex for checking the "mappings" field (was checking only the
first character).

Add test for emcc covering the basic use cases where :
- no option is given (users do path substitution as needed via their
client or server configuration).
- different prefixes are provided for source files and emscripten
dependencies.
- source content is embedded in the sourcemap.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants