Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source maps applied to wasm binaries #1051

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
72 changes: 72 additions & 0 deletions Web.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,78 @@ Transcoding failure is detected by `decodeURIComponent`, which may throw
`URIError`. If it does, the WebAssembly module will not validate. This validation
rule is only mandatory for Web embedding.

## Source Maps for WebAssembly files on the web

### Background
[Source maps](https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k)
are a debug-information format used on the web, generated by toolchains that target
JavaScript, either from other languages (e.g. TypeScript, Emscripten) or from
JavaScript itself (e.g. Closure, minifiers). A browser that supports source maps
can display the original source in its developer tools instead of the compiled
JavaScript that the VM actually executes. It can support setting breakpoints
specified as source lines, single-stepping through source lines, and showing
source lines in stack traces; however it cannot support getting or setting
variables from the original source code because the format is not "full" debug
info: it only maps locations from the compiled code back to locations in the
source files (where a generated-code location consists of a line and column
number, and a source location consists of file, line, and column). Generated
files use a specially-formatted comment to specify a URL for a source map that
describes the file.

Source maps can be made to serve the same purpose for WebAssembly binary files,
allowing parity with asm.js. By re-using an existing standard, browsers can
take advantage of their existing source map support without having to make
significant changes to their developer tools.


### Alternatives

An alternative method is to specify location mapping information in some way
other than source maps. For example, it could be encoded directly into the wasm
binary the way function and local names are written in the "name" section. This
is more convenient if the VM itself uses the information (for example to
generate a stack trace). However existing developer tools implementations are
designed around the source map model of passing a URL from the VM to the dev
tools code, which fetches and interprets the information. Therefore it is
epxected that it will be much simpler to use "real" source maps than to design
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sp: expected

another format.

Source maps also have known limitations (e.g. source variables cannot be
described). A complete debug info format will eventually be needed; however the
intention of this specification is to allow for easy integration with existing
developer tools implementations in browsers.


### Spec

With few exceptions, most aspects of the source map
[spec](https://docs.google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k/)
may be applied directly to wasm.
Because wasm is a binary format (and thus does not have lines and columns, or
comments per se), some simple reinterpretation of concepts is needed.

#### Locations

Source maps specify how to map a (zero-based) line and column position in
generated code to a file, line, and column location in the source. For
wasm binary locations, the line number is always 0, and the column number
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use the generated "line number" as the function index instead (or perhaps just the defined function index, skipping imported functions), and the generated "column" as the instruction offset in the function. But I suppose it's best to align w/ what's being discussed here.

Is it worth mentioning this in some rationale?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, On one hand the source map itself is not really meant to be read by humans. But if some tool displays it, then yeah I think it would make sense to display the offsets the way they'd be displayed in tools or browsers, yeah. And yeah it probably is worth mentioning. Maybe I'll do a PR for #990 too and add that to the rationale.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using column number (vs line number) in JS source maps gives a really good savings, to be precise the size of the wasm bytes (the ; used to separate every encoded line and since we will have only one line, smaller number of column separators will be used)

Copy link

@yurydelendik yurydelendik May 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot use line number == 0 due to map encoding specifics. And 'source-map' library throws when line == 0. We need to change that to "the line number is always 1"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. I had thought I used 0 in my original prototype, but it was actually 1.

is interpreted as a byte offset into the wasm binary content (this results
in a more efficient encoding than using the line number instead).
Source locations are interpreted as in the source map spec.

#### Linking generated code to source maps

When the generated code is JavaScript, it includes a specially-formatted line
at the end, which is the URL of the associated source map. For wasm, a custom
section named `"sourceMappingURL"` contains the URL.
As with source maps,
the URL is defined as in [RFC3986](https://tools.ietf.org/html/rfc3986) (e.g.
it must be percent-encoded if necessary) and it may be a data URI. The URL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't sufficiently precise for web APIs; a real spec will need to reference http://url.spec.whatwg.org/

is also resolved according to the source map spec, and may also be specified
with the `SourceMap:` HTTP header.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a spec/reference to how HTTP headers will be associated with wasm module (assuming it's coming from Response)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think this would only work for the new Response-taking compile and instantiate overloads.




## Security

WebAssembly's [security](Security.md) model should depend on the
Expand Down