[cfe] Option to omit details in `Source` objects from `Component.uriToSource`

[mkustermann]

Some backends may require Component.uriToSource for example for translating offsets to line/col numbers, which requires Source.lineStarts but it does not require the actual source.

I've looked at the RAM consumption when compiling a big app with dart2wasm and we're using 400 MB of RAM for storing the Uint8List of the source even though we don't need it (after CFE is done)

It seems also that Source.constantCoverageConstructors is populated even though we don't really need that information (though this is relatively small amount of memory)

/cc @jensjoha (not sure which area- label to apply here)

[biggs0125]

FWIW dart2js minimizes this by dropping the sources when it serializes the component between phases. dart2wasm can probably do this now with its phased compilation approach.

I'm not sure if dart2js does anything to drop the sources when it runs as a single process though.

[mkustermann]

We cannot drop sources entirely because we depend on them to generate source maps. What we can do is to rewrite Component.uriToSource and make new Source objects with 0-length Uint8List but valid line starts (i.e. clearing out the actual source but keep line starts).

Though this seems a little hacky, so maybe the CFE team could provide options to retain source bytes or not, retain coverage information or not (hence this issue)

[biggs0125]

The kernel's BinaryPrinter has several arguments that control this:

sdk/pkg/kernel/lib/binary/ast_to_binary.dart

Line 73 in 85a7836

this.includeSources = true,

Serializing with includeSourceBytes: false will write an empty byte array but leave the offsets intact.

[mkustermann]

But we shouldn't have to go via serialization&deserialization to achieve this. I'd prefer this not being an option when serializing but rather an option on the AST so one can get the same benefit.

Maybe what I'm asking is to get something like Component.dropSourceBytes() - or maybe Source.dropSourceBytes().

If we had that, then there would be no flags needed for the serialization which is then much clearer separation of concerns:

• dropping allows removing things from the AST that aren't needed anymore
• serialize is the process of turning AST (whatever the AST currently contains) into binary form

/cc @johnniwinther for opinions

[johnniwinther]

I think for your scenario the sources could be removed after constant evaluation/modular transformation, since all error reporting should have been performed by then. @jensjoha Do you agree?

[mkustermann]

There's one edge case: If we have --enable-asserts then the backends will generate code for the assertions and that code will throw AssertionError with messages that contain a snippet of the original source.

But in non-assertion mode I believe we can drop this information as soon as we know that no more diagnostics will be generated for that source file.