Linker compliance tests failing with source map errors

[devversion]

Which @angular/* package(s) are the source of the bug?

compiler-cli

Is this a regression?

Yes

Description

Our linker compliance tests are failing with many hundreds of unmet expectations due to broken source mappings being generated. This started happening as we updated Babel via #49914.

Aside from other regressions in compliance output due to the tests not running, the real mapping errors surface a real issue where it's not yet quite clear whether we need to adjust or Babel introduced a bug.

I've investigated this deeply and digged into internals of Babel and have a good understanding of the source map generation when translating AST to text. Let's have a look at one failure.

i0.elementStart(<..>) // should map to `<div>`

Right now this is broken and the mapping points to a totally unrelated source span. At first glance, this seems magical and it's unclear why this is happening. Debugging the linker Babel ast factory and host, it turns out that we are actually properly trying to set the source map information for e.g. CallExpressions to elementStart- still with the right offsets.

An initial assumption was that we somehow regressed/broke our source map flattening in ngtsc. Something that is non-trivial to understand very easy to break. I've tried to look directly at the intermediary sources, instead of relying on the flattened source map- and the incorrect mappings were still visible. Hence the flattening is not the culprit.

Going deeper, with more reversing of Babel's generator/traversal- I've assumed that somehow the printer is not picking up our source map information. e.g. node.loc may no longer be correct. Turns out this is still the valid field- so our logic is correct. Going further, I noticed that the traversal and state of "current source span" (it's a shared state in the AST printer in Babel) is unexpectedly reset in node traversal. e.g.

CallExpression -- // <div>
     -> MemberExpression
            -> i0 // some other span
            -> elementStart

Babel basically sets the span for the call expression, but then when the call expression is printed, it goes deeper into the member expression -> then the core import. The core import then causes another span to become the "current source span" but never is reset to the parent span when the we go up the tree again (in-order printing). This then results in the mapping to be incorrect. Everything after i0 ends up following the source span of the core import and the actual CallExpression source span is ignored.

This started happening with babel/babel@4f4b450#diff-192a3f85cae6c8a5d19e16b083d4c421b517650b00a0e8ae10cf80cb1ce7d941R590. Previously the source spans were always reset to the previous ones, when e.g. a child node completed printing.

1. visit CallExpression 
2. set span to `<div>`
3. visit MemberExpression
4. visit core import
5. set span to X
6. print core import
7. set span back to previous `<div>`
8. visit & print rest of member expressions)
9. visit & print arguments (all using the still active `<div>` mapping)

Possible fixes:

• Consider working with Babel to see if the changes actually make sense. It seems odd that a child node AST location would affect the source mappings from a higher-level source mapping that spans over the rest of children. The granularity introduced seems acceptable- but also not sure about size here. In either case the locations seem wrong.

• Ensure the import does not have a span. I determined the import always has a span because we are re-using the original AST node from the ngImport field of partial declarations. We cannot should likely not do this in the translator as that one is agnostic to e.g. Babel/TS AST.

• Keep the source span for the import, but instead make sure that the followed printed text of the call expression (i0.elementStart(xxx)) has the proper span to the HTML source

Please provide a link to a minimal reproduction of the bug

Here is a reproduction showing the incorrect source mappings:

• Click here
• Failures: https://gist.github.com/devversion/993e2c0eee657629185dee91365bd7a2
• PR showing failures: test: fix and re-enable partial compliance linking tests #51620
• Possible fixed source map, by cloning the import expression without location

Please provide the exception or error you saw

No response

Please provide the environment you discovered this bug in (run ng version)

No response

Anything else?

No response

[devversion]

Submitted babel/babel#15930 on the Babel side to see if this is the expected behavior on their side. Still seems wrong to me at first glance, but unclear if it will be a wontfix.