fix(estree/ast): fix lossy StringLiteral.value

[hi-ogawa]

I attempted fixing this but realizing #9965 (comment) and it might be out of my reach 😓 Please feel free to take over.

In terms of test262 conformance, there was no edge case like #9965 (comment). So, remaining surrogate related mismatch is a following form. This difference is actually not visible in js world since the difference of acorn output is due to how JSON.stringify automatically normalizes string.

@@ -139,7 +139,7 @@
                 "type": "Literal",
                 "start": 408,
                 "end": 416,
-                "value": "\udf06",
+                "value": "\uDF06",
                 "raw": "\"\\uDF06\""
               }
             ],
@@ -299,7 +299,7 @@
                 "type": "Literal",
                 "start": 593,
                 "end": 619,
-                "value": "\ud834𝌆\ud834",
+                "value": "\uD834\uD834\uDF06\uD834",
                 "raw": "\"\\uD834\\uD834\\uDF06\\uD834\""
               }

> console.log(JSON.stringify("\uD834\uD834\uDF06\uD834"))
"\ud834𝌆\ud834"

[hi-ogawa]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• fix(estree/ast): fix lossy StringLiteral.value #9965 👈 (View in Graphite)
• fix(parser): handle invalid surrogate pair as lossy #9964
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Instrumentation Performance Report

Merging #9965 will not alter performance

_{Comparing 03-22-fix_estree_ast_fix_lossy_stringliteral.value_ (92c54aa) with main (eaea5fd)}

Summary

✅ 33 untouched benchmarks

[hi-ogawa]

I thought this would work similar to codegen, but this doesn't seem to work.

For example, when js has single quoted with double quote inside:

'\uD800"'

this ends up as invalid json:

{
  "type": "Literal",
  "start": 0,
  "end": 9,
  "value": "\uD800"",  <-- double quote inside double quote
  "raw": "'\\uD800\"'"
}

[overlookmotel]

You could check what quotes were used in original code via the 1st byte of raw field. If ", then the way you've done it should work. If ', then you'd need to replace " with \".

I actually have a half-done PR doing exactly that. But was having trouble getting conformance to pass. Maybe after #9964 it will. I'll try and finish it off.

[hi-ogawa]

That sounds good. I think there is a weird case like this unnecessary escape '\"', so perhaps it requires traversing manually character by character? Anyways, I'll close this in favor of your fix 🙏
I was also not sure about raw deser, so I was mostly giving up. 😅