svelte2tsx incremental transformation

[dummdidumm]

Is your feature request related to a problem? Please describe.
svelte2tsx transformation is triggered on every document change / on every key stroke. This may be imperformant. Also, since it relies on the svelte parser, it will not return anything in error states.

Describe the solution you'd like
In case of addition/deletion of a single character, try some kind of update to the transformed code instead. This could improve the performance and also prevent some error states.

Logic could be the following:

• Inside style tags, use the old version and adjust offset afterwards, because style tags will never alter the output (they are blanked out)
• Inside script tags, try to add it at the correct position. Do a complete reparse in case of characters that might need special treatment, like export or $
• Template: If it's a new tag getting added <b.., try to add it, else reparse
• in error state, always reparse

Problems:

• source mapping

Describe alternatives you've considered
Wait on more robust/error forgiving and incremental svelte compiler.

[jasonlyu123]

It might also improve the completion inside mustache 👍

[jasonlyu123]

Maybe svelte2tsx should also return more info like what the svelte compiler does. Maybe the AST of typescript, the compile result of svelte, or which variables got transformed.

[shirakaba]

Just to attest to the importance of this feature, I'll quote the performance issues I've been facing as files grow in size, that I mentioned to @dummdidumm on Discord:

I hadn't brought it up, as it would only be vague moaning, but I might as well ask whether this behaviour sounds familiar.

Essentially, the Svelte Language Server (from the Svelte plugin for VS Code) slows to a crawl if <script> elements are too many lines in length (more likely if the AST gets too complex). This affects both lang="js" and lang="ts" script types, but the performance issues come sooner with lang="ts" files. Svelte is an absolute pleasure to write, particularly with lang="ts", when the <script> element in your .svelte file consists of fewer than 500 lines of text, but anything above around 500 lines, and the Language Server increasingly becomes unresponsive to the point of being unusable.

For example, in a lang="ts" file with 2,200 lines of code, I can mouse-over a symbol and it will just say "Loading..." for a whole minute, never getting information. Similarly, I can write erroneous code and not see red-line appear under it for a minute. And there are no code-completion suggestions, etc. Worst of all, the Svelte Language Server takes over a minute even to restart.

I've actually spent the last two days at work extracting as much TS code as possible out of <script> sections and into standalone .ts files. I thought this would help tame things. It does, but I'm finding it hard to reduce from 2,200 lines down to a number of lines that is responsive.

Of note, it felt to me that JS code (with TSDoc comments) scaled rather better, but the Svelte Language Server does still turn unresponsive with a large number of lines of JS code, too. It feels like TS code runs into performance issues sooner.

@dummdidumm responded:

These are some good insights. My gut says it's partly due to the way we have to transform the code. Right now, on every change to the document, everything is reparsed first by the Svelte compiler [dummdidumm clarified that this, too, was non-incremental], so we can use the AST, and the script content is also parsed by the TS compiler to get its AST, too. My guess is that this simply does not scale because at more than a few hundred lines this just becomes too slow. I opened a ticket way back about this topic on how we could do incremental generation in #253 , but never followed up on it. However, that you cannot see things a whole minute sounds too much for me even given my explanation. Then again, this all happens synchronously, so it might just add up - every compilation maybe takes 2 seconds, and then you type 10 times, triggering 10 recompiles, which in turn makes it 20 seconds unresponsive.

Maybe there also other ways to tackle this, like having all the language server request we get from the client debounced if that's possible. So for example wait 100 miliseconds before following up on a request and see what else comes in during that, so only the newest content is then actually recompiled.

We also discussed this alternative:

Describe alternatives you've considered
Wait on more robust/error forgiving and incremental svelte compiler.

Although the Svelte compiler upstream of svelte2tsx is also non-incremental, we felt that the current Svelte build speeds were acceptable as-is, so incremental compilation support in svelte2tsx would be the bigger win.

[milahu]

In case of addition/deletion of a single character, try some kind of update to the transformed code

incremental compiler

ideally we subscribe to change events directly from the code editor (CRDT, UDP websockets)

then build a pipe

• incremental parser: tree-sitter, lezer-parser
• incremental transformer
• incremental printer

hardest part is the incremental transformer (incremental semantic stage)
also because we want to chain multiple transformers, for example typescript + svelte

im collecting some links in https://github.com/milahu/treepipes

i built an evaluator based on lezer-parser in https://github.com/milahu/nix-eval-js
goal is an incremental evaluator,
but currently there is only loose coupling between parser and evaluator,
no tight coupling of change events ...

what comes closest is figwheel "live coding" https://figwheel.org/docs/reloadable_code.html
but that is based on google closure-compiler ...