[Proposal] Run js expressions in markup template through Svelte script preprocessor code

[swyxio]

Is your feature request related to a problem? Please describe.
i would like to use babel/typescript syntax inside of Svelte markup template code.

For example, let's say i have the babel optional chaining enabled:

// rollup.config.js
    svelte({
	  // ...
      preprocess: {
        script: ({ content }) => {
          return require("@babel/core").transform(content, {
            plugins: ["@babel/plugin-proposal-optional-chaining"],
          });
        },
      },
    }),

This lets me use new JS syntax in my script tag:

<script>
  let foo = {
		bar: {
			baz: true
		}
	}
  let sub = foo?.ban?.baz
</script>

<main>
  <h1>Hello {sub}!</h1>
</main>

this is great! however we try to move it down and Svelte complains:

<script>
  let foo = {
		bar: {
			baz: true
		}
	}
</script>

<main>
  <h1>Hello {foo?.ban?.baz}!</h1>
  <!-- uh oh -->
</main>

Error:

[!] (plugin svelte) ParseError: Unexpected token
src/App.svelte
 6: 
 7: <main>
 8:   <h1>Hello {foo?.ban?.baz}!</h1>
                     ^
 9: </main>
ParseError: Unexpected token
    at error (/Users/swyx/Desktop/Work/testbed/trybabelrecipe/svelte-app/node_modules/svelte/src/compiler/utils/error.ts:25:16)
    at Parser$1.error (/Users/swyx/Desktop/Work/testbed/trybabelrecipe/svelte-app/node_modules/svelte/src/compiler/parse/index.ts:96:3)

This is somewhat of a break of the mental model, since Svelte should accept the same kinds of JS inside templates as it does inside script tags. You can imagine other good usecases for this, e.g. {#if data?.foo?.bar}

in order to fix this, i would have to hook into a markup preprocessor, and then parse the template contents to sift out the js expressions, and then transpile it, and then stitch the markup back together. seems like repeat work to what Svelte already does anyway.

Describe the solution you'd like

Svelte should reuse the script preprocessor for in-template js expressions, as well as for <script> tags.

Describe alternatives you've considered

no change

How important is this feature to you?

it's a nice to have. i dont have any proof, but i suspect this might help the TypeScript effort too in case any assertions are needed (this is rare, though; more likely the "new syntax" usecase is more important than the "need to specify types in expressions" usecase)

[pngwn]

I feel like running script preprocessors on every JS expression in the template would slow down compilation considerably but have no data to back that up.

The other option here would be to expose an api to parse the template without passing the JS expressions to acorn yet. At least then you could write a relatively simple markup preprocessor to handle this case.

[Conduitry]

The main technical problem here is that it involves Svelte identifying template expressions written in a language that it cannot parse. The beginnings of template expressions are found via the { but the end is found by telling Acorn 'okay parse as much as you can here as an expression' and then Svelte makes sure there's a } after where Acorn says it parsed until.

As a sort-of solution, the preprocessor could just count opening an closing braces, but this would be thrown off by, say, expressions containing strings containing mismatched braces. A proper solution to this seems more to be to create a new callback that will be called, one at a time, with the start of each expression, and returns the preprocessed code and how many characters it ate, or else throws an exception. I don't have any opinions yet on what this API should look like or what it should be a part of.

[Conduitry]

If completely avoiding duplicating Svelte parser logic in a preprocessor is not a goal, this all could be fairly straightforwardly implemented in userland in a template preprocessor. The template preprocessor runs before the script or style preprocessors, and the initial intention was that it would do something like convert Pug to HTML, but there's nothing stopping it from transforming any part of the component that it wants. The script and style preprocessors are pretty much sugar on top of the template preprocessor, in that they extract the script or style tags and then operate only on that, leaving the rest of the component unchanged.

[swyxio]

indeed we discussed a userland preprocessor solution - but it seemed less than ideal bc it would basically duplicate work that Svelte/Acorn is already doing, and involve wiring up babel twice (once for script, once for template)

as a Svelte user, the mental model for script preprocessor is "ok this thing works on everything inside the <script> tag". but that's not the only place that Javascript appears in a Svelte component.

re: the speed - the syntax transpiles here are going to be very light. will have to benchmark to know for sure ofc but we're definitely not going thru the whole es5 dance here.

I dont know much about the Svelte-Acorn handoff, so this wrinkle with the parsing is interesting. happy to explore that new callback, but i wonder if it should just be the same callback that the preprocessor is

[chopfitzroy]

Wanted to chime in here as no googling found this issue (attributing this to the age of the issue) which lead to me creating the above duplicate.

To distill what I was trying to say in the above issue I think irrespective of the decision made here it would be good to have some formal documentation on this (also mentioned in #3388) to guide new users and help mitigate wasted effort in trying to set this up when it is currently not possible.

[swyxio]

based on what conduitry said, it sounds like some R&D is needed on improving that svelte-acorn parsing. having different systems take care of the { and the } seems a little brittle? idk

[tanhauhau]

just throwing some of my thoughts over here, the current preprocessor is more of a string based preprocessor, doing string replacements before letting svelte compiles it.
this has few implications:

• we dont want to parse the code before parsing the code, therefore, we use regex to quickly match out the <script> and <style> tag. however matching { } brackets and is hard.
• we discard ast and sourcemap from the preprocessor, for example, if using a typescript preprocessor, we use the generated js code and discard away typescript ast, and reparse the js code in svelte. if typescript AST is estree compliant, why spend extra effort on parsing them again?
  • and this could be make using sourcemap from preprocessor easier

so i would like to propose that, maybe instead of preprocessor, we can have a parser plugin.

we can have a default acorn plugin to help parse JS, but also allow user to provide custom JS parser, as long as they are estree compliant.
same idea can go to css too.

[swyxio]

i didnt quite understand the difference between preprocessor or parser plugin, but i also didnt really understand the whole post haha. if this seems like the better idea, where is a good place to start?

also...what does "same idea can go to css too" mean? how would this help?

[tanhauhau]

oh i think i didnt explain it clearly..

i think what i was trying to say is that currently it's hard to make its hard to run js expression in markup through preprocessor as it is run before the svelte parsing.

so maybe an alternative is to provide a way to tap into the parsing, to allow user to provide custom callback to parse JS expression

[Conduitry]

If we did integrate this into the parsing by making it an option to the compiler (rather than handling it in an earlier preprocessing step), we'd need to enforce that those plugins are synchronous. And then we'd have two different ways of doing what would seem to users to be very similar things, and each would have different limitations, which sounds confusing.

[swyxio]

yeah we definitely dont want that ☝️ . ok so are we back to a preprocessor solution? how hard is matching { } brackets? i have no experience with it but want to be sure this assumption is correct.

going back to conduitry's initial thoughts:

The beginnings of template expressions are found via the { but the end is found by telling Acorn 'okay parse as much as you can here as an expression' and then Svelte makes sure there's a } after where Acorn says it parsed until.

i just find this a little weird and wonder if it can be better? would there be side benefits of parsing the { and } in svelte, and then handing off the complete chunks to Acorn?

and i'll be transparent, if it just seems not worth it, im happy to back off/close. just thought like itd be a good idea if it were easy.

[Conduitry]

Without some understanding of the underlying language used within the { }, we can't determine where the expression stops. Consider {"}"}, a weird but perfectly valid expression to write in the template. Without knowing how quoted strings work in JS, Svelte can't parse this correctly. This is why we pass off "}"}blahblah... to Acorn, which says 'okay I can parse "}" as an expression', and then Svelte makes sure there's a } after the part that Acorn parsed, and then continues on its way with blahblah....

Running preprocessors on the <script> tags doesn't pose this same challenge, because these always end when </script> is found, which can be done without any understanding of the language used within the body of the tag.

There probably is still a reasonable way to handle this within Svelte using preprocessors (perhaps by running through the { } bits in series as we parse the template), but I don't have an obvious suggestion for what the API for that would look like yet. Re-parsing the contents of the { } expressions after preprocessing is probably unavoidable, but it might be possible to avoid doing a full parse of the rest of the component during preprocessing (e.g., it might work to just strip out the <script> and <style> tags, and look for {s in the rest of the file, without parsing anything, and call the callback, which returns the compiled-down JS as well as how many characters it consumed).

[Conduitry]

While looking at sveltejs/prettier-plugin-svelte#70 it occurred to me that things like the string <style> happening within the <script> tag is something that Svelte preprocessors also have trouble with. If we've found a <script> then everything up until the </script> is part of that script, no matter what it might look like. And, similarly, we wouldn't want preprocessors to try to do anything with something like {'<script>'}.

What I'm getting at is that I'm starting to look more positively on the idea of going through the input file in order and calling the preprocessors in series. Glossing over some details: In the general situation of a partially preprocessed file, we look for the next occurrence of <script> or <style> or {, whichever happens first. For <script> or <style> we find the next </script> or </style>, pass those contents off to the preprocessor, wait for it to respond, and then pick up again after the closing tag. For { we pass the entire rest of the file to the preprocessor, wait for it to respond, and then pick up again where it's told us to, and ensure that we see optional whitespace followed by a }.

As I was writing this, I realized that one of the detail I glossed over was how to handle {'<script>'} if we weren't tasked with doing anything with template expressions. Do we use our own embedded copy of Acorn to parse it anyway so that we can skip over it and not try to improperly preprocess the <script> (knowing that it's going to be parsed again anyway during compilation)? Do we not worry about trying to nicely handle this unless the user has specified that they want to preprocess template expressions (this seems confusing)?

[swyxio]

that's encouraging! altho i'm not sure i follow what the solution is. are we happy with the current behavior of {'<script>'}? is there a bug we are also trying to fix here?

i feel like we make things a little harder for ourselves with the freewheeling order of script, style, and template. i thought for a while about proposing a fixed order to make life easier for ourselves, but decided against it bc we dont want to break the language (and personally, i enjoy doing script -> template -> style, i know its a little weird).

[swyxio]

Fun thing i just found in the TS 3.9 RC: https://devblogs.microsoft.com/typescript/announcing-typescript-3-9-rc/#breaking-changes

[multics]

I was hoping the same, but it seems that vuejs can't achieve this either.

It is hard! Hope svelte can get rid of this problem, that would be awesome!!

[dummdidumm]

When thinking about a solution, please also take into consideration how to handle source maps. Right now this is already problematic because there may be a script and a module-script tag, each producing its own source maps. Ideally, only one big source map would be returned after transpiling everything, as part of the result of the preprocess function. Not sure how much of that is handled by #5015

[gfreezy]

How about we treat all codes and templates are written in typescript with types or without types? Svelte generates typescript code first, then compiles typescript to js.

[TGlide]

This is really troubling. In the following example, after doing type guards, I'd type cast, but in this case, it is impossible:

<script lang="ts">
    type Field = string | number | boolean;
    type FieldArray = Array<Field>;

    function isFieldArray(value: Field | FieldArray): value is FieldArray {
        return Array.isArray(value);
    }

    type ExampleObject = {
        document: Record<string, Field | FieldArray>;
    };

    const obj: ExampleObject = {
        document: {
            name: 'John',
            age: 30,
            isMarried: true,
            hobbies: ['coding', 'reading', 'gaming']
        }
    };
</script>

{#each Object.keys(obj.document) as k}
    {#if isFieldArray(obj.document[k])}
        <!-- Type Error: -->
        <!--   Argument of type 'Field | FieldArray' is not assignable to parameter of type 'ArrayLike<unknown>'. -->
        <!--   Type 'number' is not assignable to type 'ArrayLike<unknown>' -->
        {#each obj.document[k] as _, index}
            <input bind:value={obj.document[k][index]} />
        {/each}
    {:else}
        <!-- Attribute accepts string | number | boolean as a value. -->
        <!-- Type Error: -->
        <!--   Type 'Field | FieldArray' is not assignable to type 'string | number | boolean'. -->
        <!--   Type 'FieldArray' is not assignable to type 'string | number | boolean'. -->
        <Attribute bind:value={obj.document[k]} />
    {/if}
{/each}

[n8allan]

Disclaimer, I'm not familiar with the Svelte internals. I do have a parser/compiler background though, so allow me to speculate.

As I understand it, Svelte presently uses Acorn to parse javascript in templates. Given that typescript is a superset of javascript, couldn't Svelte use typescript's parsing layer exclusively, and kill two birds with one stone? To address the problem of parsing within {} braces, couldn't a grammar that describes the entire scope of a .svelte file be defined, and the present TS syntax be embedded in the appropriate places within that outer grammar? This might be less modular, but modularity is a secondary concern, and could in principle be addressed by using plugins within the grammar description.

[wvhulle]

For anyone ending up here looking for a way to get optional chaining and other modern syntax to work, adding esbuild (or babel) to your rollup or webpack config is the quickest way to get this to work.
Adding esbuild 0.8 to the Sapper rollup.config.js:

import esbuild from '@cush/rollup-plugin-esbuild';

// Add this after the commonjs plugin
esbuild({
  target: 'es2015',
  exclude: /inject_styles\.js/, // Needed for sapper
  loaders: {
    '.js': 'js',
    '.ts': 'ts',
    '.svelte': 'js',
  },
}),

how would this work for sveltekit in february 2023?

[sandersrd33]

Not sure if anyone is still having this problem but I solve this using reactive statements. This also helps with typing stuff for component parameters and such.

<script lang="ts">
  let foo = {
		bar: {
			baz: true
		}
	}
  let sub: boolean = false
  $: sub = foo?.ban?.baz
</script>

<main>
  <h1>Hello {sub}!</h1>
</main>

[sandersrd33]

This is really troubling. In the following example, after doing type guards, I'd type cast, but in this case, it is impossible:

<script lang="ts">
    type Field = string | number | boolean;
    type FieldArray = Array<Field>;

    function isFieldArray(value: Field | FieldArray): value is FieldArray {
        return Array.isArray(value);
    }

    type ExampleObject = {
        document: Record<string, Field | FieldArray>;
    };

    const obj: ExampleObject = {
        document: {
            name: 'John',
            age: 30,
            isMarried: true,
            hobbies: ['coding', 'reading', 'gaming']
        }
    };
</script>

{#each Object.keys(obj.document) as k}
    {#if isFieldArray(obj.document[k])}
        <!-- Type Error: -->
        <!--   Argument of type 'Field | FieldArray' is not assignable to parameter of type 'ArrayLike<unknown>'. -->
        <!--   Type 'number' is not assignable to type 'ArrayLike<unknown>' -->
        {#each obj.document[k] as _, index}
            <input bind:value={obj.document[k][index]} />
        {/each}
    {:else}
        <!-- Attribute accepts string | number | boolean as a value. -->
        <!-- Type Error: -->
        <!--   Type 'Field | FieldArray' is not assignable to type 'string | number | boolean'. -->
        <!--   Type 'FieldArray' is not assignable to type 'string | number | boolean'. -->
        <Attribute bind:value={obj.document[k]} />
    {/if}
{/each}

It turns out type guards handle the values of the object better when parsed this way for some reason.
This implementation resolved all the errors for me.

{#each Object.values(obj.document) as v}
      {#if isFieldArray(v)}
          {#each v as _, index}
              <input bind:value={v[index]} />
          {/each}
      {:else}
          <Attribute bind:value={v} />
      {/if}
{/each}

[TGlide]

This is really troubling. In the following example, after doing type guards, I'd type cast, but in this case, it is impossible:

<script lang="ts">
    type Field = string | number | boolean;
    type FieldArray = Array<Field>;

    function isFieldArray(value: Field | FieldArray): value is FieldArray {
        return Array.isArray(value);
    }

    type ExampleObject = {
        document: Record<string, Field | FieldArray>;
    };

    const obj: ExampleObject = {
        document: {
            name: 'John',
            age: 30,
            isMarried: true,
            hobbies: ['coding', 'reading', 'gaming']
        }
    };
</script>

{#each Object.keys(obj.document) as k}
    {#if isFieldArray(obj.document[k])}
        <!-- Type Error: -->
        <!--   Argument of type 'Field | FieldArray' is not assignable to parameter of type 'ArrayLike<unknown>'. -->
        <!--   Type 'number' is not assignable to type 'ArrayLike<unknown>' -->
        {#each obj.document[k] as _, index}
            <input bind:value={obj.document[k][index]} />
        {/each}
    {:else}
        <!-- Attribute accepts string | number | boolean as a value. -->
        <!-- Type Error: -->
        <!--   Type 'Field | FieldArray' is not assignable to type 'string | number | boolean'. -->
        <!--   Type 'FieldArray' is not assignable to type 'string | number | boolean'. -->
        <Attribute bind:value={obj.document[k]} />
    {/if}
{/each}

It turns out type guards handle the values of the object better when parsed this way for some reason. This implementation resolved all the errors for me.

{#each Object.values(obj.document) as v}
      {#if isFieldArray(v)}
          {#each v as _, index}
              <input bind:value={v[index]} />
          {/each}
      {:else}
          <Attribute bind:value={v} />
      {/if}
{/each}

Huh, that actually makes sense. Thank you!

[benbucksch]

This problem is made much worse by svelte-check with TypeScript throwing an error on this line:

<div on:keydown={(event) => showErrors(() => onKey(event))} tabindex={0} />

whereas onKey is defined as onKey(event: KeyboardEvent), yet:
svelte-check gives me: Error: Parameter 'event' implicitly has an 'any' type. (ts)

When I add the type, as demanded by svelte-check:

<div on:keydown={(event: KeyboardEvent) => showErrors(() => onKey(event))} tabindex={0} />

then Svelte barfs: Unexpected token and cannot parse it at all, due to this issue here.

(This svelte-check error apparently happens only on generic HTML elements which don't have on:keydown specifically known to Svelte. <svelte:window on:keydown={(event) => showErrors(() => onKey(event))} /> and similar work without type.)

The fact that TypeScript doesn't work in {} expressions in the HTML section, even though my <script lang="ts"> section is TypeScript, is highly surprising for me. But even if I accept that: Then why is svelte-check not aware of that and tries to enforce type checks in these expressions, causing the svelte-check "any type" error above? If TypeScript cannot work in expressions, then surely svelte-check should not try to enforce it?

The only workaround I know is to add another dummy wrapper function in the <script> section, but that is useless code and I don't want to do that everywhere.

Most importantly, there doesn't seem to be any solution for the svelte-check error. I don't know how to make svelte-check happy, because it creates a hard error, which makes our CI pipeline fail, and Svelte has a parse error on the solution that svelte-check demands.

[brunnerh]

There is something else going on with that keydown code, the event is known and should not cause errors (and does not for me either).

You might want to create a minimal example that reproduces the issue.

[benbucksch]

svelte-check throws the error, both locally and on CI (which is a hard block in our CI pipeline, so I cannot land). I'm not the only one, as the Stack Overflow question shows.

A simple function call might work, due to type inference. To reproduce, you need the function parameter as in my example.

[brunnerh]

The editor tooling uses the same libraries as svelte-check (assuming everything is up to date) and I cannot produce any such error, neither in editor nor via svelte-check.

I even tried to recreate the structure of the handler's code and it does nothing.

function onKey(e: KeyboardEvent) {  }
function showErrors(cb: () => any) {  }

<div on:keydown={event => showErrors(() => onKey(event))} tabindex={0}>

As I said, please create a minimal example.
If this actually is an issue, it it something separate from this one.

[ota-meshi]

I found the Acorn parser TypeScript plugin today.
https://github.com/TyrealHu/acorn-typescript

I think using acorn-typescript inside the Svelte parser could be a step forward for Svelte to be able to use TypeScript in the template part.
We can use parseExpressionAt() to parse the end of an expression even if there is TypeScript inside the {...}.

[benmccann]

Thanks for sharing @ota-meshi! We're pretty interested in this approach. If you have made any progress we'd love to see what you're working on. Or we'd be happy to chat about how we might integrate such an approach if you'd like to hop into the #contributing channel on the Svelte Discord and leave us a message

[veselints]

svelte-check really makes things crazy. As we have a TypeScript project, the svelte-check expects all JS in the file to be actually TS. In strict mode it errors out in this case:

<Button
    ...
    clickLogic={(ev) => {
	    doSomething(ev);
    }}
/>

that the Parameter 'ev' implicitly has an 'any' type.

Nevertheless, as this is JS and TS, I cannot specify a type here. As a result, I cannot see any way to make svelte-check happy.

Things get even worse when using enumerations. This:

<div>
	{SampleEnum[enumMemberKey]}
</div>

errors out that the type 'string' cannot be used to index type 'SampleEnum'. Again, this is not the case, so I cannot say something like: SampleEnum[enumMemberKey as SampleEnum].

At the same time we would love to have the svelte-check as part of our CI. Even if the code in the mustaches is not a TS, I find it useful how it works for disciplinary reasons. Therefore, I do not want to entirely disable it in the HTML templates. The perfect solution would be to allow TS in HTML templates of Svelte components. While this is still not the case, my question is: Can we disable svelte-check warnings case by case?

[brunnerh]

• Support @ts-ignore comments in template markup language-tools#1026
• Syntax for ignoring type errors language-tools#1915

[benbucksch]

svelte-check expects all JS in the file to be actually TS
<Button clickLogic={(ev) => { doSomething(ev); }} />
Parameter 'ev' implicitly has an 'any' type.

Exactly that. I reported that 4 months ago above and elsewhere.

Until a proper fix (like this feature here) is in place, could svelte-check at least be configured by default to not check such expressions at all, or (if possible) to not expect TypeScript in them? At least that would avoid that all developers try to "fix" their own code, just to find that this is a bug in svelte-check (or svelte).

[16Integer]

Is there any progress towards this? I love Svelte, but this is bugging me so much, I'm seriously thinking to switch back to Next.js or something else.

[brunnerh]

Svelte 5 will have native support for TypeScript in the template (#9482).