Proposal/RFC: Rewrite the astro compiler in rust

[Eveeifyeve]

The Proposal

What if I thought I could make the astro compiler faster but also contributing massively, so today I am opening this as a proposal for what if astro compiler was rewritten in rust that is faster and less in terms of size.

The Motivation, Why it should be considered? and if unaccepted?

My motivation is for me to contribute to and one thing I really know is rust, but the main motivation could be making a big significant change to astro. but I feel like the astro compiler could do a performance upgrade and make the size less, so thats why it should be considered.
and it doesn't matter If this RFC isn't approved, then have a maintained fork of astro compiler then if you do reconsider it it's not a waste of time, plus for the community to use if they wanted to.

Major changes:

It involves a change in the usage of esbuild and switch to lightning css so css would change.
It also involves the change of the compiler team to learn rust.

Thanks, Eveeifyeve (CEO/Founder at TeaClient)

[Eveeifyeve]

I will starting the development in stage 2 in here: https://github.com/Eveeifyeve/compiler/tree/rewrite-in-rust.

[ematipico]

I'm not particularly happy about the motivations, because if there's something where the compiler shines, it's speed. It's something we never had problems with.

Other than that, I already started working on an error resilient parser in Rust

[Eveeifyeve]

Yeah agreed, the compiler is already very fast.

I also note contributions to the existing compiler would also be super valuable! If people are interested in contributing, they could check out the open issues here: https://github.com/withastro/compiler/issues

I mean yeah it's still already fast but it could be better one your locked behind a garbage collector which introduces more memory usage when compiling as well as it speed de-increase which could run into issues when running on low memory devices and two we can use lighting css to add support for modern css syntax like oklab for example that esbuild doesn't have the support for.

[MoustaphaDev]

+1. Not sure how a complete rewrite would provide enough benefits to warrant the effort and possible disruption. The current implementation is already quite fast!

In a benchmark from Bluwy I found, it took only ~4 seconds to transpile all the .astro files in the Astro docs codebase—a non-trivial one—so I’d argue it’s already good enough for most projects. Compile times are rarely the bottleneck in practice; they’re often negligible compared to the broader build pipeline. For example, the page generation step and Rollup generating the module graph take minutes in the Astro docs.

That said, I can see the Astro language server and other tools using the compiler benefiting from reduced memory usage and faster compile times. However, IMO, first trying to further improve the current codebase rather than rewriting the whole thing would be a more pragmatic approach.

Just my two cents 🙂

[delucis]

we can use lighting css to add support for modern css syntax like oklab for example that esbuild doesn't have the support for

The Astro compiler does not handle CSS at all. This is entirely handled by Vite and Lightning CSS is already supported via Vite’s Lightning CSS support.

[Eveeifyeve]

+1. Not sure how a complete rewrite would provide enough benefits to warrant the effort and possible disruption. The current implementation is already quite fast!

In a benchmark from Bluwy I found, it took only ~4 seconds to transpile all the .astro files in the Astro docs codebase—a non-trivial one—so I’d argue it’s already good enough for most projects. Compile times are rarely the bottleneck in practice; they’re often negligible compared to the broader build pipeline. For example, the page generation step and Rollup generating the module graph take minutes in the Astro docs.

That said, I can see the Astro language server and other tools using the compiler benefiting from reduced memory usage and faster compile times. However, IMO, first trying to further improve the current codebase rather than rewriting the whole thing would be a more pragmatic approach.

Just my two cents 🙂

Well definitely the astro language server and other tools could have a benefit of a decreased of memory usage and faster compile times which I have been seeing an issue on developing on a 2 cpu and less ram in a vm and error resilient parser from the current go backend.

[Eveeifyeve]

But also it might be worth spending my time working on rolldown support and page generation optimization stage before starting this step.

[ruizdiazever]

I’m convinced this is the kind of innovation that’ll push the project into hyperspace.

[riderx]

my website has 11k pages with i18n and the build takes a lifetime (11 min on GitHub action). I'm super happy about this test, if you need feedback or help I'm up for it

[riderx]

the website https://github.com/Cap-go/website

[Eveeifyeve]

This is not a astro-compiler issue... It's more of a rolldown performance issue. The Astro compiler is only used in the vscode extension which will have the only benefit to performance.

[Princesseuh]

Hello, I've started on this some time ago now, it works and is available to try out now in Astro 6 Beta (set rustCompiler: true in the experimental section of your config) https://github.com/withastro/compiler-rs

Docs here: https://v6.docs.astro.build/en/reference/experimental-flags/rust-compiler/

I know there's a lot of hype around this, but I'd just like to make it clear that compilation time is very far from being the bottleneck, both in build time and in memory usage. As such, this new compiler only vaguely improves both of these aspects. Especially if you already had a fast CPU / not a lot of .astro files, build time will be more or less the same as the previous compiler.

[riderx]

Can you give us a bit of context what is the difference between both ? Why both can't use the rust ?
Right now it seems the build is long on our side but i dont get why

[Princesseuh]

Sorry, I'm not sure I understand what you mean. There are various factors making builds slow, in Astro most of the time it's the bundling step. In the future we'll migrate to Rolldown and maybe it'll solve all of our problems there, not sure.

[Eveeifyeve]

Sorry, I'm not sure I understand what you mean. There are various factors making builds slow, in Astro most of the time it's the bundling step. In the future we'll migrate to Rolldown and maybe it'll solve all of our problems there, not sure.

What they mean as replacement for the go compiler. The issue with rust compiler is there is no minification for compressing html and only benefits when you have more astro files/slow cpu.

[Princesseuh]

The Rust compiler has support for compressHTML and has for a few days now. It's breaking in a few ways, but will completely replace the Go compiler in Astro 7 down the line