Custom parser and visitor API

[devongovett]

This is an experiment that adds support for custom (non-standard) at rules to be parsed via the Rust API, allowing something like Tailwind to be implemented. It's accomplished by passing a custom AtRuleParser trait implementation to the ParserOptions struct. During parsing, the methods of this trait are called if no standard at rule is seen. The CssRule type has been extended to support a Custom variant, which wraps the AtRuleParser::AtRule type. This allows custom structs provided by the user to be stored as part of the stylesheet.

An example is included that implements simple @tailwind and @apply support via this API. It can be run by cloning the branch and running cargo run --example custom_at_rule test.css, where test.css might look something like this:

@tailwind base;
@tailwind components;
@tailwind utilities;

.a {
  @apply bg-blue-400 text-red-400;
}

output:

TAILWIND BASE HERE

TAILWIND COMPONENTS HERE

TAILWIND UTILITIES HERE

.a {
  background-color: #00f; color: red; 
}

We'd need to expose more things if we wanted to implement it for real, e.g. the printer for better formatting/sourcemaps and maybe a way to hook into the transform/minify pass. Mainly wanted to get feedback to see if this is a useful direction before continuing too far. Tailwind also has custom functions like theme, and I'm not sure how we'd handle those yet (probably by exposing the PropertyHandler trait for unparsed properties).

It's worth noting that this approach is a bit different from how something like PostCSS works. PostCSS would accept any unknown at rule and try to guess what it can contain (does it have a block? does the block accept declarations, rules, or something else? etc.), whereas Parcel CSS works more like how browsers parse CSS where only known rules are supported which control how the rule is parsed. This results in much better performance, better error handling, and more predictable results. Hopefully it is flexible enough. If not, I suppose we could try to implement that but I'm not sure it's possible without potentially unbounded lookahead in the parser...

[joelmoss]

Oooo yes please! 👌

Sounds like this could be used to build in some sort of @apply or @mixin functionality.

[joelmoss]

@devongovett Any plans to continue with this?

[devongovett]

Sure. Mainly I was waiting for someone to have a need for it.

[joelmoss]

I have a need for it! 🙋🏻‍♂️😃

[tmjoen]

This looks very cool! We have a framework we use that is a bunch of postcss plugins implementing different at-rules with a centralized config file at the heart of it. It would be supercool if we could implement it in parcel-css/lightning-css instead!

[stevenpetryk]

I have a need for something similar to this: custom value parsing. At Discord we (unfortunately) have a bunch of legacy code that uses $variables (from postcss-simple-vars), and it'd help us migrate to Lightning if we could reimplement that plugin without forking 😀

[devongovett]

Having it in the rust api and having plugins are two different things. With this, you'd need to compile your own CLI or node bindings to use it. Plugin support would be much much more difficult.

[joelmoss]

I would imagine plugins being a huge deal, both as far as implementing support for them - and the effort it would take - and the impact they could make. Realistically, the only alternative is PostCSS, but that can be very slow, especially when compared to lightningcss ⚡️

FWIW I'd be happy with compiling my own CLI. My use case is simply a way to implement mixin's, which is kinda similar to Tailwind's @apply. But really is just replacing an at rule with some standard CSS rules.

Come to think of it, this would be the same syntax as CSS modules composes, but would actually mixin the rules from the referenced class, instead of just referencing the class name.

Anyway, any kind of progress on this front would be amazing 😄

[devongovett]

Plugins are much harder with a compiled tool than with a scripting language. The main challenge is that Rust is not ABI-stable. What this means is that the binary interfaces that are generated for functions, structs, enums, etc. may differ between compiler versions (even minor ones). So plugins would have to be compiled using exactly the same Rust compiler version in order to work. There are ways of making this work by exposing a very intentionally designed C API instead, but it would be a huge amount of work to do this given the scope of Lightning CSS's AST. Not to mention maintaining both forward and backward compatibility over time.

Personally, I think PostCSS is a great tool for building custom plugins. You can use PostCSS and Lightning CSS together by passing the output of one to the other. Even just reducing the number of PostCSS plugins you use to just the custom ones, and removing autoprefixer/preset-env/cssnano will improve the performance of your build by quite a bit.

Another way to handle this could be to use PostCSS to codemod your source CSS. For example, you could write a PostCSS plugin to convert the postcss-simple-vars syntax to native css var() syntax, and commit the results to your repo. From then forward, you'd author CSS variables instead, and Lightning CSS and other tools can handle those natively.

Lightning CSS will generally try to stick to standards and not deviate too much into supporting non-standard syntax. We may eventually have plugins, but it is a long way off and will be a lot of work for IMO questionable gain. I'd love to collect some of the top PostCSS plugins that people use, and discuss what alternatives could be used on a case-by-case basis.

That said, I do want Lightning CSS to be able to be used by other higher level tools. For example, it would be cool if Tailwind CSS could be implemented in Rust using our parser. That was the goal of this PR. This inverts the plugin model - rather than injecting things into a pre-compiled CLI or Node module, you'd compile your own CLI/Node module with the customizations you need. That solves all of the hard points of a plugin system, at the expense of ease of use. But for tooling, it might be ok.

[stevenpetryk]

Another way to handle this could be to use PostCSS to codemod your source CSS. For example, you could write a PostCSS plugin to convert the postcss-simple-vars syntax to native css var() syntax, and commit the results to your repo.

Yeah, that's the fallback plan, and certainly better from a "make your code as non-weird as possible" standpoint. I'll also keep in mind what you mentioned about combining Lightning CSS and PostCSS as well, perhaps I'm overestimating the time PostCSS spends parsing.

[devongovett]

This has expanded to include a Visitor API, which will allow custom transforms to be built much more easily. For example, you could implement a custom function like tailwind's theme(), convert all px to rems, prepend something to URLs, convert colors, etc.

The Visit trait is mostly automatically derived for all values within a stylesheet, and the Visitor trait can be used to implement a visitor that visits specific types of values. You must declare what types of values you want to visit with bitflags. This enables us to completely skip visiting entire branches of the AST when they don't contain any relevant values, statically, at compile time, which improves performance. For example, if you declare you only want to visit urls, we don't need to visit any properties that don't contain URLs somewhere in their type (recursively).

The example implements a basic transform for @apply and theme(), as well as a few other test transforms. The visitor covers rules, properties, and most of the common types of values within them, but if more things are needed we can add them.

cc. @adamwathan @RobinMalfait this may be relevant to your interests 😉

[Oriblish]

Please, Do u think you could help me? How mucg would u charge me to do this for me? I definitely need a professional! (I have way too many email addresses) Thank you, Renee Brutcher 6364284242 ***@***.***

…

On Sun, Sep 11, 2022, 11:17 AM Joel Moss ***@***.***> wrote: I would imagine plugins being a huge deal, both as far as implementing support for them - and the effort it would take - and the impact they could make. Realistically, the only alternative is PostCSS, but that can be very slow, especially when compared to lightningcss ⚡️ FWIW I'd be happy with compiling my own CLI. My use case is simply a way to implement mixin's, which is kinda similar to Tailwind's @apply. But really is just replacing an at rule with some standard CSS rules. Come to think of it, this would be the same syntax as CSS modules composes, but would actually mixin the rules from the referenced class, instead of just referencing the class name. Anyway, any kind of progress on this front would be amazing 😄 — Reply to this email directly, view it on GitHub <#94 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AX26CFBBFDJXVJDG6NJET6LV5YAX5ANCNFSM5O4YZ6DA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>