Parse synchronously without fetching @imports?

[papandreou]

Hi!

I'm working on a web build system (https://github.com/One-com/assetgraph) that operates on a graph of assets and relations, and I would like to add support for less assets. For that to work I will need to use the parse tree and model @import statements as relations. Unfortunately it looks like the less parser not only parses these statements, but also tries to fetch the files from disc.

Is there any way the actual parsing phase can be separated from the importing phase?

Thanks :)

Best regards,
Papandreou

[panosru]

@papandreou Hope you are not the George Papandreou ex president of my country (Greece) :P your project seems to be interesting :) I'll keep an eye on it, specially if you add support for less files :)

[papandreou]

@panosru: Don't worry, I'm only an imposter :)

Actually I did add support for compiling .less files a while back using the compileLessToCss transform: assetgraph/assetgraph#40

It works well enough, but if this issue was resolved, I could allow manipulating the less AST during the build process, which would be even more powerful.

[panosru]

great! I'll try it out! :)

[papandreou]

@panosru: Awesome, feel free to open an issue if you run into any troubles or would like to discuss a specific use case. I'm aware that it's currently a little hard to get into due to... Let's say a lack of focus in the documentation.

[panosru]

@papandreou (omg seriously change your nick-name dude... I barely can type it without not breaking my keyboard on my screen!) yes indeed it seems a complex system (based on the presentation) and it should be more documented but I will try to explore it, currently I use bundle-up which is great and i18next-node which is also great, they are both simple to use and get the job done very good :)

[papandreou]

@cloudhead Ping -- is this something you would consider?

At work we've also been toying around with a less plugin for require.js, and it turns out we need this separation to ensure that the stylesheets are injected in the correct order. So that's already two use cases.

[semperos]

I'm compiling LESS using Rhino. I've found that the path of the current file being compiled is undefined when the parser tries to resolve @import statements, so LESS compilation fails with @imports.

I've looked at https://github.com/marceloverdijk/lesscss-java which provides a solution for compiling LESS using Rhino. That code takes the approach of using a Regular Expression to pull out the @import statements in a LESS source file and then provide a workaround to the above. However, this seems brittle and error-prone, given that LESS already exposes its own parser.

I would like to be able to dig into the tree available in the parser's callback to edit things like the path or rootpath of an @import statements rule, but this is impossible given that the LESS parser throws an error as soon as it can't resolve the @import. From an outside perspective, it's surprising that a parser does this resolution.

Would it be possible to move resolution of @import statements outside of the parser itself?

[matthew-dean]

Well..... isn't parsing what _allows_ the @import keyword to be recognized and used? Likewise, you can't finish parsing until those files are retrieved. But.... I guess I get what you're saying that you wouldn't have to necessarily throw a compile error but, hmm..... It could take some significant re-architecting, I think.

[jonschlinkert]

+1 to this, although only a nice-to-have for me. The require.js example in particular is something we've discussed on my team.

[semperos]

I understand your reply; if at parse time you have to be able to resolve
all token definitions, then you have to be able to pull in @imports to
add definitions to your environment/scope.

This case calls for a separate lexer and parser, wherein the lexer limits
itself to identifying parts of the LESS grammar. That would provide a
cleaner separation of concerns in the LESS architecture, and would allow
one to edit the AST before imports are resolved. I glanced briefly through
the code to see where the import behavior was located, so I understand
refactoring the import handling in this way would be non-trivial.

I appreciate you giving this thread such quick attention. I'll do my best
to contribute more than prose when I have some free cycles.

[lukeapage]

The real issue is that rhino doesn't work with the latest changes right - it doesn't surprise me as supporting rhino is a pain and not really integrated very well.

Less already works in the way that you describe - when you parse it calls parser.importer.push - This then calls onto less.importer and has a callback for when we have the file. In async mode that just starts the process of getting the file.. The importer is over-ridden so that the browser and node have different implementations. They resolve files differently and get the files differently. Have a look at browser.js and index.js for the two implementations.
The idea then is that the importer creates a parser and parses the file once it is downloaded.

Once all the parser's have returned it then goes on to the next stage - evaluation. This stage converts from less into css. We want to make this asyncronous to support variables in imports.

The final stage is that you have css and call toCSS on the tree, which then converts to css.

[lukeapage]

moved rhino issue to #1170

[semperos]

@agatronic Thank you for the break down. I'm not well versed in LESS' implementation, so I appreciate the explanation.

While that does show that there's a separation of the language-level parsing and the resolution of imports, the fact still remains that the importer is invoked during parsing, whereas a pure lexer is a deterministic step that generally relies only on a grammar definition and the target source code. Some languages even support incremental parsing, where you don't even need full legal language forms to get back meaningful results from a parsing step.

From what you say, perhaps making the importer a pluggable part of the parsing architecture as exposed to consumers of LESS' API would be one avenue.

All that said, I'm not trying to dictate a particular design for LESS; I just wanted to bring up the fact that the current implementation prevents a certain use-case, which I agree is outside of LESS' main concern.

As for Rhino support, I'm not even trying the less-rhino version at this point, having had no success. Instead, I'm loading up Env.js inside my Rhino run-time before loading the regular release of LESS. This works great, except for the issue raised here, which I've currently worked around by performing transformations on LESS source files before they ever get to the parser in Rhino.