Context: #3148
A user was seeing bad behavior because of illegals causing their highlighting to fail... and they considered just turning it off... and I wondered if that isn't the best choice for 99% of users. Illegals are good for two things(?):
- Fail fast while doing auto-detect.
Illegal rules are probably one of the reasons our auto-detect system can be so fast despite literally trying almost 200 languages in a worse case scenario. They have immense value there as they can allow auto-detection to short circuit when a language seems to "make no sense".
- Possible doing some type of error checking or code validation?
I feel like I'm reaching here... but I feel like the only other good use for illegal is some type of code validation? Wanting to know "is this value C++ code or not?"... while it might have SOME use in that situation we also do not aim (at all) to be a 100% parser for complex languages, nor do we have much interest in highlighting incorrect or invalid code. (it's also pretty hard to do that without being a full parser)
IE, we do not think we are a good tool for this use case.
So here is the current end user experience. You say:
"Hey, here is some C++ code, please highlight."
And we say:
"Nope. Don't think so, doesn't look like C++ to me. I'm going to not highlight it at all."
It's very "all or nothing" and and if the problem is actually with our illegal ruleset itself (it often is) then we're failing to highlight valid C++ code for no obvious reason... when if we just ignored illegals entirely we'd probably make at least a passable attempt - perhaps even a good one.
I think for 99% of use cases you'd want us to still make our "best attempt" at highlighting rather than just highlight NOTHING because we get a little confused.
Given that illegal really only/mostly seems to serve the purposes of internal auto-detect I'd actually even consider removing it entirely (not exposing it externally)... but we can save that for another day... right now it seems clear to me that we should reverse the default (and possible rename the option) for v11...
If anyone reads this and has a valid use case or is doing something interested with illegal I'd certainly love it if you weighed in.
CC @highlightjs/core
Context: #3148
A user was seeing bad behavior because of illegals causing their highlighting to fail... and they considered just turning it off... and I wondered if that isn't the best choice for 99% of users. Illegals are good for two things(?):
Illegal rules are probably one of the reasons our auto-detect system can be so fast despite literally trying almost 200 languages in a worse case scenario. They have immense value there as they can allow auto-detection to short circuit when a language seems to "make no sense".
I feel like I'm reaching here... but I feel like the only other good use for illegal is some type of code validation? Wanting to know "is this value C++ code or not?"... while it might have SOME use in that situation we also do not aim (at all) to be a 100% parser for complex languages, nor do we have much interest in highlighting incorrect or invalid code. (it's also pretty hard to do that without being a full parser)
IE, we do not think we are a good tool for this use case.
So here is the current end user experience. You say:
And we say:
It's very "all or nothing" and and if the problem is actually with our illegal ruleset itself (it often is) then we're failing to highlight valid C++ code for no obvious reason... when if we just ignored illegals entirely we'd probably make at least a passable attempt - perhaps even a good one.
I think for 99% of use cases you'd want us to still make our "best attempt" at highlighting rather than just highlight NOTHING because we get a little confused.
Given that illegal really only/mostly seems to serve the purposes of internal auto-detect I'd actually even consider removing it entirely (not exposing it externally)... but we can save that for another day... right now it seems clear to me that we should reverse the default (and possible rename the option) for v11...
If anyone reads this and has a valid use case or is doing something interested with illegal I'd certainly love it if you weighed in.
CC @highlightjs/core