feat!: check for parsing errors in suggestion fixes #101
Conversation
bmish
force-pushed
the
suggestion-parse-errors
branch
from
c4a3930
to
0fc6808
Compare
bmish
force-pushed
the
suggestion-parse-errors
branch
3 times, most recently
from
5243c16
to
8203128
Compare
@bradzacher I agree that suggestions should be allowed to provide incomplete or unsafe fixes, but I don't think suggestions should be allowed to provide syntactically-invalid/unparsable fixes. Producing broken code puts an unnecessary burden on the user to figure out how to un-break it and prevents ESLint and other tools from running on the code in the meantime. There should be a way to turn any desired suggestion into valid code/syntax, even if it's still incomplete or not production ready. I have updated the RFC to mention this. |
bmish
force-pushed
the
suggestion-parse-errors
branch
2 times, most recently
from
c146f9b
to
4d20ba5
Compare
| you can remove this section. | ||
| --> | ||
|
|
||
| 1. Should we check for parsing errors when applying suggestions in core too (not just in rule tester)? Checking for parsing errors in suggestions doesn't have the same performance concerns that it would have for autofixes, as suggestions are applied one-at-a-time vs. autofixes which can be applied in bulk. Showing an exception to end-users about the parsing error could be more clear than simply allowing the suggestion to produce broken code for them, and could encourage the end-user to file a ticket to have the issue fixed with the plugin author. However, this additional assertion may be of limited value, as the vast majority of invalid suggestions would have already been caught by rule tests (as long as the rule actually has tests). |
There was a problem hiding this comment.
I think performance concerns are the same because ESLint doesn't apply suggestions so if we want to warn about parse errors in advance, then we'd have to parse fixed versions of the code at the time the suggestions are created.
There was a problem hiding this comment.
By performance concerns, I was referring to how autofixes can be applied in bulk (e.g. potentially thousands at a time when running on a large codebase) and so any extra validation applied to autofixes could actually add up to a meaningful slowdown. Whereas, even if there's overhead from adding validation to a suggestion, it's still only a single suggestion.
Regardless, I lean toward not bothering with this validation in core. Autofixes and suggestions should both be held to the same validation standards, i.e. we can check them both in rule tester but not in core.
There was a problem hiding this comment.
I still think it's the same problem. If linting produces thousands of suggestions we'd need to additionally parse a thousand times as we don't know which of them will be applied and we don't get any notice when a suggestion is applied.
There was a problem hiding this comment.
Oh, my bad, had a misconception about that. So as of now, only planning to check for parsing errors in tests.
There was a problem hiding this comment.
I agree that it doesn't makes sense to validate in the core.
There was a problem hiding this comment.
Updated to reflect this decision.
mdjermanovic
added
Initial Commenting
| you can remove this section. | ||
| --> | ||
|
|
||
| 1. Should we check for parsing errors when applying suggestions in core too (not just in rule tester)? Checking for parsing errors in suggestions doesn't have the same performance concerns that it would have for autofixes, as suggestions are applied one-at-a-time vs. autofixes which can be applied in bulk. Showing an exception to end-users about the parsing error could be more clear than simply allowing the suggestion to produce broken code for them, and could encourage the end-user to file a ticket to have the issue fixed with the plugin author. However, this additional assertion may be of limited value, as the vast majority of invalid suggestions would have already been caught by rule tests (as long as the rule actually has tests). |
There was a problem hiding this comment.
I agree that it doesn't makes sense to validate in the core.
bmish
force-pushed
the
suggestion-parse-errors
branch
from
4d20ba5
to
f411892
Compare
FWIW, I agree with this. The end result of apply any suggestion should be valid syntax even if the fix itself might be invalid on its own. |
|
Looks like we've got a few days left before we can move to Final Commenting, so placing a bookmark to come back and do that. |
btmills
added
Final Commenting
|
We've passed the minimum Initial Commenting period, so moved to Final Commenting! |
|
Oops, looks like we forgot to merge this. Approved! |
* main: chore: Add Mastodon status post to workflow (eslint#110) feat: ESLint Language Plugins (eslint#99) feat: support for testing invalid rule schemas and runtime exceptions (eslint#103) feat!: check for parsing errors in suggestion fixes (eslint#101)
Summary
Check for parsing errors in suggestion fixes when running rule tests, the same way we do with rule autofix output.
Related Issues