Use custom word boundary matchers #35
Conversation
|
Can you please add some specs to this to ensure that nothing regressed or will regress? |
The regexp engine and the shell grammar don't agree on what comprises
a “word boundary”. This leads to problems where a command or path
ending in a shell keyword would be interpreted as that keyword.
This change replaces `\b` word boundary matchers with lookahead/-behind
matchers on whitespace, line breaks, and command separators (";" and "&").
It also eliminates ad-hoc custom word boundaries, where subsets of [-=/]
were pre- or appended to some word boundaries,
which fixed some similar problems.
|
I've added a bunch of specs for cases that weren't handled before. for…in loops and case blocks seem covered well enough already (except at least one spec is wrong, but that's the subject of #36), which leaves while and until loops, and function declarations. Do you want me to add specs for their "happy paths" as well? |
|
|
||
| expect(tokens[0]).toEqual value: 'iffy', scopes: ['source.shell'] | ||
| for string of strings |
There was a problem hiding this comment.
Sorry for the long delay...but shouldn't this be for string in strings?
There was a problem hiding this comment.
Um, yes, I think it should be. Not sure what I got mixed up there 😅
Can you elaborate on this? |
I was talking about while and until loops, and function declarations. Those don’t have proper tests yet, and I only added tests that ensure that many strange characters are not considered to be word boundaries by the shell. What’s missing are tests that tokenize a whole block, like the one for for…in loops. (Tests for for…in loops without the |
|
Ok, since this is only testing for word boundaries, I think it's fine that we don't add new tests for the other loops. |
The regexp engine and the shell grammar don't agree on what comprises a “word boundary”. This leads to problems where a command or path ending in a shell keyword would be interpreted as that keyword.
This change replaces
\bword boundary matchers with lookahead/-behind matchers on whitespace, line breaks, and command separators (;and&).It also eliminates ad-hoc custom word boundaries, where subsets of
[-=/]were pre- or appended to some word boundaries, which fixed some similar problems.This is basically a cherry-pick of textmate/shellscript.tmbundle@cb6e72e, modified for atom.