Rewrite Shell Script (Bash) #434

wbond · 2016-05-26T11:21:39Z

Probably one of the buggiest syntaxes right now is Shell Script (Bash). Some current issues/PRs:

Work was done on improving the current state of things by @jfcherng with https://packagecontrol.io/packages/ShellScriptImproved. He indicated that he didn't think that was maintainable long term, hence this issue.

Issues that need to be addressed with the rewrite:

Update regular expressions to only use new engine
Consume identifiers including - so partial commands/functions aren't highlighted
Create variations for TCSH and ZSH (http://hyperpolyglot.org/unix-shells)
Fix first line detection
Fix detection of function definitions
Detect command/operator names and don't highlight when used as params (e.g. dd if)
Handle $(), $(()) and ```` expansions
Heredocs

There are surely many more.

The text was updated successfully, but these errors were encountered:

michaelblyons · 2016-05-26T14:12:30Z

Wishlist:

$(()) can include other expressions.
```
 foo=$(($(cat bar.txt | wc -l) + 5))
```
Escape sequences within strings recognize the "color" patterns and don't just capture the \e.
```
 echo -e "\e[31;44mRed on Blue\e[0m"
```

jfcherng · 2016-05-26T15:57:25Z

Here's one that I do not fix.

empty=$((echo hello) > output.txt)

It looks like that it executes just like

empty=$(echo hello > output.txt)

so may be not parsed as $((.... But I am not sure.

MattDMo · 2016-05-30T02:15:32Z

It would be great if the new syntax had a separate scope for ALLCAPS_CONSTANTS.

[[:upper:]][[:upper:][:digit:]_]*

should do it IIRC.

Also, don't dump everything into keyword.other.shell, it's rather pointless. It would also be kind of cool if you could match unquoted strings containing / as paths, but that may have too many exceptions to be useful.

jrappen · 2016-08-13T22:11:14Z

looking at #46 @jfcherng linked this test file by @okdana

jfcherng · 2016-08-14T12:00:46Z

There is a syntax test file that has been used for ShellScriptImproved. Feel free to re-use it if it helps.

cjwelborn · 2017-01-17T03:52:34Z

I thought I was going crazy watching ShellScript not highlight my function definitions (function foo { ...). I'm very thankful for ShellScriptImproved and hope to see this Issue get resolved. Thanks everyone 😄.

@jfcherng

Issue links ----------- Intends to close sublimehq#434 Fixes sublimehq#626 Fixes sublimehq#190 Fixes sublimehq#46 Goodies ------- * Proper context sensitive command invocations * Variable expansion as from ShellScriptImproved * Backtick handling as from ShellScriptImproved * No with_prototype contexts * Generic options handling * Builtin-commands know their valid options * Renamed Shell-Unix-Generic to bash, as it's really just bash (and dash) About 1300 syntax tests, tested out in the wild with various bash scripts. Deficiencies ------------ The heredocs handling is still not perfect, but it gets the job done if you don't have a function definition, inline if-block or compound command on the same line as the start of the heredoc token. * It's easy to bring back the embedded language highlighting for particular heredoc tokens, but this is omitted for now. * It's easy to embed the regex language for regex matching in if-blocks at this point. Acknowledgements ---------------- * @jfcherng for starting ShellScriptImproved and the huge amount of syntax tests, * @okdana for creating syntax tests, * @FichteFoll for pointing me to the Ruby syntax for heredoc handling, * @KingKeith, @wbond for their suggestions about scope names.

@jfcherng

Issue links ----------- Intends to close sublimehq#434 Fixes sublimehq#626 Fixes sublimehq#190 Fixes sublimehq#46 Goodies ------- * Proper context sensitive command invocations * Variable expansion as from ShellScriptImproved * Backtick handling as from ShellScriptImproved * No with_prototype contexts * Generic options handling * Builtin-commands know their valid options * Renamed Shell-Unix-Generic to bash, as it's really just bash (and dash) About 1300 syntax tests, tested out in the wild with various bash scripts. Deficiencies ------------ The heredocs handling is still not perfect, but it gets the job done if you don't have a function definition, inline if-block or compound command on the same line as the start of the heredoc token. * It's easy to bring back the embedded language highlighting for particular heredoc tokens, but this is omitted for now. * It's easy to embed the regex language for regex matching in if-blocks at this point. Acknowledgements ---------------- * @jfcherng for starting ShellScriptImproved and the huge amount of syntax tests, * @okdana for creating syntax tests, * @FichteFoll for pointing me to the Ruby syntax for heredoc handling, * @keith-hall, @wbond for their suggestions about scope names.

@jfcherng

Issue links ----------- Intends to close sublimehq#434 Fixes sublimehq#626 Fixes sublimehq#190 Fixes sublimehq#46 Goodies ------- * Proper context sensitive command invocations * Variable expansion as from ShellScriptImproved * Backtick handling as from ShellScriptImproved * No with_prototype contexts * Generic options handling * Builtin-commands know their valid options * Renamed Shell-Unix-Generic to bash, as it's really just bash (and dash) About 1300 syntax tests, tested out in the wild with various bash scripts. Deficiencies ------------ The heredocs handling is still not perfect, but it gets the job done if you don't have a function definition, inline if-block or compound command on the same line as the start of the heredoc token. * It's easy to bring back the embedded language highlighting for particular heredoc tokens, but this is omitted for now. * It's easy to embed the regex language for regex matching in if-blocks at this point. Acknowledgements ---------------- * @jfcherng for starting ShellScriptImproved and the huge amount of syntax tests, * @okdana for creating syntax tests, * @FichteFoll for pointing me to the Ruby syntax for heredoc handling, * @keith-hall, @wbond for their suggestions about scope names.

@jfcherng

Issue links ----------- Intends to close sublimehq#434 Fixes sublimehq#626 Fixes sublimehq#190 Fixes sublimehq#46 Goodies ------- * Proper context sensitive command invocations * Variable expansion as from ShellScriptImproved * Backtick handling as from ShellScriptImproved * No with_prototype contexts * Generic options handling * Builtin-commands know their valid options * Renamed Shell-Unix-Generic to bash, as it's really just bash (and dash) About 1300 syntax tests, tested out in the wild with various bash scripts. Deficiencies ------------ The heredocs handling is still not perfect, but it gets the job done if you don't have a function definition, inline if-block or compound command on the same line as the start of the heredoc token. * It's easy to bring back the embedded language highlighting for particular heredoc tokens, but this is omitted for now. * It's easy to embed the regex language for regex matching in if-blocks at this point. Acknowledgements ---------------- * @jfcherng for starting ShellScriptImproved and the huge amount of syntax tests, * @okdana for creating syntax tests, * @FichteFoll for pointing me to the Ruby syntax for heredoc handling, * @keith-hall, @wbond for their suggestions about scope names.

wbond added the significant label May 26, 2016

dannywarren mentioned this issue May 26, 2016

[ShellScript] Add support for ksh and csh to Shell-Unix-Generic #87

Closed

wbond mentioned this issue Jun 8, 2016

[Bash] Nested variable substitution doesn't highlight correctly #465

Closed

wbond mentioned this issue Jun 17, 2016

Remove Incompatible Regex Patterns #481

Open

17 tasks

jrappen mentioned this issue Aug 13, 2016

[ShellScript] add missing extensions #581

Closed

keith-hall mentioned this issue Mar 27, 2017

Bash highlighting glitches with weird combination of curly braces jskinner/DefaultPackages#7

Closed

keith-hall mentioned this issue Apr 14, 2017

Bash syntax highlighting: keywords match when next to hyphen or slash jskinner/DefaultPackages#24

Closed

keith-hall mentioned this issue Apr 27, 2017

Many issues with bash / shell script highlighting jskinner/DefaultPackages#125

Closed

keith-hall mentioned this issue May 9, 2017

Shell syntax coloring bug with single quote strings sublimehq/sublime_text#1708

Closed

rwols mentioned this issue Jun 26, 2017

[ShellScript] Rewrite #1057

Merged

wbond closed this as completed in #1057 Oct 5, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rewrite Shell Script (Bash) #434

Rewrite Shell Script (Bash) #434

wbond commented May 26, 2016

michaelblyons commented May 26, 2016

jfcherng commented May 26, 2016 •

edited

Loading

MattDMo commented May 30, 2016 •

edited

Loading

jrappen commented Aug 13, 2016

jfcherng commented Aug 14, 2016

cjwelborn commented Jan 17, 2017

Rewrite Shell Script (Bash) #434

Rewrite Shell Script (Bash) #434

Comments

wbond commented May 26, 2016

michaelblyons commented May 26, 2016

jfcherng commented May 26, 2016 • edited Loading

MattDMo commented May 30, 2016 • edited Loading

jrappen commented Aug 13, 2016

jfcherng commented Aug 14, 2016

cjwelborn commented Jan 17, 2017

jfcherng commented May 26, 2016 •

edited

Loading

MattDMo commented May 30, 2016 •

edited

Loading