v0.5.0

• The parser is now unbash, a small JavaScript parser, in the extension, the web page, the command-line tool and the Language Server. Before, it was tree-sitter-bash running as WebAssembly. The .vsix is a quarter smaller (703 KB, was 947 KB) and the web page downloads 1.5 MB less (no .wasm files); a large script is parsed about three times faster; nothing is loaded at start-up. Scripts that the old grammar misread are read as bash reads them: cat <<EOF; cmd (the delimiter is EOF, not EOF;), a pipeline followed by a later && line (tree-sitter-bash#347), for i do, <<- with tabs, coproc, [ ] with escaped parentheses ([ \( a = b \) -o c = d ] is grouped), [ ! = a ], and a here string after }. An unfinished script ({ echo, case $x in, echo "abc) is explained with the parts it has, not as a syntax error.
• About 400 more commands are explained, with their frequent options, instead of "not in the dictionary": the ones named in the LPIC-1/2 and LinuC-1/2 objectives (fdisk, parted, mdadm, pvcreate/lvextend, grub-install, rpm, zypper, chage, at, hwclock, nmcli, tcpdump, nmap, rndc, smbclient, iptables options, firewall-cmd, semanage, iostat, sar, virsh, ansible-playbook…), basic ones (man, locate, whatis, cal, w, last, shutdown, reboot, init, traceroute, tree, chroot, tmux, /usr/bin/time…) and the common macOS tools (open, pbcopy, say, caffeinate, diskutil, launchctl, networksetup, mdfind, xattr, codesign, plutil, sips, system_profiler, dscl, lipo…). Options that only some implementations have say which (nc -e is not in OpenBSD netcat, shutdown -t and -F are ignored by systemd, mail -a attaches a file in s-nail but sets a header in bsd-mailx), and options that do nothing are described as such (chronyc -a, lp -c, more -u). Older commands also know many more options (ls, cp, tar, ps, sort, mount, useradd, ssh, curl, rsync, ulimit…) and subcommands (systemctl list-units, dnf history, ip netns…). systemctl and ip explain their global options after the subcommand too (systemctl status -l x); before, only the options before it were known. /usr/bin/time ls is explained as the external time.
• Options written without a dash in the first argument are explained letter by letter: tar cvzf a.tgz dir, ps aux and ps axjf (BSD options, whose letters mean something else than -a -x -u), ar rcs lib.a x.o, jar cvf, dump 0uf and restore rf. Letters that take a value (f, C, o…) use the following words in order (tar cfC a.tar dir). Only a first argument made of known letters is read this way (tar --create, ps 1234 and ps -ef are not); before, it was "Word cvzf". ar, jar, dump and restore are in the dictionary, and tar knows more options (-r, -u, -A, -d, -p, -k, -m, -O, -h, -P, -a, -b, -T, -X).
• A command with redirections is now a redirect token that holds the command and its redirections side by side, as { …; } > f already was: for tr -d '\r' < windows.txt > unix.txt, the description and the worked example are on tr -d '\r', and < windows.txt and > unix.txt are separate tokens next to it. This holds when the redirections are all before or all after the command's words (>f echo x, <<<'1 2' read -r a b); with one in the middle (echo a > f b) the command is still one token. The summary line is unchanged. Before, the redirections were children of the command, and its description covered them.
• A script with Windows line endings (CRLF) is explained as if it had LF, and a warning (crlf, on the last word of the first such line) says that bash reads the CR as part of the last word of each line (fi becomes fi\r), so the script fails. The warning underlines the last word of the line, and # bash-explainer disable-file=crlf anywhere in the file turns it off. Before, the extension said nothing, and the unbash path read fi as a command.
• An alias is used only on the lines read after the alias ran, as in bash: alias e='echo A'; e on one line, e before the alias, and a function or { } body read before it are explained as the command e (with a note), not as the alias; eval e reads the string again, so it can use it. A later alias of the same name replaces the earlier one. Before, an alias applied to the whole file, and the first definition always won.
• set -eou pipefail and set -oeu pipefail are explained as bash reads them: an o anywhere in a bundle takes the next word as the option name (pipefail), and the other letters (u, e) still apply; the same for set +eo pipefail. Before, -eou lost the u and -oeu was explained as -o alone. The "strict mode" note is shown whenever one set command turns on errexit, nounset and pipefail, in any order and form (-eou pipefail, -o pipefail -eu, -e -u -o pipefail, -o errexit -o nounset -o pipefail); before, only -…e…u…o pipefail was matched.
• ${x@Z} and ${#a-b} are explained as a bad substitution, as bash reports them; ${x@Z} made the explanation fail. ${(f)var} is explained as a bad substitution too, as bash reports it (with the shell set to zsh, it is marked as zsh syntax that is not parsed, as before). A bare echo ( is still accepted without an error, where bash reports a syntax error.
• declare a[1] and declare a[@] explain the element or all elements of the array.
• Words that start with a digit in arithmetic (0b11, 1e3, 08) are flagged as numbers bash rejects, and an empty $() in a command name ($()echo hi) is removed, with the unbash parser too. The base-64 digit @ in arithmetic is a limit of unbash: $((64#@)) alone is read as 62, but next to other terms ($((64#@ + 1))) the whole expression becomes one word.
• printf $"Copyright %s Inc." 2024 is explained like the same format in plain double quotes: the breakdown of the format and the example are shown. Before, the format of a $"…" string was not looked into.
• On the right of ==, = and != in [[ ]], any glob or extended glob gets a pattern token: [[ $x == *.txt ]], [[ $x == --?(a)b ]] and [[ $x == -@(!(-*)e*|-edit) ]]. Before, only some patterns had one, and the extglob that did not start the right side had none.
• A label that ends in a blank, such as ' ' in tr -d ' ' or "PV ", no longer gets a " …" at its end; that is only added when the text goes on in a later line.
• A pipeline that starts with printf '\231' | … (a raw byte) is explained without the example, instead of failing with Unsupported: raw byte. Other formats that the example cannot show (\x without digits, %q with control characters…) too.
• A case pattern list that goes on after a backslash (a|b|\ then c)): the later line shows the pattern c. Before, its | was explained as a pipe, the ) as the end of a subshell and the pattern as a word.
• An array element with spaces in its subscript, a=([1 + 1]=x) or declare -A m=([a b]=1), is explained as one element with an index (2) or a key (a b), as bash reads it up to the ]. Before, the tree-sitter path split it into three words ([1, +, 1]=x) and said three elements. [1 + 1] =x, [1 + 1] = x and x[1 + 1]=x are still separate words, as in bash.
• ${a[@]@K} and ${a[@]@k} say that an indexed array gives index and value pairs too, not only an associative one. The Japanese for @E, @A, @K, @a and @k is smoother (it had 「値を…を…する」).
• [ 1+1 -eq 2 ], [ x -eq 2 ] and [ 1.5 -eq 1 ] say that [ ] reads only integers for -eq and the like, so the operand is an error (integer expression expected), and [[ ]] would calculate it. [[ 1+1 -eq 2 ]] says that it is read as arithmetic (2). Before, they said "numbers are equal" only.
• An array element whose key is quoted, such as ["a b"]=1, [']']=3 or ['q r']=z in h=( … ), is explained as an element with a key and a value, like [c]=2; the key is shown without its quotes (a b). Before, it was a "concatenation" of [, the string and ]=1. The closing ] is the first one outside quotes, expansions and nested [ ], as in bash, so ['a=b']=c and [a\]]=c work too.
• ${!prefix@} and ${!prefix*} are explained as the list of the names of the variables that start with the prefix (@: separate words, *: one word inside double quotes), with the @ and * described as such. Before, they were an "indirection" through a variable named like the prefix, the @ was called a transform, and ${!zq_*} made the explanation fail. ${!x@Q} says indirection first, then the transform.
• = and == in [ ] and test are explained as comparing plain strings ([ a = a* ] is false). The note that the right side is a glob pattern is shown in [[ ]] only; before, [ ] had it too.
• Extended globs (+(a), @(a|x), *(b), ?(a), !(a)) in ${s#…}, ${s//…/…} and the like are explained as extended globs that need shopt -s extglob. Before, they said "contains expansions, so it is decided at run time"; that is said now only when the pattern has a $… or a backquote.
• An unclosed backquote, $( or $(( in the pattern of ${s^^…}, ${s#…}, ${s:-…} and the like, followed by a backslash at the end of the text, no longer makes the unbash path throw. Before, unbash ended such a part past the text, and the explanation failed.
• A backquote substitution inside double quotes is read as bash reads it: the backslash before $, a backquote, \ and " is removed before the command is parsed, so echo "`echo \"x\" y`" shows the string x (the unbash path mapped the positions, but the text still had the stray backslash). \\\\ in a backquote is now \ (one backslash), not \\, in both parsers. Before, the explanation of such words removed only the quotes.
• [[ x =~ "$re" ]], =~ 'a.c' and =~ a"."c say that the quoted part is literal text, not a regex (a variable that holds a regex is written $re, not "$re"). Before, they were explained as a regex, as the unquoted form is.
• ${p//\//_}, ${p/#\/a/X} and the like: the header splits the pattern and the replacement at the first slash that is not escaped, quoted or inside an expansion. Before, it split at the / of \/ (p 内で \ に一致する部分を /_ に置換).
• ${s:1?2:3}, ${s:(1?2:3)} and ${s: a?1:2 :1}: the : that closes a ? belongs to the ternary of the offset, not to the length, as in bash. Before, the offset was 1?2 and the length 3.
• n>&m- and n<&m- are explained as moving descriptor m to n (copy, then close m), and the note on the order of redirections tracks it. Before, 4>&3- was "write both stdout and stderr to the file 3-".
• {name}>file, {name}<file, {name}>&1 and {name}>&- are explained: bash picks a free descriptor from 10 up and stores its number in the variable, or, with -, closes the descriptor the variable holds. Before, {fd} was read as a descriptor named {fd} (unbash path).
• for i in; do …; done and select i in; … say that the empty list means the body never runs; only a loop with no in goes over "$@". select is explained as a menu, not as a for loop.
• ${s~~} and ${s~} are explained as swapping the case (every character, or the first), with an optional pattern. Before, they were read as the plain variable s.
• The examples of ${s,,[HL]}, ${s^^pat}, ${s^pat} and ${s,pat} are computed with the pattern, which limits the characters that change (and, for ^ and ,, tests the first character only). Before, they showed the result without the pattern (hello world, where bash gives hEllO World).
• Bracket expressions in patterns are read as bash does: [[:alpha:]] is one letter (the example listed a]), and []], [!]a], [\^a], [a\]b], [[.a.]] and ranges are handled in case examples, file globs and the descriptions of patterns. Before, the first ] always closed the expression.
• a[1]+=x is explained as appending to the element (b becomes bx), also for associative arrays (m[k]+=x); with declare -i it adds (c[0]+=5). Before, it read "Assign to element".
• Tilde expansion is explained in ordinary arguments: echo ~, ls ~/x, cd ~, ~user/x, ~+ and ~- get a child saying what they become. Quoted "~", x~ and --o=~ get none, as bash does not expand them. Before, only the tilde in an assignment (x=~/y) had a note, and ~user and ~+ were described as the home directory.
• A function is called only after its definition has run, as in bash: echo -- -n; echo(){ …; } explains the first echo as the builtin (with a note that the function comes later), and command echo and builtin echo explain the builtin's arguments, not "argument $1 to the function". A call inside a function body still sees functions defined further down. Before, a function was applied to the whole file.
• >&${fd} and <&$x are explained as duplications of the descriptor the word expands to (a number), with >&word as a file only for other text. Before, >&${COPROC[1]} was "write both to the file". coproc is explained (it was the raw coproc_statement), and COPROC and COPROC_PID are described; the wrong example COPROC=(a b c) is gone.
• Brace expansion is explained for plain words in the unbash path too (a{b,c{d,e}f}g, x{a,b}), and {echo,hi} as a command name runs echo hi. {1..$n} is said to be no range (braces go before variables), and eval is said to expand it later.
• -eq, -lt and the like say how their operands are read: decimal in [ ] (010 is 10, 0x10 is an error), arithmetic in [[ ]] (010 is octal 8, 08 is an error). Before, both said "numbers are equal".
• set -f / set -o noglob turn off the glob explanations and examples after them, until set +f / set +o noglob. set +f is "undo -f: enable globbing again"; before, it read "disable globbing".
• & in the replacement of ${s/b/[&]} is explained as the matched text (bash 5.2, patsub_replacement), and a quoted & (\&, "&") as the character.
• Notes: unquoted expansions say which IFS they split on, and an assignment to IFS says what it does; $y on an array is its first element; $_ after an assignment-only command is empty; $x inside $(( )) is pasted as text, unlike the name x; a redirection after a function body is applied on every call; the $variables of a trap string are expanded when it runs (single quotes) or now (double quotes).
• time ls -a explains ls itself ("List directory contents") under time, as it does after time in a pipeline; before, only its options were shown, and the command was left out.
• time is explained as timing the whole pipeline after it, as bash does: time cat f | sort is one timed pipeline, with an example for the whole pipeline; it was time cat f piped into sort. The commands after time are checked like any other, for dialects (time echo -e x under dash) and pitfalls (time read x, time rm $f); they were left out.
• A redirection after the last command of a pipeline or list belongs to that command, as in bash: in echo A && echo B > f only echo B is redirected, and echo a | cat 2>/dev/null redirects cat; both were explained as redirecting the whole pipeline or list.
• ! a | b is explained as negating the exit status of the whole pipeline, as bash does (! true | false succeeds); it was explained as negating only a.
• What follows a here-document's delimiter on its line is explained as bash reads it: in cat <<EOF | tr a-z A-Z && echo after, tr and echo are commands of the pipeline and the list, with examples and pitfalls; they were taken as a part of the here-document and not explained. Redirections there (cat <<EOF > out.txt) are listed beside the here-document, not under it.
• a | time b is explained as running the program time, as bash does: time is a keyword only at the start of a pipeline (a && time b times b); it was explained as timing b.
• Under zsh, a syntax error nested in a larger one is also reported as "could not be parsed" on its line, as every other error is.
• An unfinished script (if x; then with no fi) no longer lists a keyword of no width ("End of the if statement") for what is missing.
• A bare seq (and seq with more than three numbers) gets no example, as GNU seq reports an error for it; before, explaining the line threw and no explanation was shown. printf with a backslash at the end of the format (printf 'a\\') or before a newline is explained and prints the backslash, as bash does, instead of throwing. printf formats that bash rejects (\x, \u without digits) or that write bytes which are not text (\400, \xe9, \ud800) get no example; before, they showed a wrong output (\x lost its x, \400 printed Ā).
• ${#} is explained as the number of arguments, like $#, and ${##} as the length of $#; before, they were "remove the shortest prefix matching ``" and "remove the prefix # from `##`". `${#@}` and `${#*}` are the number of arguments (not a length in characters). `${##x}`, `${#-x}`, `${#:-x}` and `${#%3}` are `$#` with an operator, as bash reads them; before, they were a syntax error or had a wrong example (`#=hello`). `$?`, `$$`, `$!` and `$-` with an operator are shown with their `$`.
• Numbers in arithmetic are explained with their base and decimal value: 2#1010 is binary 10, 0x1F is hexadecimal 31, 010 is octal 8, 64#@ is 62. Numbers bash rejects (08, 2#2, 1#0, 0b11) are flagged. Before, 0x1F was "variable 0x1F" and the others were just "number". The same rules read the offset and length of ${s:0x1:010}.
• The comma operator in $((x=3,y=x++*2,x+y)) and (( a=1, b=2 )) is explained: the expressions run left to right and the last one gives the value.
• A word with quotes or backslashes shows the string it becomes: h\i is hi, \$HOME and $'\x24HOME' are the text $HOME (nothing is expanded). A command name made of such parts is explained as the command it spells when that is known: $(:)e\c'h'"o" is echo, 'echo' and e\cho too. Before, it was "Run the expanded value as a command", or an unknown command named 'echo'.
• ${*: -1} and ${@: -2} count from the last argument ("the last argument; it is not $-1"); before, they said "from $-1". For arrays, a negative offset counts from one past the highest index, which on a sparse array is not the last element; the length is "1 element" and "1 character" in the singular. The offset and length can be written 0x1 or 010, as bash reads them.
• ${!#} is explained as the last argument (with a checked example, r for p q r); before, "When #=HOME" with the value of $HOME.
• The replacement in ${s/#h/\}} is shown after quote removal (}, with the example }ello world); before, it showed \} and \}ello world. A replacement with & (the match, in bash 5.2) gets no example.
• In a case with ;& or ;;&, the statement says what the three endings do, and the branch after one says it is run without its pattern (;&) or tested again (;;&); a *) after them is no longer "runs when nothing else matched".
• local -n r=$1 is explained as making r another name for the variable named by $1, and a later r=7 says it sets that variable; before, both were plain assignments to r.
• PIPESTATUS and BASH_REMATCH are described as what they are, with examples that run (false | true gives 1 0; [1] is the text of group 1, ${#BASH_REMATCH[@]} is the number of groups + 1); before, the examples were PIPESTATUS=(a b c).
• $[1+2] is marked as the obsolete form of $((…)).
• Constant array indexes are evaluated: ${a[1+1]} is "an arithmetic expression: 2", ${a[010]} is 8, and ${a[-1]} is "the last element".
• z=2 echo $z says $z is expanded before the assignment applies (it is empty if unset), and x=1 y=$x says $x is already set because assignments run left to right.
• Brace expansions that the parser splits into words ({a,b\,c}, {,}, {a..e..2}, project/{src,test}) are one "Brace expansion" with the words they give: {a,b\,c} is a b,c, and {,} is nothing. Before, they were a "Concatenation" of the pieces {, a,b\,c and }.
• A case pattern made of several pieces is one pattern: in [\)]|"("*) the second pattern is "("* (a literal ( and anything after it), and the | tells the patterns apart; before, it was split into "(" and *, as if the line had three patterns.
• The default word in ${u-'x'} is shown after quote removal: unquoted it gives x (and ${u-'}'} gives }), and only in double quotes ("${u-'x'}") do the single quotes stay; before, both examples showed 'x'. The same holds for +, =, :? and in a here-document.
• A variable declared with declare -i (or local -i) is explained as taking arithmetic: n=3+4 is evaluated (7), and n+=1 adds (8) instead of appending text (71); before, it was "append to variable".
• Words after the format of printf are values, as in bash: printf '%d|' 1 -0 prints 1|0|; before, -0 was called an option. Builtins read options only up to the first operand (echo a -n prints a -n, declare x -i is an error for the name -i), and echo knows only -n, -e and -E (echo -x, echo -nx and echo -- a print their words).
• A quoted right side of ==, = and != in [[ ]] is explained as literal text ([[ x == "*" ]] is false); before, it was "a glob pattern". For *"$y"*z the quoted part is literal and the rest is a pattern.
• In declare -A m=([1+1]=v) and m[1+1]=v the key is the string 1+1; in an indexed array i=([1+1]=v) the index is evaluated (2), as ${!m[@]} and ${!i[@]} show in bash (1+1 2). ${m[k]} says "key" for an associative array. Examples for an associative array no longer list 0 1 2 for ${!m[@]} (the keys come in no fixed order); ${#m[@]} is 3 for ([a]=1 [b]=2 [c]=3).
• "$@"x and p"${a[@]}"q are explained as one word per element, with the text before glued to the first one and the text after to the last ([a], [b cx]); before, "the parts are joined into one word".
• {Z..a} is listed with its eight words (the lone backslash is gone after quote removal and stays as an empty word, shown as ''), also on the unbash adapter, and {01..10..3} says that a leading 0 pads every number.
• The arguments of let are explained as arithmetic expressions, one each, left to right: let "y = x << 1" z=y%3 shows the assignments, the shift and the modulo; before, they were a string and a plain word. The value of a constant one is shown (let 0x10+1 is 17).
• IFS=, read -ra p <<< "a,b,,c," shows the line it reads ("a,b,,c,") and what is stored (a, b, an empty field, c), and a note says that a separator that is not whitespace keeps empty fields and a trailing one adds none; before, the example used a made-up line.
• An arithmetic expansion of constants that overflows says the result wraps around ($((1<<63)) is -9223372036854775808: bash integers are 64-bit); the shift operators of constants are computed.