CHROMIUM: Merge 'upstream/master' into chromeos-0.7

Merge of upstream through 218deb6

Conflicts:
	src/ShellCheck/Analytics.hs

Manual changes:
        Renumbered tests to avoid conflict with new jq cases.

Changelog:
----------------------------------------------------------------
Aurelio Jargas (1):
      SC1102: Fix typo in error message: substition

Benjamin Gordon (6):
      CHROMIUM: Exclude D/ED from SC2115 in ebuilds
      CHROMIUM: Don't warn about usev splits in ebuilds
      CHROMIUM: Renumber local warnings
      CHROMIUM: Reorganize variable lists
      CHROMIUM: Add stack.yaml.lock to gitignore
      CHROMIUM: Merge 'upstream/master' into chromeos-0.7

Brennan Vincent (1):
      Update README.md

Commit Bot (1):
      Merge "ShellCheck: Change PortageAutoInternalVariables.hs to a map" into chromeos-0.7

Gandalf- (1):
      Issue 1759 mapfile and process substition

Glen Mailer (1):
      Mention the CircleCI shellcheck orb in the README.

Mason Wilde (3):
      CHROMIUM: Update portageInternalVariables in Data.hs
      ShellCheck: Split auto and manual portage var lists
      ShellCheck: Change PortageAutoInternalVariables.hs to a map

Mike Frysinger (1):
      CHROMIUM: add new cros-workon variable

Onno Zweers (1):
      Rephrase: *Shellcheck* can't follow non-constant source

Peter Gromov (1):
      Don't trigger SC2154 (unassigned var) in `-n`/`-z` expressions koalaman#1583

Simon Shine (1):
      Fix whitespace in README.md

Stavros Ntentos (1):
      Autolink https://www.shellcheck.net/

Vidar Holen (36):
      Clarify that SC1090 refers to ShellCheck, not sh
      Warn about extra spaces between ((s in for((;;))
      Merge branch 'patch-1' of https://github.com/onnozweers/shellcheck into onnozweers-patch-1
      Merge branch 'onnozweers-patch-1'
      Merge pull request koalaman#1999 from aureliojargas/patch-1
      Merge pull request koalaman#2002 from stdedos/patch-1
      Treat $x/ or $(x)/ as ./ when finding sourced files (fixes koalaman#1998)
      Use TravisCI workspaces
      Remove trailing whitespace
      Re-enable Windows job
      Warn about non-POSIX case modification expansions (fixes koalaman#1977)
      Improve handling of command prefixes like exec/command (fixes koalaman#2008)
      Handle literal linefeeds in printf format strings (fixes koalaman#2007)
      Handle tilde expansion in pattern matching (fixes koalaman#1769)
      Improve SC1033/SC1034 message
      Warn when shell functions blatantly recurse (fixes koalaman#1994)
      Merge pull request koalaman#2031 from umanwizard/patch-1
      Merge pull request koalaman#2028 from Lin-Buo-Ren/github-issue-1643
      Suppress SC2216 for du --files0-from or --exclude-from (fixes koalaman#1286)
      Merge branch 'patch-1' of https://github.com/glenjamin/shellcheck into glenjamin-patch-1
      Merge branch 'glenjamin-patch-1'
      Merge branch 'issue_1759_mapfile_proc_substition' of https://github.com/Gandalf-/shellcheck into Gandalf--issue_1759_mapfile_proc_substition
      Modernize getting mapfile array name
      Merge branch 'Gandalf--issue_1759_mapfile_proc_substition'
      Merge branch 'supportMinusNZ' of https://github.com/donnerpeter/shellcheck into donnerpeter-supportMinusNZ
      Consider variables in -z/-n tests to be checked
      Merge branch 'donnerpeter-supportMinusNZ'
      Warn when using &/| between test statements
      Upgrade SC2169 (unsupported in dash) from warning to error (fixes koalaman#2013)
      Merge pull request koalaman#2042 from sshine/patch-1
      Parse assignments according to spec (fixes koalaman#2022)
      Suppress SC2035 for echo * and printf * (fixes koalaman#2036)
      Allow specifying ranges in disable directives
      Give each sh/dash compatibility warning its own SC3xxx error code
      Improve compatibility checks
      Update SC2091/SC2092 message and ignore in quotes.

林博仁(Buo-ren, Lin) (1):
      Fix snap distribution unable to process scripts in Unicode(Chinese) (fixes koalaman#1643)

 .compile_binaries                              |   14 +-
 .github/ISSUE_TEMPLATE.md                      |    4 +-
 .gitignore                                     |    1 +
 .prepare_deploy                                |    6 +-
 .travis.yml                                    |   37 +-
 CHANGELOG.md                                   |    9 +
 README.md                                      |    9 +-
 ShellCheck.cabal                               |    1 +
 nextnumber                                     |    2 +-
 shellcheck.1.md                                |    3 +-
 snap/snapcraft.yaml                            |    2 +
 src/ShellCheck/AST.hs                          |    2 +-
 src/ShellCheck/ASTLib.hs                       |  119 ++-
 src/ShellCheck/Analytics.hs                    |  189 +++-
 src/ShellCheck/AnalyzerLib.hs                  |   74 +-
 src/ShellCheck/Checker.hs                      |    8 +-
 src/ShellCheck/Checks/Commands.hs              |   48 +-
 src/ShellCheck/Checks/ShellSupport.hs          |  137 +--
 src/ShellCheck/Data.hs                         |  529 +----------
 src/ShellCheck/Parser.hs                       |  142 ++-
 src/ShellCheck/PortageAutoInternalVariables.hs | 1205 ++++++++++++++++++++++++
 21 files changed, 1770 insertions(+), 771 deletions(-)
 create mode 100644 src/ShellCheck/PortageAutoInternalVariables.hs

BUG=chromium:1131488
TEST=stack test; spot check cros lint

Change-Id: Iad5d2aa67b20bfdfddb0fa30a48865a087ff4695

.compile_binaries

@@ -16,8 +16,8 @@ build_linux() {
   ls -l shellcheck
   ./shellcheck myscript
   for tag in $TAGS
-  do 
-    cp "shellcheck" "deploy/shellcheck-$tag.linux-x86_64"; 
+  do
+    cp "shellcheck" "deploy/shellcheck-$tag.linux-x86_64";
   done
 }
 
@@ -35,17 +35,17 @@ build_armv6hf() {
   # Linux armv6hf static executable
   docker run -v "$PWD:/mnt" koalaman/armv6hf-builder -c 'compile-shellcheck'
   for tag in $TAGS
-  do 
-    cp "shellcheck" "deploy/shellcheck-$tag.linux-armv6hf"; 
+  do
+    cp "shellcheck" "deploy/shellcheck-$tag.linux-armv6hf";
   done
 }
 
 build_windows() {
   # Windows .exe
   docker run -v "$PWD:/appdata" koalaman/winghc cuib
   for tag in $TAGS
-  do 
-    cp "dist/build/ShellCheck/shellcheck.exe" "deploy/shellcheck-$tag.exe"; 
+  do
+    cp "dist/build/ShellCheck/shellcheck.exe" "deploy/shellcheck-$tag.exe";
   done
 }
 
@@ -66,7 +66,7 @@ build_osx() {
   [[ -e "$path" ]]
 
   for tag in $TAGS
-  do 
+  do
     cp "$path" "deploy/shellcheck-$tag.darwin-x86_64";
   done
 }

.github/ISSUE_TEMPLATE.md

@@ -2,10 +2,10 @@
 - Rule Id (if any, e.g. SC1000): 
 - My shellcheck version (`shellcheck --version` or "online"): 
 - [ ] The rule's wiki page does not already cover this (e.g. https://shellcheck.net/wiki/SC2086)
-- [ ] I tried on shellcheck.net and verified that this is still a problem on the latest commit
+- [ ] I tried on https://www.shellcheck.net/ and verified that this is still a problem on the latest commit
 
 #### For new checks and feature suggestions
-- [ ] shellcheck.net (i.e. the latest commit) currently gives no useful warnings about this
+- [ ] https://www.shellcheck.net/ (i.e. the latest commit) currently gives no useful warnings about this
 - [ ] I searched through https://github.com/koalaman/shellcheck/issues and didn't find anything related
 
 

.prepare_deploy

@@ -1,7 +1,7 @@
 #!/bin/bash
 # This script packages up Travis compiled binaries
 set -ex
-shopt -s nullglob
+shopt -s nullglob extglob
 cd deploy
 
 cp ../LICENSE LICENSE.txt
@@ -59,7 +59,11 @@ do
   rm "shellcheck"
 done
 
+rm !(*.xz|*.zip)
+
 for file in ./*
 do
   sha512sum "$file" > "$file.sha512sum"
 done
+
+ls -l

.travis.yml

@@ -7,14 +7,48 @@ services:
 jobs:
   include:
     - stage: Build
-
       # This must weirdly not have a dash, otherwise an empty job is created
       env: BUILD=linux
+      workspaces:
+        create:
+          name: ws-linux
+          paths: deploy
     - env: BUILD=windows
+      workspaces:
+        create:
+          name: ws-windows
+          paths: deploy
     - env: BUILD=armv6hf
+      workspaces:
+        create:
+          name: ws-armv6hf
+          paths: deploy
     - env: BUILD=aarch64
+      workspaces:
+        create:
+          name: ws-aarch64
+          paths: deploy
     - env: BUILD=osx
       os: osx
+      workspaces:
+        create:
+          name: ws-osx
+          paths: deploy
+
+    - stage: Upload Artifacts to GitHub
+      workspaces:
+        use:
+          - ws-osx
+          - ws-linux
+          - ws-armv6hf
+          - ws-aarch64
+          - ws-windows
+      script:
+        - ls -la ${CASHER_DIR}/ || true
+        # Kludge broken TravisCI workspaces
+        - tar -xvf ${CASHER_DIR}/ws-osx-fetch.tgz --strip-components=5
+        - ls -la deploy
+        - ./.github_deploy
 
     - stage: Deploy docker image
       # Deploy only for pushes to master branch, not other branches, not PRs.
@@ -39,7 +73,6 @@ script:
   - ./striptests
   - set -ex; build_"$BUILD"; set +x;
   - ./.prepare_deploy
-  - ./.github_deploy
 
 # This is in global context and runs for every stage that doesn't override it.
 after_failure: |

CHANGELOG.md

@@ -1,13 +1,22 @@
 ## Git
 ### Added
+- `disable` directives can now be a range, e.g. `disable=SC3000-SC4000`
 - SC2259/SC2260: Warn when redirections override pipes
 - SC2261: Warn about multiple competing redirections
 - SC2262/SC2263: Warn about aliases declared and used in the same parsing unit
+- SC2264: Warn about wrapper functions that blatantly recurse
+- SC2265/SC2266: Warn when using & or | with test statements
 
 ### Fixed
 - SC1072/SC1073 now respond to disable annotations, though ignoring parse errors
   is still purely cosmetic and does not allow ShellCheck to continue.
 
+### Changed
+- Assignments are now parsed to spec, without leniency for leading $ or spaces
+- POSIX/dash unsupported feature warnings now have individual SC3xxx codes
+- SC1090: A leading `$x/` or `$(x)/` is now treated as `./` when locating files
+- SC2154: Variables appearing in -z/-n tests are no longer considered unassigned
+
 
 ## v0.7.1 - 2020-04-04
 ### Fixed

README.md

@@ -109,7 +109,8 @@ Services and platforms that have ShellCheck pre-installed and ready to use:
 * [Codacy](https://www.codacy.com/)
 * [Code Climate](https://codeclimate.com/)
 * [Code Factor](https://www.codefactor.io/)
-* [Github](https://github.com/features/actions)(only Linux)
+* [CircleCI](https://circleci.com) via the [ShellCheck Orb](https://circleci.com/orbs/registry/orb/circleci/shellcheck)
+* [Github](https://github.com/features/actions) (only Linux)
 
 Services and platforms with third party plugins:
 
@@ -167,7 +168,7 @@ On FreeBSD:
 
     pkg install hs-ShellCheck
 
-On OS X with homebrew:
+On macOS (OS X) with Homebrew:
 
     brew install shellcheck
 
@@ -220,7 +221,7 @@ Alternatively, you can download pre-compiled binaries for the latest release her
 * [Linux, x86_64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.x86_64.tar.xz) (statically linked)
 * [Linux, armv6hf](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.armv6hf.tar.xz), i.e. Raspberry Pi (statically linked)
 * [Linux, aarch64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.aarch64.tar.xz) aka ARM64 (statically linked)
-* [MacOS, x86_64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.darwin.x86_64.tar.xz)
+* [macOS, x86_64](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.darwin.x86_64.tar.xz)
 * [Windows, x86](https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.zip)
 
 or see the [GitHub Releases](https://github.com/koalaman/shellcheck/releases) for other releases
@@ -264,7 +265,7 @@ This section describes how to build ShellCheck from a source directory. ShellChe
 
 ShellCheck is built and packaged using Cabal. Install the package `cabal-install` from your system's package manager (with e.g. `apt-get`, `brew`, `emerge`, `yum`, or `zypper`).
 
-On MacOS (OS X), you can do a fast install of Cabal using brew, which takes a couple of minutes instead of more than 30 minutes if you try to compile it from source.
+On macOS (OS X), you can do a fast install of Cabal using brew, which takes a couple of minutes instead of more than 30 minutes if you try to compile it from source.
 
     $ brew install cabal-install
 

nextnumber

@@ -6,7 +6,7 @@ then
   exit 1
 fi
 
-for i in 1 2
+for i in 1 2 3
 do
   last=$(grep -hv "^prop" ./**/*.hs | grep -Ewo "${i}[0-9]{3}" | sort -n | tail -n 1)
   echo "Next ${i}xxx: $((last+1))"

shellcheck.1.md

@@ -233,7 +233,8 @@ Valid keys are:
 **disable**
 :   Disables a comma separated list of error codes for the following command.
     The command can be a simple command like `echo foo`, or a compound command
-    like a function definition, subshell block or loop.
+    like a function definition, subshell block or loop. A range can be
+    be specified with a dash, e.g. `disable=SC3000-SC4000` to exclude 3xxx.
 
 **enable**
 :   Enable an optional check by name, as listed with **--list-optional**.

snap/snapcraft.yaml

@@ -31,6 +31,8 @@ apps:
   shellcheck:
     command: usr/bin/shellcheck
     plugs: [home, removable-media]
+    environment:
+      LANG: C.UTF-8
 
 parts:
   shellcheck:

src/ShellCheck/AST.hs

@@ -145,7 +145,7 @@ data InnerToken t =
     deriving (Show, Eq, Functor, Foldable, Traversable)
 
 data Annotation =
-    DisableComment Integer
+    DisableComment Integer Integer -- [from, to)
     | EnableComment String
     | SourceOverride String
     | ShellOverride String