Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shellcheck to CI (and fixup shell scripts to get the check to pass) #4015

Merged
merged 13 commits into from Jan 31, 2023

Conversation

joshuacwnewton
Copy link
Member

@joshuacwnewton joshuacwnewton commented Jan 27, 2023

Description

This PR expands the lint_code.yml workflow file so that it runs shellcheck in addition to flake8, which would help us ensure the quality of scripts like install_sct and batch_processing.sh.

Linked issues

Fixes #3143.

This does two things:
  - Makes it clearer what's being done to people unfamiliar
    with the word "linting"
  - Expands the scope of the workflow to allow shellcheck
@joshuacwnewton joshuacwnewton added the CI category: TravisCI, GitHub Actions, etc. label Jan 27, 2023
@joshuacwnewton joshuacwnewton added this to the 6.0 milestone Jan 27, 2023
@joshuacwnewton joshuacwnewton self-assigned this Jan 27, 2023
@joshuacwnewton
Copy link
Member Author

joshuacwnewton commented Jan 27, 2023

Neat, we have some hits right off the bat:

Notice: ./.ci.sh:25:10: note: Not following: python/etc/profile.d/conda.sh was not specified as input (see shellcheck -x). [SC1091]
Notice: ./install_sct:36:15: note: Double quote to prevent globbing and word splitting. [SC2086]
Notice: ./install_sct:115:3: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:116:3: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:116:6: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:117:5: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:118:8: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:120:5: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:122:5: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:128:3: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:129:3: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:135:7: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:136:7: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
Notice: ./install_sct:137:7: note: Command appears to be unreachable. Check usage (or ignore if invoked indirectly). [SC2317]
- Error: ./install_sct:156:57: error: < is for string comparisons. Use -lt instead. [SC2071]
Notice: ./batch_processing.sh:45:19: note: Use $(...) notation instead of legacy backticks `...`. [SC2006]
- Warning: ./batch_processing.sh:49:32: warning: Prefer [ p ] && [ q ] as [ p -a q ] is not well defined. [SC2166]
Notice: ./batch_processing.sh:55:7: note: Use $(...) notation instead of legacy backticks `...`. [SC2006]
Notice: ./batch_processing.sh:259:5: note: Use $(...) notation instead of legacy backticks `...`. [SC2006]
Notice: ./batch_processing.sh:262:24: note: Use $(...) notation instead of legacy backticks `...`. [SC2006]
Notice: ./batch_processing.sh:263:24: note: Use $(...) notation instead of legacy backticks `...`. [SC2006]
Notice: ./batch_processing.sh:264:27: note: $/${} is unnecessary on arithmetic variables. [SC2004]
Notice: ./batch_processing.sh:264:52: note: $/${} is unnecessary on arithmetic variables. [SC2004]
Notice: ./batch_processing.sh:264:80: note: $/${} is unnecessary on arithmetic variables. [SC2004]

I'll see if I can get the check to pass.

I thought since `install_sct` doesn't have a `.sh` extension, then
the action would miss it. But, looking at the logs, it caught it!
So, `additional_files` causes a redundant check of `install_sct`
@joshuacwnewton joshuacwnewton changed the title Add shellcheck to CI Add shellcheck to CI (and fixup shell scripts to get the check to pass) Jan 27, 2023
From https://www.shellcheck.net/wiki/SC2166:

> Prefer `[ p ] && [ q ]` as `[ p -a q ]` is not well defined.
From https://www.shellcheck.net/wiki/SC2006:

> Use `$(...)` notation instead of legacy backticked `...`.
From https://www.shellcheck.net/wiki/SC2004:

> `$` / `${}` is unnecessary on arithmetic variables.
From https://www.shellcheck.net/wiki/SC2071:

< and >, in both [[ and [ (when escaped) will do a lexicographical
comparison, not a numerical comparison.

This means that [[ 5 > 10 ]] is true because 5 comes after 10
alphabetically. Meanwhile [[ 5 -gt 10 ]] is false because 5 does
not come after 10 numerically.

If you want to compare numbers by value, use the numerical
comparison operators -gt, -ge, -lt and -le.
From https://www.shellcheck.net/wiki/SC2086:

> Double quote to prevent globbing and word splitting.
From https://www.shellcheck.net/wiki/SC2317:

> Command appears to be unreachable. Check usage (or ignore if invoked indirectly).

Specifically:

> ShellCheck may incorrectly believe that code is unreachable
> if it's invoked by variable name or in a trap. In such a case,
> please Ignore the message.

These functions are invoked via traps, so we ignore as instructed.
From https://www.shellcheck.net/wiki/SC1091:

> Not following: python/etc/profile.d/conda.sh was not specified
> as input (see shellcheck -x)

This error occurs because this file is not present when shellcheck
is run. This is intentional, we don't want to run `install_sct`
just to lint the code. So, we intentionally ignore the issue.
@joshuacwnewton
Copy link
Member Author

joshuacwnewton commented Jan 27, 2023

All fixed! No notices, warnings, or errors here. So, we should be able to turn on stricter checks.

@joshuacwnewton joshuacwnewton marked this pull request as ready for review January 27, 2023 23:25
Copy link
Member

@mguaypaq mguaypaq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for enabling this check, and for fixing all the reported problems!

@mguaypaq mguaypaq merged commit 842b042 into master Jan 31, 2023
@mguaypaq mguaypaq deleted the jn/3143-add-shellcheck-to-ci branch January 31, 2023 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI category: TravisCI, GitHub Actions, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Lint shell in CI
2 participants