Use --strict to report yaml linting warnings #2295

r-bennett · 2022-01-05T17:18:48Z

Fixes #

Proposed Changes

Use --strict for yamllint so that warnings are also picked up - as per https://yamllint.readthedocs.io/en/stable/configuration.html#errors-and-warnings we currently get a 0 return code if there are only warnings but no errors, which results in the warnings not being reported. Adding --strict brings the behaviour into line with e.g. xmllint which already returns a non-zero code for warnings only:

$ xmllint https://somesite.xml
warning: failed to load external entity "https://somesite.xml"

$ echo $?
1

Readiness Checklist

Author/Contributor

If documentation is needed for this change, has that been included in this pull request

Reviewing Maintainer

Label as breaking if this is a large fundamental change
Label as either automation, bug, documentation, enhancement, infrastructure, or performance

admiralAwkbar · 2022-01-05T17:28:27Z

@r-bennett i like the idea :)

massongit · 2022-01-05T22:52:32Z

https://github.com/github/super-linter/runs/4718836007?check_suite_focus=true
CI Deploy Docker Image is failed due to warnings from yamllint.
You need to fix yaml files.

r-bennett · 2022-01-06T12:04:37Z

I can patch up the warnings on the files - is it possible to run those tests on a branch to check the fixes cover all the cases?

massongit · 2022-01-06T13:25:44Z

If you fork this repository and create pull request in your forked repository, you can run CI Deploy Docker Image.

massongit · 2022-01-06T15:11:18Z

PR which fixes warnings is created: https://github.com/github/super-linter/pull/2301

The latest point release includes https://github.com/github/super-linter/pull/2295 which enables a bunch of checks that break existing use.

adamralph · 2022-01-27T06:19:30Z

The problem with this is that the on: which appears near the top of almost all GitHub Actions workflow files now causes the action to error. E.g. https://github.com/adamralph/bullseye/runs/4959121891?check_suite_focus=true#step:4:1160

nantiferov · 2022-01-27T07:58:56Z

So, from my perspective warnings in yamllint and other linters are supposed to show you that something is not ideal, but not critical enough to fail pipelines.

With this change you basically reduced three level quality gates in yamllint to just 2: it's either pass or not.
What's the point then in warnings if they will fail pipelines?

I would say that this should be rolled back or at least add option to disable it.

ferrarimarco · 2022-01-27T08:11:44Z

It's a yamllint issue: adrienverge/yamllint#430

adamralph · 2022-01-27T08:24:06Z

Thanks @ferrarimarco. The specific issue that I mention is indeed adressed there, but @nantiferov may have a valid point about warnings in general. It seems it would be better left to yamllint to decide what should be a warning and what should be an error.

At a mimimum, maybe this change should be reverted until the yamllint bug is fixed. As it stands, all GitHub Actions workflow files will fail to lint.

ferrarimarco · 2022-01-27T08:31:50Z

Hi @adamralph! I think it's worth opening a new issue to discuss this particular yamllint case, and maybe the general rule. IIRC, there may other linters that we invoke in "strict" mode.

r-bennett · 2022-01-27T09:29:17Z

@nantiferov I'm with you on warnings and errors. I don't use the GitHub action so I'm not familiar with how that works, but from my pov you've three things reporting success/warnings/errors:

yaml-lint which reports success/warnings/errors
super-linter which runs yaml-lint, inspects the output and then reports those success/warnings/errors in its own format
The GitHub action which runs super-linter, inspects the output and reports success/warnings/errors

1 is broken because of adrienverge/yamllint#430
2 was broken for yaml because yaml-lint warnings were being swallowed by super-linter (unless there was also a yaml-lint error) - this PR fixes that issue and now yaml-lint warnings are correctly reported by super-linter as warnings
3 looks like where your problem is. From the sound of it that's always how the GitHub action has worked, but I'd agree it's a bit surprising and probably worth some separate discussion (although this is outside of my usage so I'm not sure I have much more to add myself)

nantiferov · 2022-01-27T09:36:55Z

This one adrienverge/yamllint#430 is not related to my point at all afaik.

Yamllint is working from the beginning in the way that warnings are just shown in output, but not fails pipelines.

I have custom yamllint config that put some questionable stuff like line-length to warnings.
Now I have to either increase it or ignore at all.

r-bennett · 2022-01-27T10:14:56Z

Agreed that adrienverge/yamllint#430 is essentially unrelated, that's just where it fits into the picture and it was mentioned previously.

Yamllint is working from the beginning in the way that warnings are just shown in output, but not fails pipelines.

Disagreed on this - the warnings weren't shown, they were just swallowed. Here's the before and after.

PS H:\> docker run  -e RUN_LOCAL=true  -v "/path/yaml:/tmp/lint" github/super-linter:latest
--------------------------------------------------------------------------------

                              /@@#///////@@/(@//@%/(@.@(       @@
                          @@//////////////////////////////#*  @@@
                        @////@//(///////////@@@@@///@//@/@**//@@(
                      @///////@///////////////@@@@    (           @,
                     @/(&/@////////////////////                     @
                    @////////////////////////@@                      @
                  @%////////(//////////%/////&@            @@       *,@           ______________
             @@@@@/@/#/////(&//////////////////                       .@         /              \
        *@@@@@.    .%///(//@//////////////////&.   .@@,                 @%      / Don't mind me  \
      @@%           .&@&&/@.@//&/////(//////////    @@@@@@@@@         .. &@    / I'm just looking \
    @@%               @@@@@   @&/////////////////#   @/       V  @@/ ,@@@ @   <  for some trash... |
@@@%                   @@@@        .%@@@@//////#@ @   @@         @     .,.     \__________________/
                                          @@@/@(  (@@@@% @/\      %
                                           @@@@(    .     .@@/\   #
                                             @                  %@%

--------------------------------------------------------------------------------
2022-01-27 10:04:41 [INFO]   ---------------------------------------------
2022-01-27 10:04:41 [INFO]   --- GitHub Actions Multi Language Linter ----
2022-01-27 10:04:41 [INFO]    - Image Creation Date:[2022-01-11T18:47:47Z]
2022-01-27 10:04:41 [INFO]    - Image Revision:[7d250fef4c357c4fce25d3ade5a17ba15de68d8d]
2022-01-27 10:04:41 [INFO]    - Image Version:[7d250fef4c357c4fce25d3ade5a17ba15de68d8d]
2022-01-27 10:04:41 [INFO]   ---------------------------------------------
2022-01-27 10:04:41 [INFO]   ---------------------------------------------
2022-01-27 10:04:41 [INFO]   The Super-Linter source code can be found at:
2022-01-27 10:04:41 [INFO]    - https://github.com/github/super-linter
2022-01-27 10:04:41 [INFO]   ---------------------------------------------
2022-01-27 10:04:41 [INFO]   --------------------------------------------
2022-01-27 10:04:41 [INFO]   Gathering GitHub information...
2022-01-27 10:04:41 [INFO]   NOTE: ENV VAR [RUN_LOCAL] has been set to:[true]
2022-01-27 10:04:41 [INFO]   bypassing GitHub Actions variables...
2022-01-27 10:04:41 [INFO]   Linting all files in mapped directory:[/tmp/lint]
2022-01-27 10:04:41 [INFO]   Successfully found:[GITHUB_TOKEN]
2022-01-27 10:04:41 [INFO]   --------------------------------------------
2022-01-27 10:04:41 [INFO]   Gathering user validation information...
2022-01-27 10:04:45 [INFO]   ---------------------------------
2022-01-27 10:04:45 [INFO]   ------ File list to check: ------
2022-01-27 10:04:45 [INFO]   ---------------------------------
2022-01-27 10:04:45 [INFO]   ----------------------------------------------
2022-01-27 10:04:45 [INFO]   Successfully gathered list of files...
2022-01-27 10:04:45 [INFO]
2022-01-27 10:04:45 [INFO]   ----------------------------------------------
2022-01-27 10:04:45 [INFO]   ----------------------------------------------
2022-01-27 10:04:45 [INFO]   Linting [GITLEAKS] files...
2022-01-27 10:04:45 [INFO]   ----------------------------------------------
2022-01-27 10:04:45 [INFO]   ----------------------------------------------
2022-01-27 10:04:46 [INFO]   ---------------------------
2022-01-27 10:04:46 [INFO]   File:[/tmp/lint/test.yaml]
2022-01-27 10:04:46 [INFO]    - File:[test.yaml] was linted with [gitleaks] successfully
2022-01-27 10:04:46 [INFO]
2022-01-27 10:04:46 [INFO]   ----------------------------------------------
2022-01-27 10:04:46 [INFO]   ----------------------------------------------
2022-01-27 10:04:46 [INFO]   Linting [JSCPD] files...
2022-01-27 10:04:46 [INFO]   ----------------------------------------------
2022-01-27 10:04:46 [INFO]   ----------------------------------------------
2022-01-27 10:04:46 [INFO]   ---------------------------
2022-01-27 10:04:46 [INFO]   File:[/tmp/lint/test.yaml]
2022-01-27 10:04:46 [INFO]    - File:[test.yaml] was linted with [jscpd] successfully
2022-01-27 10:04:48 [INFO]
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [INFO]   Linting [YAML] files...
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [INFO]   ---------------------------
2022-01-27 10:04:48 [INFO]   File:[/tmp/lint/test.yaml]
2022-01-27 10:04:48 [ERROR]   Found errors in [yamllint] linter!
2022-01-27 10:04:48 [ERROR]   Error code: 2. Command output:
------
/tmp/lint/test.yaml:1:1: [warning] missing document start "---" (document-start)
------
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [INFO]   The script has completed
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [INFO]   ----------------------------------------------
2022-01-27 10:04:48 [ERROR]   ERRORS FOUND in YAML:[1]
2022-01-27 10:04:48 [FATAL]   Exiting with errors found!
PS H:\> docker run  -e RUN_LOCAL=true  -v "/path/yaml:/tmp/lint" github/super-linter:v4.8.5
--------------------------------------------------------------------------------

                              /@@#///////@@/(@//@%/(@.@(       @@
                          @@//////////////////////////////#*  @@@
                        @////@//(///////////@@@@@///@//@/@**//@@(
                      @///////@///////////////@@@@    (           @,
                     @/(&/@////////////////////                     @
                    @////////////////////////@@                      @
                  @%////////(//////////%/////&@            @@       *,@           ______________
             @@@@@/@/#/////(&//////////////////                       .@         /              \
        *@@@@@.    .%///(//@//////////////////&.   .@@,                 @%      / Don't mind me  \
      @@%           .&@&&/@.@//&/////(//////////    @@@@@@@@@         .. &@    / I'm just looking \
    @@%               @@@@@   @&/////////////////#   @/       V  @@/ ,@@@ @   <  for some trash... |
@@@%                   @@@@        .%@@@@//////#@ @   @@         @     .,.     \__________________/
                                          @@@/@(  (@@@@% @/\      %
                                           @@@@(    .     .@@/\   #
                                             @                  %@%

--------------------------------------------------------------------------------
2022-01-27 10:06:12 [INFO]   ---------------------------------------------
2022-01-27 10:06:12 [INFO]   --- GitHub Actions Multi Language Linter ----
2022-01-27 10:06:12 [INFO]    - Image Creation Date:[2021-12-20T18:26:00Z]
2022-01-27 10:06:12 [INFO]    - Image Revision:[b8641364ca9a79b3cf07f3c4c59a82709cd39094]
2022-01-27 10:06:12 [INFO]    - Image Version:[b8641364ca9a79b3cf07f3c4c59a82709cd39094]
2022-01-27 10:06:12 [INFO]   ---------------------------------------------
2022-01-27 10:06:12 [INFO]   ---------------------------------------------
2022-01-27 10:06:12 [INFO]   The Super-Linter source code can be found at:
2022-01-27 10:06:12 [INFO]    - https://github.com/github/super-linter
2022-01-27 10:06:12 [INFO]   ---------------------------------------------
2022-01-27 10:06:13 [INFO]   --------------------------------------------
2022-01-27 10:06:13 [INFO]   Gathering GitHub information...
2022-01-27 10:06:13 [INFO]   NOTE: ENV VAR [RUN_LOCAL] has been set to:[true]
2022-01-27 10:06:13 [INFO]   bypassing GitHub Actions variables...
2022-01-27 10:06:13 [INFO]   Linting all files in mapped directory:[/tmp/lint]
2022-01-27 10:06:13 [INFO]   Successfully found:[GITHUB_TOKEN]
2022-01-27 10:06:13 [INFO]   --------------------------------------------
2022-01-27 10:06:13 [INFO]   Gathering user validation information...
2022-01-27 10:06:16 [INFO]   ---------------------------------
2022-01-27 10:06:16 [INFO]   ------ File list to check: ------
2022-01-27 10:06:16 [INFO]   ---------------------------------
2022-01-27 10:06:17 [INFO]   ----------------------------------------------
2022-01-27 10:06:17 [INFO]   Successfully gathered list of files...
2022-01-27 10:06:17 [INFO]
2022-01-27 10:06:17 [INFO]   ----------------------------------------------
2022-01-27 10:06:17 [INFO]   ----------------------------------------------
2022-01-27 10:06:17 [INFO]   Linting [GITLEAKS] files...
2022-01-27 10:06:17 [INFO]   ----------------------------------------------
2022-01-27 10:06:17 [INFO]   ----------------------------------------------
2022-01-27 10:06:17 [INFO]   ---------------------------
2022-01-27 10:06:17 [INFO]   File:[/tmp/lint/test.yaml]
2022-01-27 10:06:17 [INFO]    - File:[test.yaml] was linted with [gitleaks] successfully
2022-01-27 10:06:18 [INFO]
2022-01-27 10:06:18 [INFO]   ----------------------------------------------
2022-01-27 10:06:18 [INFO]   ----------------------------------------------
2022-01-27 10:06:18 [INFO]   Linting [JSCPD] files...
2022-01-27 10:06:18 [INFO]   ----------------------------------------------
2022-01-27 10:06:18 [INFO]   ----------------------------------------------
2022-01-27 10:06:18 [INFO]   ---------------------------
2022-01-27 10:06:18 [INFO]   File:[/tmp/lint/test.yaml]
2022-01-27 10:06:18 [INFO]    - File:[test.yaml] was linted with [jscpd] successfully
2022-01-27 10:06:20 [INFO]
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [INFO]   Linting [YAML] files...
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [INFO]   ---------------------------
2022-01-27 10:06:20 [INFO]   File:[/tmp/lint/test.yaml]
2022-01-27 10:06:20 [INFO]    - File:[test.yaml] was linted with [yamllint] successfully
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [INFO]   The script has completed
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [INFO]   ----------------------------------------------
2022-01-27 10:06:20 [NOTICE]   All file(s) linted successfully with no errors detected
2022-01-27 10:06:20 [INFO]   ----------------------------------------------

I do also agree with you that the overall behaviour is a bit surprising and sub-optimal, that's why it's probably worth taking this to a fresh issue to discuss as @ferrarimarco has suggested. Afaik this particular change is just bringing yaml behaviour into line with the current behaviour for all other linters

nantiferov · 2022-01-27T10:41:58Z

Disagreed on this - the warnings weren't shown, they were just swallowed. Here's the before and after.

Ok, probably I saw them when was running locally.

it's probably worth taking this to a fresh issue

Created a new issue for this https://github.com/github/super-linter/issues/2396

The latest point release includes https://github.com/github/super-linter/pull/2295 which enables a bunch of checks that break existing use.

Use --strict to report yaml linting warnings

0f245b7

r-bennett requested review from admiralAwkbar, ferrarimarco, GaboFDC, Hanse00, nemchik and zkoppert as code owners January 5, 2022 17:18

admiralAwkbar merged commit be2b62e into super-linter:main Jan 5, 2022

r-bennett deleted the patch-1 branch January 6, 2022 12:05

creachadair referenced this pull request in tendermint/tendermint Jan 26, 2022

Pin super-linter to a less noisy version.

6ccc1ee

The latest point release includes https://github.com/github/super-linter/pull/2295 which enables a bunch of checks that break existing use.

This was referenced Jan 26, 2022

Pin super-linter to a less noisy version. tendermint/tendermint#7706

Merged

Clean up YAML lint warnings from super-linter tendermint/tendermint#7707

Closed

creachadair referenced this pull request in tendermint/tendermint Jan 26, 2022

Pin super-linter to a less noisy version. (#7706)

09c41dc

The latest point release includes https://github.com/github/super-linter/pull/2295 which enables a bunch of checks that break existing use.

nantiferov mentioned this pull request Jan 27, 2022

PR #2295 introduced breaking change to yamllint workflow #2396

Closed

evan-forbes referenced this pull request in celestiaorg/celestia-core Feb 2, 2022

Pin super-linter to a less noisy version. (#7706)

3367916

The latest point release includes https://github.com/github/super-linter/pull/2295 which enables a bunch of checks that break existing use.

sarahc23 pushed a commit to 23andMe/super-linter that referenced this pull request May 6, 2022

Use --strict to report yaml linting warnings (super-linter#2295)

e8e5cd1

sarahc23 pushed a commit to 23andMe/super-linter that referenced this pull request May 6, 2022

Use --strict to report yaml linting warnings (super-linter#2295)

ce6fdc8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use --strict to report yaml linting warnings #2295

Use --strict to report yaml linting warnings #2295

r-bennett commented Jan 5, 2022

admiralAwkbar commented Jan 5, 2022

massongit commented Jan 5, 2022 •

edited

Loading

r-bennett commented Jan 6, 2022

massongit commented Jan 6, 2022

massongit commented Jan 6, 2022

adamralph commented Jan 27, 2022

nantiferov commented Jan 27, 2022 •

edited

Loading

ferrarimarco commented Jan 27, 2022

adamralph commented Jan 27, 2022

ferrarimarco commented Jan 27, 2022

r-bennett commented Jan 27, 2022

nantiferov commented Jan 27, 2022

r-bennett commented Jan 27, 2022

nantiferov commented Jan 27, 2022

Use --strict to report yaml linting warnings #2295

Use --strict to report yaml linting warnings #2295

Conversation

r-bennett commented Jan 5, 2022

Proposed Changes

Readiness Checklist

Author/Contributor

Reviewing Maintainer

admiralAwkbar commented Jan 5, 2022

massongit commented Jan 5, 2022 • edited Loading

r-bennett commented Jan 6, 2022

massongit commented Jan 6, 2022

massongit commented Jan 6, 2022

adamralph commented Jan 27, 2022

nantiferov commented Jan 27, 2022 • edited Loading

ferrarimarco commented Jan 27, 2022

adamralph commented Jan 27, 2022

ferrarimarco commented Jan 27, 2022

r-bennett commented Jan 27, 2022

nantiferov commented Jan 27, 2022

r-bennett commented Jan 27, 2022

nantiferov commented Jan 27, 2022

massongit commented Jan 5, 2022 •

edited

Loading

nantiferov commented Jan 27, 2022 •

edited

Loading