Support Using .mega-linter.yml Config File and Inline Command-Line Parameters Together

[AErmie]

Describe the solution you'd like

MegaLinter would be a more versatile tool to use if the command-line options could be enhanced as follows. When trying to use it as a template pipeline, which is called from another pipeline, if there are no values passed into the parameters (ie. 'DISABLE_DESCRIPTORS'), then it should react/execute as if that environment variable wasn't even passed into it.

This seems to work, since, when not passing any values, it does not prevent the Docker container (and MegaLinter itself) from running. However, even though the actual execution shows the parameter values being properly passed through...

... the MegaLinter does not pick up on these requested refined scopes, and instead, runs all linters.

Additionally, from testing, when using a combination of the .mega-linter.yml config file along with inline command-line parameters (ie. --env 'ENABLE=TERRAFORM'), it seems like the command-line parameters are ignored.

.mega-linter.yml File

PRINT_ALPACA: true
REPORT_OUTPUT_FOLDER: MegaLinterReport
SHOW_ELAPSED_TIME: true

Calling Pipeline

- template: templates/megalinter.yml
  parameters:
    TerraformDirectory: MyTerraformDIR
    ENABLE_DESCRIPTORS: TERRAFORM
    ENABLE_LINTERS: tflint,terraform-fmt

MegaLinter Template Pipeline

parameters:
  TerraformDirectory: '' # This parameter is passed through from the source/originating pipeline
  ENABLE_DESCRIPTORS: # List of enabled descriptors keys
  ENABLE_LINTERS: # List of enabled linters keys
  DISABLE_DESCRIPTORS: # List of disabled descriptors keys
  DISABLE_LINTERS: # List of disabled linters keys

- script: |
        mkdir MegaLinterReport
        docker pull megalinter/megalinter:v5

      docker run \
        --volume $(System.DefaultWorkingDirectory)/terraform/${{ parameters.TerraformDirectory }}:/tmp/lint \
        --volume $(System.DefaultWorkingDirectory)/pipelines/.mega-linter.yml:/tmp/lint/.mega-linter.yml \
        megalinter/megalinter \
        --env 'ENABLE=${{ parameters.ENABLE_DESCRIPTORS }}' \
        --env 'ENABLE_LINTERS=${{ parameters.ENABLE_LINTERS }}' \
        --env 'DISABLE=${{ parameters.DISABLE_DESCRIPTORS }}' \
        --env 'DISABLE_LINTERS=${{ parameters.DISABLE_LINTERS }}' \
        --path /tmp/lint
  displayName: 'Code Scan using MegaLinter'

Describe alternatives you've considered
The alternative would be to create a dedicated .mega-linter.yml file for each customized execution (which negates the dynamic approach of automation).

Additional context
I am testing this using Azure DevOps online (not the on-premises server version), specifically via a Docker image execution, with the following Docker image:

• Image Creation Date: 2022-01-03T22:40:14Z
• Image Revision: 06b1e76
• Image Version: v5

[nvuillam]

That's strange, MegaLinter is supposed to concatenate parameters from ENV and .mega-linter.yml , with priority to ENV vars (if there is a common variable on both side, it overwrites, not append values)

Please can you add LOG_LEVEL: DEBUG in .mega-linter.yml and share the full log ? (truncate what you need if confidentiality is involved)

Note: it's probably not related, but you should add the image version in your docker run command :) megalinter/megalinter :v5

[AErmie]

Hey @nvuillam, I do now see the following, which would indicate that it's finding the config file and the command-line variables being passed through.

[config] /tmp/lint/.mega-linter.yml + Environment variables

However, from some additional testing, it appears that it does not give priority to the ENV vars. I tested this by including --env 'PRINT_ALPACA=false' in the command-line, but then including PRINT_ALPACA: true in the config file. The result experienced was that the Alpaca was printed to the console.

Further, even though it sees/identifies my terraform files (in this example), for some unknown reason it doesn't keep any of them to scan. When I check a previous run, I notice that the filter is different (where it lists file extensions and file names). This time, though, it just shows:

Listing all files in directory [/tmp/lint], then filter with:
<NO FILTERS ARE SHOWN HERE>
`Kept [0] files on [19] found files`

My interpretation is that it's completely ignoring any ENV variables, because when I pass --env 'ENABLE_LINTERS=tflint' in the command-line, it actually ends up running ALL linters, instead of just the one I told it I wanted.

[nvuillam]

Thanks for your analysis, I think I found the origin of the issue, I'll manage that in my next PR :)

[nvuillam]

@AErmie You'll be able to test with megalinter/megalinter:beta version when this job will be completed :)

Please can you provide feedback ? Thanks !

[AErmie]

Hey @nvuillam,
I was able to pull the beta image and run it, but it still seems like it's ignoring any environment variables that are being passed through.

And interestingly, even though it shows the list of linters that should be enabled (ie. ENABLE_LINTERS=['TFLINT', 'terraform-fmt', 'Terrascan']), it doesn't actually run them. In fact, it doesn't run any linters!

Kept [0] files on [19] found files
Kept files before applying linter filters:
- 

+----MATCHING LINTERS-+----------+----------------+------------+
| Descriptor | Linter | Criteria | Matching files | Format/Fix |
+------------+--------+----------+----------------+------------+

MegaLinter flavor is "all", no need to check match with linters
[Post] No commands declared in user configuration