Documentation Improvement: Adding Detailed Steps for Implementing a Simple Linter and Warnings for Actions to Avoid

[ThomasSanson]

I would like to propose an improvement to the existing documentation by adding detailed steps for implementing a simple linter and important warnings for actions to avoid.

I suggest adding detailed steps for implementing an existing simple linter, explaining each step in detail so that users can easily understand and follow the instructions. Additionally, I propose implementing clear warnings for actions to avoid, such as touching generated files created by the linter.

I am willing to contribute to writing these instructions and assisting with the implementation of these warnings.

Thank you in advance for considering my proposal.

[bdovaz]

Having this reference: https://megalinter.io/beta/contributing/#add-a-new-linter

Points that come to mind:

• Clearly indicate where to create the descriptor.
• Explain the automatic testing system that generates a class for each linter with several methods (success, fail, get help, get version, sarif, ...):
  • Skips certain tests based on certain fields in the linter or certain conditions.
  • The magic of those "_good", "_bad" and "_fix" in folder or filenames
  • Indicate that input files must be created for each test that requires them (success, fail, fix).

I personally added these 2 blocks that I understand that they make development much easier and at the time I did not have this information:

https://megalinter.io/beta/contributing/#execute-the-tests-locally-visual-studio-code

https://megalinter.io/beta/contributing/#execute-linter-tests-inside-the-container

It helps you not to depend so much on CI and those eternal executions that frustrate a lot.

If you see something that can be improved, you can say it.

[nvuillam]

Maybe we could split the contributing into several pages ?
It would maybe look less messy ^^

[ThomasSanson]

test.sh

#!/bin/bash

name="${1:-kubernetes_helm}"
image_name="oxsecurity/megalinter-only-${name}"

# Build and tag the Docker image
docker buildx build -f "linters/${name}/Dockerfile" . --tag "${image_name}"

# Use the case substitution to get the linter name in uppercase
docker run \
  -e TEST_CASE_RUN=true \
  -e OUTPUT_DETAIL=detailed \
  -e TEST_KEYWORDS="${name^^}" \
  -e MEGALINTER_VOLUME_ROOT="." \
  -v "/var/run/docker.sock:/var/run/docker.sock:rw" \
  -v "$(pwd)":/tmp/lint \
  "${image_name}"

[ThomasSanson]

note :

if you don't want to mess with other kubernetes linters, just put your test files in a kubernetes_helm folder, and add test_folder: kubernetes_helm in the helm linter descriptor

[nvuillam]

Maybe something more generic like "If your linter requires different test files than in the descriptor default test folder, you can override the test folder by defining test_folder property at linter level , create the folder in .automation/tests and put your test files in it" :)

[ThomasSanson]

run_linter_locally.sh

#!/bin/bash

# Check that the user provided a linter name, ex : kubernetes_helm
if [ -z "$1" ]; then
  echo "$(tput setaf 1) Error: Please provide the name of the linter to run as the first argument like kubernetes_helm $(tput sgr0)"
  exit 1
fi

# Specify the name of the linter to run in lowercase {,,}
name="${1,,}"

# Specify the name of the Docker image to build
image_name="oxsecurity/megalinter-only-${name}"

# Build and tag the Docker image
docker buildx build -f "linters/${name}/Dockerfile" . --tag "${image_name}" || {
  echo "$(tput setaf 1) Error: Failed to build the Docker image. $(tput sgr0)"
  exit 1
}

# Use the case substitution to get the linter name in uppercase
docker run \
  -e TEST_CASE_RUN=true \
  -e OUTPUT_DETAIL=detailed \
  -e TEST_KEYWORDS="${name^^}" \
  -e MEGALINTER_VOLUME_ROOT="." \
  -v "/var/run/docker.sock:/var/run/docker.sock:rw" \
  -v "$(pwd)":/tmp/lint \
  -v "$(pwd)/reports:/tmp/reports" \
  "${image_name}" ||
  {
    echo "$(tput setaf 1) Error: Failed to run the linter. $(tput sgr0)"
    exit 1
  }

[ThomasSanson]

• Add linter desciptor

  # HELM LINT
  - linter_name: helm
    name: KUBERNETES_HELM
    linter_repo: https://github.com/helm/helm
    linter_url: https://helm.sh/docs/helm/helm_lint/
    linter_version_cache: "v3.11.1"
    files_sub_directory: "" 
    test_folder: kubernetes_helm
    cli_lint_mode: project
    cli_help_arg_name: -h
    cli_lint_extra_args:
      - "lint"
    linter_banner_image_url: https://user-images.githubusercontent.com/19731161/142411871-f695e40c-bfa8-43ca-97c0-94c256749732.png
    linter_text: |
      `helm lint` examine a chart for possible issues.
    examples:
      - helm lint .
      - helm lint --with-subcharts .
    install:
      apk:
        - helm

• add tests

  • create folder kubernetes_helm
    • create subfolder good
      • add your good file
    • create subfolder bad
      • add your bad file

• run ./build.sh to generate code

• run ./run_linter_locally.sh {name in lowercase}

[ThomasSanson]

rm -rf ./site
run npx mega-linter-runner --release beta --flavor python before pushing pull request

[ThomasSanson]

@nvuillam Hello,

When I run the command npx mega-linter-runner --release beta --flavor python, it hangs indefinitely and does not return control:

✅ Linted [SPELL] files with [cspell] successfully - (67.86s)
- Using [cspell v6.27.0] https://megalinter.io/cd00bc2af728511f450c232490fa45b5d5417d38/descriptors/spell_cspell
- MegaLinter key: [SPELL_CSPELL]
- Rules config: [/.github/linters/.cspell.json]
- Number of files analyzed: [747]

How can I fix this issue? Is the command I used correct to test at the very end?

[Kurt-von-Laven]

That looks right to me; I suspect you're running into #2348, which is the highest priority bug at the moment. Sorry about this. You can probably test on rootful Docker and/or in Windows.

[ThomasSanson]

Need to automate venv creation

python3 -m venv .venv
. .venv/bin/activate
echo ".venv/" >> .git/info/exclude
python3 -m pip install -U pip
python3 -m pip install -r requirements.dev.txt

[bdovaz]

Document /build command: #2445 (comment) #2445 (comment)

[echoix]

Document /build command: #2445 (comment) #2445 (comment)

But only internal collaborators/maintainers can trigger those, we chose to require the write permission for triggering these commands.

[bdovaz]

Document /build command: #2445 (comment) #2445 (comment)

But only internal collaborators/maintainers can trigger those, we chose to require the write permission for triggering these commands.

But it is also clearly indicated on the page (write permission or not), we need to add it in the appropriate section, isn't it?

https://megalinter.io/latest/contributing/#with-write-access

Edit: okay, I just realized it's already documented 😅

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[nvuillam]

So.... do we have an incoming PR for this issue ? :)

[ThomasSanson]

So.... do we have an incoming PR for this issue ? :)

Hi there! I apologize for any confusion, as I was actually working on another project where I plan to integrate Mega-Linter (https://github.com/vitabaks/postgresql_cluster).

Rest assured, I'll be addressing this issue and submitting a PR within a maximum of two weeks.

Thanks for your patience and understanding! 😊

[nvuillam]

@ThomasSanson no need to apologize, everyone works on open-source when available time allows that , there is no deadline, I just wanted to know if I needed to keep this issue open ;)

[ThomasSanson]

@nvuillam Thanks for understanding! I'll work on it when I can and keep you updated. You can keep the issue open until I submit the PR. Cheers!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.

[ThomasSanson]

I need to complete this pull request (#2737) before I can address this issue.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this issue should stay open, please remove the O: stale 🤖 label or comment on the issue.