Tag some automatically updated files as generated files

[echoix]

Some committed helper files are automatically generated and are often updated.
These files are a good candidate to be tagged as generated files in the .gitattributes file.
As described in the docs here, https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github, adding linguist-generated=true to a pattern (files in our case), hides changes by default in diffs (collapsed like large diffs), and are ignored in the repo's language statistics.

For now, I added only two files.

• .automation/generated/flavor-stats.json
• .automation/generated/linter-helps.json

I'd like some input on at least the inclusion of the linter-helps.json file. On one side, seeing in the diff is quite verbose for nothing since some python linters use a non-deterministic set to store the options, so the order is always different at each invocation. It is redundant with the content that we will see underneath for the doc pages. But on the other side, it may help to see a little faster if a linter has its options changed if we happen to look at it: if the diff is not collapsed as a large diff, it would be at the top of the diff view.

For other files, especially the linter-versions.json, I think it is useful to have a quick view of what changes are concerned in the PR.
For megalinter-users.json, it wasn't updated in a week, but before was updated more often before that. This is another file that is simply storing data.

Proposed Changes

1. Add some files in to the .gitattributes file with a tag linguist-generated=true

Readiness Checklist

Author/Contributor

• Add entry to the CHANGELOG listing the change and linking to the corresponding issue (if appropriate)
• If documentation is needed for this change, has that been included in this pull request

Reviewing Maintainer

• Label as breaking if this is a large fundamental change
• Label as either automation, bug, documentation, enhancement, infrastructure, or performance

[nvuillam]

🦙 MegaLinter status: ⚠️ WARNING

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ BASH	bash-exec	7		0	0.03s
✅ BASH	shellcheck	7		0	0.46s
✅ BASH	shfmt	7	0	0	0.41s
✅ COPYPASTE	jscpd	yes		no	3.22s
✅ DOCKERFILE	hadolint	105		0	10.49s
✅ JSON	eslint-plugin-jsonc	21	0	0	2.36s
✅ JSON	jsonlint	19		0	0.26s
✅ JSON	v8r	21		0	15.95s
⚠️ MARKDOWN	markdownlint	305	0	229	7.41s
✅ MARKDOWN	markdown-link-check	305		0	6.38s
✅ MARKDOWN	markdown-table-formatter	305	0	0	21.8s
✅ OPENAPI	spectral	1		0	0.92s
⚠️ PYTHON	bandit	176		45	2.77s
✅ PYTHON	black	176	0	0	4.97s
✅ PYTHON	flake8	176		0	4.07s
✅ PYTHON	isort	176	0	0	0.83s
✅ PYTHON	mypy	176		0	8.07s
✅ PYTHON	pylint	176		0	14.52s
⚠️ PYTHON	pyright	176		282	20.95s
✅ REPOSITORY	checkov	yes		no	32.32s
✅ REPOSITORY	git_diff	yes		no	0.48s
✅ REPOSITORY	secretlint	yes		no	9.44s
✅ REPOSITORY	trivy	yes		no	28.91s
✅ SPELL	cspell	726		0	22.82s
✅ SPELL	misspell	547	0	0	1.08s
✅ XML	xmllint	3	0	0	0.4s
✅ YAML	prettier	81	0	0	3.81s
✅ YAML	v8r	23		0	64.3s
✅ YAML	yamllint	82		0	1.53s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by

[nvuillam]

Nice :)

There are also the following files, that always appear in diff whereas it is only caused by a randomization of list orders in help content

• docs/descriptors/python_black.md
• docs/descriptors/repository_checkov.md
• docs/descriptors/terraform_checkov.md

About megalinter-users, it is not used anymore, as we use github-dependent-infos to collect the list of MegaLinter users into file used-by-stats.md

[echoix]

I had a question about the CHANGELOG checkbox in the PR templates. If it is caught by the automatic updating, is it always necessary?

[nvuillam]

@echoix auto update linters job updates it, but it can also be updated manually for other PRs :)

When it is not done, I accept PRs anyway but it is more work for me when I release a new version, because I have to complete it :p

[echoix]

There are also the following files, that always appear in diff whereas it is only caused by a randomization of list orders in help content

• docs/descriptors/python_black.md
• docs/descriptors/repository_checkov.md
• docs/descriptors/terraform_checkov.md

Are you sure you'd want these too considered as excluded from the stats and all? Ideally, we would have the fixes applied upstream, like it was done for Flake8 in February 2022 (PyCQA/flake8#1550).

[nvuillam]

Good :)

[echoix]

I finally went and wrote the issue and fix psf/black#3515

[codecov-commenter]

Codecov Report

Merging #2285 (3883074) into main (23ced5e) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #2285      +/-   ##
==========================================
+ Coverage   82.39%   82.41%   +0.02%     
==========================================
  Files         171      171              
  Lines        4521     4521              
==========================================
+ Hits         3725     3726       +1     
+ Misses        796      795       -1     

Impacted Files	Coverage Δ
megalinter/reporters/UpdatedSourcesReporter.py	89.74% <0.00%> (+2.56%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[nvuillam]

🦙 MegaLinter status: ⚠️ WARNING

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
✅ BASH	bash-exec	7		0	0.03s
✅ BASH	shellcheck	7		0	0.49s
✅ BASH	shfmt	7	0	0	0.05s
✅ COPYPASTE	jscpd	yes		no	3.1s
✅ DOCKERFILE	hadolint	105		0	11.11s
✅ JSON	eslint-plugin-jsonc	21	0	0	2.37s
✅ JSON	jsonlint	19		0	0.26s
✅ JSON	npm-package-json-lint	yes		no	0.83s
✅ JSON	v8r	21		0	16.52s
⚠️ MARKDOWN	markdownlint	305	0	229	7.5s
✅ MARKDOWN	markdown-link-check	305		0	6.27s
✅ MARKDOWN	markdown-table-formatter	305	0	0	18.27s
✅ OPENAPI	spectral	1		0	1.08s
⚠️ PYTHON	bandit	176		45	3.04s
✅ PYTHON	black	176	0	0	5.02s
✅ PYTHON	flake8	176		0	2.46s
✅ PYTHON	isort	176	0	0	0.52s
✅ PYTHON	mypy	176		0	9.91s
✅ PYTHON	pylint	176		0	15.87s
⚠️ PYTHON	pyright	176		280	23.49s
✅ REPOSITORY	checkov	yes		no	34.38s
⚠️ REPOSITORY	devskim	yes		61	1.68s
✅ REPOSITORY	dustilock	yes		no	1.93s
✅ REPOSITORY	git_diff	yes		no	0.03s
✅ REPOSITORY	secretlint	yes		no	5.87s
✅ REPOSITORY	syft	yes		no	3.88s
✅ REPOSITORY	trivy	yes		no	24.11s
✅ SPELL	cspell	726		0	27.74s
✅ SPELL	misspell	547	0	0	0.67s
✅ XML	xmllint	3	0	0	0.04s
✅ YAML	prettier	81	0	0	3.12s
✅ YAML	v8r	23		0	64.18s
✅ YAML	yamllint	82		0	1.94s

See detailed report in MegaLinter reports

You could have same capabilities but better runtime performances if you request a new MegaLinter flavor.

• Click here to request the new flavor

MegaLinter is graciously provided by

[echoix]

I reran only the java_checkstyle failed build and it didn't fail. It failed somewhere with the curl command. Either the pipe was broken when grepping the url, so something like this could help us see : https://stackoverflow.com/a/28879552, or the folder (output) destination didn't exist or wrong permissions (we could always use the --create-dirs), or what I think happened (but can't seem plausible) is that there were no space left on the runner, if it wanted to create the file. The silent and no progress switch is on, so we can't see the problem.

[nvuillam]

Such issue is usually the call limits to api.github .com that has been reached

It just has to be run again :)

And this problem will disappear when we'll succeed to send GITHUB_TOKEN in auth headers ^^

[echoix]

I finally went and wrote the issue and fix psf/black#3515

The PR associated with this issue was merged and now the new Black version containing my fix is released!

[nvuillam]

If the new version of black is not caught by auto-update job, it's because we need to upgrade python version... still waiting for a release of multiprocessing-logging to do that :/

jruere/multiprocessing-logging#56