Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve speed of mega-linter #1087

Closed
BD103 opened this issue Mar 10, 2024 · 3 comments · Fixed by #1090
Closed

Improve speed of mega-linter #1087

BD103 opened this issue Mar 10, 2024 · 3 comments · Fixed by #1090

Comments

@BD103
Copy link
Member

BD103 commented Mar 10, 2024

mega-linter is a tool that we use to check all HTML, Markdown, and Sass files. It currently takes around 3 minutes to run, which is the longest of the 5 initial lint jobs. Since it is run first, it blocks the generate-* and build-website jobs from starting. Increasing the speed of mega-linter would increase the speed of the entire CI.

The slowest part of using mega-linter is not running it, but instead downloading it. It requires downloading a 3.5 GB docker image, which contains a whole host of unused linters.

mega-linter allows using custom flavors, which are slim versions of the default image. If the CI could be configured to use one of those flavors instead, the CI times would be decreased. We currently only use 3 linters, so only they are required:

ENABLE_LINTERS:
- HTML_DJLINT
- MARKDOWN_MARKDOWNLINT
- EDITORCONFIG_EDITORCONFIG_CHECKER

I additionally asked in Github Discussions how to create a custom flavor of mega-linter that just includes the linters we need. If that's not possible and a provided flavor is not found, it may be worth just installing each linter manually.

@TrialDragon
Copy link
Member

If making our own custom flavor isn't worth it, then I think https://megalinter.io/latest/flavors/javascript/ should work fine. It contains all the things that I think we would use. But like you said manually installing is also viable given how few there is right now.

@BD103
Copy link
Member Author

BD103 commented Mar 10, 2024

Ok, then I can see this going in two steps:

  1. Switch the current flavor to Javascript, which should decrease the download size to ~800 MB.
  2. If it's still too slow (I'm thinking >30 seconds), then we can try creating our own flavor.
    • Someone respond to my question with good information on how to create a custom flavor.

I'm going to mark this as good first issue so someone can try item 1.

@BD103
Copy link
Member Author

BD103 commented Mar 12, 2024

With #1090 merging, I don't think we need to create a custom flavor. The Javascript one speeds up mega-linter enough for now that it's not worth the effort of maintaining a fork.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants