gitlint slowdown inside docker #129
Comments
UpdateStory:I've searched cProfile outputs some more, and found that cal of _git is more likely to be source of a trouble. Than i found that _git uses sh call and this call is repeated multiple times. WorkaroundI've tried workaround from this post and it's seems to work for me! Result: But i think it still should be further investigated for regression of tool behavior. |
|
Hi! Thanks for the elaborate bug report! I was able to reproduce this # Native MacOS, MBP 2019 (2.6 GHz 6-Core Intel Core i7), gitlint 0.13.1
$ time gitlint -s --commits HEAD
gitlint -s --commits HEAD 5.66s user 7.40s system 101% cpu 12.835 total
# Docker on MacOS, same system
# Note that docker on Mac will always be slower since it runs in a VM
$ time docker run -v $(pwd):/repo jorisroovers/gitlint:latest -s --commits HEAD
docker run -v $(pwd):/repo jorisroovers/gitlint:latest -s --commits HEAD 0.05s user 0.03s system 0% cpu 8:33.18 total
$ time docker run --ulimit nofile=1024 -v $(pwd):/repo jorisroovers/gitlint:latest -s --commits HEAD
docker run --ulimit nofile=1024 -v $(pwd):/repo jorisroovers/gitlint:latest - 0.03s user 0.07s system 0% cpu 1:17.45 totalSo ~8.5 min vs ~1.5min using your suggested workaround. I repeated this a few times and got similar results each time. I'll make sure to update the docs and Dockerfile the next time I work on gitlint and before closing this out :-) |
- IMPORTANT: Gitlint 0.14.x will be the last gitlint release to support Python 2.7 and Python 3.5, as both are EOL which makes it difficult to keep supporting them. - Python 3.9 support - New Rule: title-min-length enforces a minimum length on titles (default: 5 chars) (#138) - New Rule: body-match-regex allows users to enforce that the commit-msg body matches a given regex (#130) - New Rule: ignore-body-lines allows users to ignore parts of a commit by matching a regex against the lines in a commit message body (#126) - Named Rules allow users to have multiple instances of the same rule active at the same time. This is useful when you want to enforce the same rule multiple times but with different options (#113, #66) - User-defined Configuration Rules allow users to dynamically change gitlint's configuration and/or the commit before any other rules are applied. - The commit-msg hook has been re-written in Python (it contained a lot of Bash before), fixing a number of platform specific issues. Existing users will need to reinstall their hooks (gitlint uninstall-hook; gitlint install-hook) to make use of this. - Most general options can now be set through environment variables (e.g. set the general.ignore option via GITLINT_IGNORE=T1,T2). The list of available environment variables can be found in the configuration documentation. - Users can now use self.log.debug("my message") for debugging purposes in their user-defined rules. Debug messages will show up when running gitlint --debug. - Breaking: User-defined rule id's can no longer start with 'I', as those are reserved for built-in gitlint ignore rules. - New RegexOption rule option type for use in user-defined rules. By using the RegexOption, regular expressions are pre-validated at gitlint startup and compiled only once which is much more efficient when linting multiple commits. - Bugfixes: - Improved UTF-8 fallback on Windows (ongoing - #96) - Windows users can now use the 'edit' function of the commit-msg hook (#94) - Doc update: Users should use --ulimit nofile=1024 when invoking gitlint using Docker (#129) - The commit-msg hook was broken in Ubuntu's gitlint package due to a python/python3 mismatch (#127) - Better error message when no git username is set (#149) - Options can now actually be set to None (from code) to make them optional. - Ignore rules no longer have "None" as default regex, but an empty regex - effectively disabling them by default (as intended). - Contrib Rules: - Added 'ci' and 'build' to conventional commit types (#135) - Under-the-hood: minor performance improvements (removed some unnecessary regex matching), test improvements, improved debug logging, CI runs on pull requests, PR request template. Full Release details in CHANGELOG.md.
Problem
When using gitlint from docker container (either jorisroovers/gitlint or custom) it's speed drops in orders of magnitude. I've tried to lint my own repo in ci, using docker container, and encountered slowdown for 30 second, than i've tested same command using my host OS - and saw results of linting right away.
Expected Behavior
Fast linting of repository git history tree from docker container.
Actual Behavior
Super slow linting in docker container.
Steps to Reproduce the Problem
Test running natively in host system
real 0m4,951s
user 0m3,709s
sys 0m2,624s
Test running in docker container
real 6m31,495s
user 0m0,056s
sys 0m0,026s
Specifications
--debug flag output from host run: Click me
--debug flag output inside docker: Click me
Hints
I'm not a python programmer, so this maybe misleading but, what i've tested so far:
Problem exists even if:
and found that big difference is between this calls:
native 4 seconds :
10207/7710 0.008 0.000 4.859 0.001 cache.py:7(_try_cache)vs in docker 394 seconds:
10207/7710 0.008 0.000 394.278 0.051 cache.py:7(_try_cache)maybe this can give you a hint.
Ps
The text was updated successfully, but these errors were encountered: