Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[r8] Apply pylint excessive memory usage fix #3017

Merged
merged 1 commit into from
Nov 25, 2020

Conversation

martinpitt
Copy link
Contributor

@martinpitt martinpitt commented Nov 24, 2020

With current astroid 2.4.2, pylint requires a ginormous amount of
memory. Apply the recent fix during container build, until a newer
version gets released.

Revert the hack from commit 2835615 and go back to pylint
auto-selecting the number of jobs based on number of CPUs.

Backported from master PR #3018

Related: rhbz#1885635

Copy link
Contributor

@rvykydal rvykydal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@martinpitt
Copy link
Contributor Author

On our github runner I built a new container from this branch. The tests are now already running with the new one. Memory usage is now a bit more reasonable, and with four jobs the test is also faster:

  11704 root      20   0  945148 593648  23432 R  99.3   7.3   1:04.85 runpylint.py                                                                                
  11705 root      20   0 1145828 725152  25744 R  98.0   8.9   1:05.25 runpylint.py                                                                                
  11706 root      20   0  963308 675964  25088 R  98.0   8.3   1:05.58 runpylint.py                                                                                
  11707 root      20   0  902964 559268  22064 R  97.7   6.9   1:05.12 runpylint.py                                                                               

See also my recent follow-up to pylint-dev/pylint#1495 (comment)

@martinpitt martinpitt changed the title Apply pylint excessive memory usage fix [r8] Apply pylint excessive memory usage fix Nov 24, 2020
With current astroid 2.4.2, pylint requires a ginormous amount of
memory. Apply the recent fix during container build, until a newer
version gets released.

Revert the hack from commit 2835615 and go back to pylint
auto-selecting the number of jobs based on number of CPUs.

Backported from master PR rhinstaller#3018

Related: rhbz#1885635
@martinpitt
Copy link
Contributor Author

Argh, forgot the magic "Related #rhbz#, and I also added a reference to PR #3018 (the corresponding master PR)

@martinpitt martinpitt merged commit c752463 into rhinstaller:rhel-8 Nov 25, 2020
@martinpitt martinpitt deleted the r8-pylint-mem-fix branch November 25, 2020 05:04
@jkonecny12
Copy link
Member

Argh, forgot the magic "Related #rhbz#, and I also added a reference to PR #3018 (the corresponding master PR)

That could be a nice workflow check on PR for rhel-8 branch really.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants