Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit flake8 to a single job per test #55

Closed
wants to merge 1 commit into from

Conversation

timj
Copy link

@timj timj commented Oct 29, 2018

Without this change flake8 creates many subprocesses every time
it is invoked. This causes significant overhead and slows down
test execution. In conjunction with pytest-xdist many hundreds
of additional flake8 processes can be created which causes
significant load issues.

@timj
Copy link
Author

timj commented Oct 29, 2018

I am getting the tox failure even on master so I'm not sure what is going wrong. This patch fixes my local installation and makes things significantly faster.

@peleccom
Copy link

peleccom commented Nov 6, 2018

if case this is true
"Without this change flake8 creates many subprocesses every time
it is invoked. "
it should be fixed.
For some CI servers (CirlceCI for example) it might cause serious performance issues. It is 22 cpus cores visible but due to virtualization design only 2 can be used on default free configuration. flake8 should be single thread on CI servers

I am looking forward for this change merged.

In connection to failed Travis build. I think it was temporary issues on Travis or Github. I think new build could fix this error.

@timj timj force-pushed the u/timj/limit-njobs branch 2 times, most recently from 8fcf45b to 387c08d Compare January 30, 2019 23:46
@timj
Copy link
Author

timj commented Jan 30, 2019

I've rebased against current master and I hope the travis checks work this time. This fix really makes a huge performance difference when using this plugin and is critical for people who use pytest-xdist.

@timj
Copy link
Author

timj commented Feb 5, 2019

@tholo I rebased and I still get tox errors. They don't seem to be related to my PR.

Without this change flake8 creates many subprocesses every time
it is invoked. This causes significant overhead and slows down
test execution. In conjunction with pytest-xdist many hundreds
of additional flake8 processes can be created which causes
significant load issues.
@timj
Copy link
Author

timj commented Feb 6, 2019

Rebasing again has finally sorted out the travis failures.

@timj
Copy link
Author

timj commented Jun 11, 2019

Is there anything else I can do to help this PR get merged?

@frossie
Copy link

frossie commented Sep 21, 2019

@tholo What can we do to help this PR get merged?

@timj
Copy link
Author

timj commented May 25, 2020

It looks like flake8 has been modified to make this issue go away.

@timj timj closed this May 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants