-
Notifications
You must be signed in to change notification settings - Fork 10.6k
[update-checkout] switch to using a ThreadPool for scheduling git commands #84911
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[update-checkout] switch to using a ThreadPool for scheduling git commands #84911
Conversation
|
@swift-ci please smoke test |
25bb836 to
40c3f1d
Compare
|
@swift-ci please smoke test |
40c3f1d to
a333f7b
Compare
|
@swift-ci please smoke test linux |
a333f7b to
f1435cc
Compare
|
@swift-ci please smoke test linux |
f1435cc to
f2a0c23
Compare
|
@swift-ci please smoke test linux |
f2a0c23 to
7122f0a
Compare
|
@swift-ci please smoke test |
7122f0a to
051d570
Compare
|
@swift-ci please smoke test |
051d570 to
7d41b5c
Compare
|
@swift-ci please smoke test |
7d41b5c to
9d3d43c
Compare
|
@swift-ci please smoke test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Based on the explanation this sounds like a good change.
This patch makes
update-checkoutuse aThreadPoolExecutorinstead of amultiprocessing.Pool. The main benefits are:update-checkoutget re-executed every time a subprocess starts. This causes problems when updating theswiftrepository as theupdate-checkoutfiles are edited which can cause functions and modules to no longer be available. With threads, the modules are already loaded in memory, we don't rely on the filesystem.gitcommands are invoked throughPopenmeaning they are executed in a subprocess. There is no need to open a subprocess in themultiprocessing.Poolto run other subprocesses. A Python thread is a more light weight solution for scheduling tasks.This patch also refactors the
shell.runandshell.captureinvocations: before, they were relying onpushdandpopdwith Python context managers. This does not work with multithreading as theshellmodule is now shared between threads. Now,gitcommands are executed withgit -C /repo_pathinstead of relying on the working directory.rdar://162775786