Using multiprocessing with a pool size option #14
Conversation
|
|
||
| vladfile = find_vladfile(options.vladfile) | ||
| if not vladfile: | ||
| logger.error( | ||
| "Could not find any vladfile! Ensure file ends in '.py' and see --help for available options.") | ||
| sys.exit(1) | ||
| return EXIT_SUCCESS |
There was a problem hiding this comment.
This should be a failure, not a success.
|
Thanks for doing this! Can you add some documentation for this feature to the |
adityanatraj
force-pushed
the
multiprocesses
branch
3 times, most recently
from
fd53c44
to
dfefcae
Compare
|
updated! I wasn't sure the best way to reveal the feature in the There doesn't appear to be a clear section dedicated to command-line usage; what do you think? |
|
Hmm, good point. It probably deserves it's own "Command Line Args" section instead which can sit before the "Testing" section, "Features" is a little more high-level. It's OK if it's just the one flag for now, I made #15 to eventually document the rest. |
| **Validate multiple files in parallel with multiprocessing** | ||
| specify `-p <number_of_processes>` or `--processes <number_of_processes>` to | ||
| parallelize across up to that many processes. | ||
|
|
There was a problem hiding this comment.
Can you add the default value to the docs as well?
adityanatraj
force-pushed
the
multiprocesses
branch
from
dfefcae
to
9f15bc1
Compare
|
updated with default value in readme |
| **Validate multiple files in parallel with multiprocessing** | ||
| specify `-p <number_of_processes>` or `--processes <number_of_processes>` to | ||
| parallelize across up to that many processes. (it defaults to 1 when unspecified) | ||
|
|
There was a problem hiding this comment.
Ah sorry, I was unclear. Can you move this into it's own section "Command Line Args" after the "Testing" section?
adityanatraj
force-pushed
the
multiprocesses
branch
2 times, most recently
from
d3efbf2
to
11be216
Compare
adityanatraj
force-pushed
the
multiprocesses
branch
from
11be216
to
2fb6274
Compare
|
retry :) |
|
👍 !!! |
Using multiprocessing with a pool size option
This allows one to set a pool size that the Vlad Classes will get distributed over to get validated. After all the classes are
vladiated, the vladiation success is combined to allow the exit code to be used in other scripts.