Multithread keogram #664
Merged
Multithread keogram #664
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Inspired by @sebmueller. This splits up the input images into batches of work more or less evenly divided across the number of processors. Two new flags are added: * -n / --nice to make the workers be nice. default = 10 * -m / --max-threads to control the number of worker threads. default = ncpus Allows me to process a night of startrails on my workstation in under 3.5s. On my pi3b+, it'll use between 350% - 395% CPU without making it feel slow ``` root@skycam:/home/allsky/src# time ./startrails -b 0.255 -d ../images/20210912/ -e jpg -o /tmp/junk.jpg -s 1280x960 Minimum: 0.0265446 maximum: 0.804403 mean: 0.143959 median: 0.137787 real 5m5.951s user 11m22.243s sys 1m5.347s root@skycam:/home/allsky/src# time ./startrails -b 0.255 -d ../images/20210912/ -e jpg -o /tmp/junk2.jpg -s 1280x960 --max-threads 1 Minimum: 0.0265446 maximum: 0.804403 mean: 0.143959 median: 0.137787 real 12m8.758s user 11m3.073s sys 0m54.200s ``` Fixes #628
Inspired by @sebmueller. This splits up the input images into batches of work more or less evenly divided across the number of processors. Two new flags are added (-m and -n were already used): * -q / --level to make the workers be nice. default = 10 * -Q / --max-threads to control the number of worker threads. default = ncpus Generates a keogram on my laptop in under 3.5s, even after doing horrible string manipulations to compute time of exposure for label generation. Tested on rpi3b too.
now both of these tools use -Q / --max-threads to control the number of worker threads, and -q / --nice-level to set the scheduler priority. Q is vaguely mnemonic for queue (job scheduling) and it's not likely that you will want to change the settings without a good reason. Use as much CPU as possible, without dragging the whole system down, right?
EricClaeys
approved these changes
Oct 16, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Apply similar logic to keogram as was done for startrails (#628 and #649)