New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GDALWarp doesn't use multiple cores #778
Comments
Found something here: http://osgeo-org.1560.x6.nabble.com/gdal-dev-gdalwarp-and-gdaladdo-in-multi-threaded-mode-td5252818.html Apparently there's 2 parameters on gdalwarp to speed up calculations using multiple processors/cores: |
This may not be a bug: if this task is I/O bound there's not much we can do besides using faster disks... |
I find myself constantly shaking my fists at GDAL. The code is found here: Can you try adding
|
Nor |
OK then. @pierotofy contributed this code, perhaps he can provide some insight |
http://www.gdal.org/gdalwarp.html So try to pass both? |
Also what version of gdalwarp are we running? |
I've tried passing both options too with no sucess. |
Did some digging, what happens if you pass:
-wo --> options passed to warp algorithm (which doesn't affect speed here, because we don't warp anything, we are just cropping) If it works, could you open a PR? |
Would be interesting to also see if performance increases when passing |
For For Also this |
If passing:
Doesn't improve performance, then I'm not sure what's the cause. When I pass both -co and -wo I see full usage of all my cores. 😕 |
Running |
GDAL Version: Command used: Runs only on 1 core. |
Thanks for the screenshots/info. Mm, could you share your odm_georeferenced_model.bounds.shp and odm_orthophoto.original.tif file? Trying to understand why I'm observing different results 😄 |
I've sent you the requested files on a private channel on Gitter, as I can't make them publicly available. |
So, the performance is almost certainly I/O and memory bound based on my observations. This is especially true for larger GeoTIFFs (which is what you are testing with). Options --> Time for 1 tick of processing -wo NUM_THREADS=ALL_CPUS --> 2:59 So the bottom line is that I don't think (hope somebody proves me wrong) there's not much to be gained by adding more cores (this might have been true if we were doing warping, but since we're just cropping I suspect most of the time is spent just doing I/O). We should tweak GDAL_CACHEMAX and -wm, but we need to be careful, choosing too high of a value will make the program fail (bad). Perhaps we can use Python to query the available memory, divide by 3 and use that. PR for this would be welcome if anyone wants to take a stab at it. |
--config GDAL_CACHEMAX $VALUE -wm $VALUE Can I put these parameters on all gdal_options? PR Incoming... 😄 |
I would recommend using %X for GDAL_CACHEMAX: wm requires more thought. https://trac.osgeo.org/gdal/wiki/UserDocs/GdalWarp `The -wm flag affects the warping algorithm. The warper will total up the memory required to hold the input and output image arrays and any auxilary masking arrays and if they are larger than the "warp memory" allowed it will subdivide the chunk into smaller chunks and try again. If the -wm value is very small there is some extra overhead in doing many small chunks so setting it larger is better but it is a matter of diminishing returns.` So adding more is not necessarily going to improve performance. I wouldn't add it to all commands unless you can measure a tangible improvement in performance. |
Perhaps related, but are we using |
I should say, I've never tested its effect on future GDAL operations, but it'd be good practice to use |
I just compared gdalwarp (GDAL 3.2.2, released 2021/03/05) with and without -multi option with NUM_THREADS=10 with some test data. I can see a 20% time reduction for this step 10 cpu multi and 10 cpu See also this threed @pierotofy Can you add this option ? |
Thanks @sbonaime, but did you test these benchmarks within ODM? |
Hey!
I'm running a very big project to identify bottlenecks on opendronemap and I've found out that gdalwarp is not running on multiple cores.
Output line from OpenDroneMap:
[DEBUG] running gdalwarp -cutline /code/odm_georeferencing/odm_georeferenced_model.bounds.shp -crop_to_cutline -co NUM_THREADS=ALL_CPUS -co BIGTIFF=IF_SAFER -co BLOCKYSIZE=512 -co COMPRESS=DEFLATE -co BLOCKXSIZE=512 -co TILED=YES -co PREDICTOR=2 /code/odm_orthophoto/odm_orthophoto.original.tif /code/odm_orthophoto/odm_orthophoto.tif
htop screenshot:
Isn't
-co NUM_THREADS=ALL_CPUS
suposed to make it run on multiple threads?The text was updated successfully, but these errors were encountered: