pdal performance vs. lastools / pdal is slow on las files larger than 50mb

[KlemenSpruk]

Hello!
I'm testing pdal crop filter and compairing time performance to lastools.
Tests are running on Ubuntu 16.04. Lastools algorithms are run using wine.

Pdal is approximately 15 slower than lastools. Does any one have any experience how to
set pdal to process faster (e. g.: number of core settings, ...)? Or this is the speed achieved for now?

Thanks for any info!

Have a nice day, Klemen!

[abellgithub]

See #726 and #754.

1. If you are building the libraries, make sure you're using Release mode. Debug is very slow.
2. You'll have to provide more details in order for us to provide you with any information.

[KlemenSpruk]

Thanks for answer! I didn't check which mode I was using when compiling. Will do and report back!

[KlemenSpruk]

Hello!

I compiled pdal again.

My settings:

cmake version 3.5.1

pdal master branch

sudo mkdir build
cd build
sudo cmake .. -DCMAKE_BUILD_TYPE=Release -DWITH_PYTHON=OFF -DWITH_TESTS=OFF -DWITH_LASZIP=ON -DWITH_GEOTIFF:BOOL=OFF

Output:
CMake Warning (dev) at dimbuilder/CMakeLists.txt:26 (target_link_libraries):
Policy CMP0023 is not set: Plain and keyword target_link_libraries
signatures cannot be mixed. Run "cmake --help-policy CMP0023" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

The keyword signature for target_link_libraries has already been used with
the target "dimbuilder". All uses of target_link_libraries with a target
should be either all-keyword or all-plain.

The uses of the keyword signature are here:

• dimbuilder/CMakeLists.txt:24 (target_link_libraries)

This warning is for project developers. Use -Wno-dev to suppress it.

-- The following features have been enabled:

• PDAL application , the PDAL command line application

-- The following OPTIONAL packages have been found:

• PostgreSQL
• Curl
• JSONCPP (required version >= 1.6.2)

-- The following RECOMMENDED packages have been found:

• LASzip
  Provides LASzip compression
  Provides LASzip compression

-- The following REQUIRED packages have been found:

• GDAL (required version >= 1.9.0)
  Provides general purpose raster, vector, and reference system support
• GEOS (required version >= 3.3)
  Provides general purpose geometry support
• ZLIB
  Compression support in BPF
• Threads , The thread library of the system

-- The following features have been disabled:

• Bash completion , completion for PDAL command line
• CPD plugin , run Coherent Point Drift on two datasets
• GeoWave plugin , Read and Write data using GeoWave
• Greyhound plugin , read points from a Greyhound server
• Hexbin plugin , determine boundary and density of a point cloud
• Icebridge plugin , read data in the Icebridge format
• Matlab plugin , write data to a .mat file
• MrSID plugin , read data in the MrSID format
• NITF plugin , read/write LAS data wrapped in NITF
• PCL plugin , provides PCL-based readers, writers, filters, and kernels
• PostgreSQL PointCloud plugin , read/write PostgreSQL PointCloud objects
• SQLite plugin , read/write SQLite objects
• RiVLib plugin , read data in the RXP format
• Python plugin , add features that depend on python
• Unit tests , PDAL unit tests

-- Configuring done
-- Generating done
-- Build files have been written to: /usr/src/PDAL/build

sudo make
sudo make install

After testing performance again pdal is 1.95 slower than lastools. This is already simmilar to results to data in #726.

Is there any optimization still possible? Because average time processing same data takes 6.7 sec on Ubuntu 16.04 and 6.18 sec on Ubuntu 16.04 through docker pdal container.

Thanks Klemen!

[KlemenSpruk]

Hello!

With further testing I noticed that pdal becomes slow when dealing with larger files (size > 50MB).

My pipeline example:

{
"pipeline": [
{
"filename": "/var/petrol_000191.las",
"tag": "0"
},
{
"filename": "/var/petrol_000212.las",
"tag": "1"
},
{
"type": "filters.crop",
"polygon": "POLYGON((495292.23078440886 155586.21489483098,495292.23078440886 155507.57366920903,495389.0836622801 155508.81537277147,495370.0442076558 155596.56242451805,495292.23078440886 155586.21489483098))",
"inputs": [
"0"
],
"tag": "0_clipped"
},
{
"type": "filters.crop",
"polygon": "POLYGON((495292.23078440886 155586.21489483098,495292.23078440886 155507.57366920903,495389.0836622801 155508.81537277147,495370.0442076558 155596.56242451805,495292.23078440886 155586.21489483098))",
"inputs": [
"1"
],
"tag": "1_clipped"
},
{
"type": "filters.merge",
"inputs": [
"0_clipped",
"1_clipped"
],
"tag": "merged"
},
{
"type": "writers.las",
"filename": "/var/www/tmp/582335bad40e3-merged-clips.las"
}
]
}

If I alter the clip and merge operation there is no significant diffrence in speed.

Any advice for optimisation besides splitting input las files before processing?

Thanks Klemen!

[abellgithub]

This can be simplified, but I have no idea how it might impact performance on your system. Again, PDAL is NOT going to be as fast as lastools. If you're expecting that, you should use lastools. PDAL supports lots of formats/algorithms/extensibility that isn't supported with lastools and therefore can't be as optimized for the specific cases that lastools handles.

{
"pipeline": [
"/var/petrol_000191.las",
"/var/petrol_000212.las",
{
"type": "filters.crop",
"polygon": "POLYGON((495292.23078440886 155586.21489483098,495292.23078440886 155507.57366920903,495389.0836622801 155508.81537277147,495370.0442076558 155596.56242451805,495292.23078440886 155586.21489483098))"
},
"/var/www/tmp/582335bad40e3-merged-clips.las"
]
}

[abellgithub]

Make sure you have plenty of memory on your system that is usable by PDAL.

[KlemenSpruk]

Hello!

I will try to simplify the pipeline. I will try to optimize input file size and test different pipeline workflows.

Thanks for all the help!