WIP r.in.pdal - PDAL-based version of r.in.lidar #61
Conversation
|
liblas is deprecated on Ubuntu: https://trac.osgeo.org/grass/ticket/4020 |
|
@wenzeslaus What is missing to merge this PR to master? It would be nice to have |
|
Related issue: What about moving liblas-based modules to addons? Than |
|
Just a note that v.in.pdal does not show up in the menu. Both r.in.pdal and v.in.pdal should be in the import menus at least. |
|
The branch is now rebased and the code is available for testing: |
Reads points using PDAL, uses several PDAL filters including reprojection, filters using custom filter, writes into memory structures. Reuses r.in.lidar functionality with moving code into PDAL Filter and Writer classes. Preserves most of r.in.lidar functionality, but misses several features most notably scanning and auto extent. Furthermore, it now assumes that the whole output raster fits into memory. The multi-pass code was removed and it should be replaced by use of the improved Segment with all-in-memory mode.
|
@wenzeslaus thanks for having written this new module. I am currently testing it with the "open.NRW LiDAR" data (all LiDAR data of the federal state Nordrhein-Westfalen/DE are freely available, 2.2 TB of LAZ files). While compiling, I got these warnings ( |
|
I checked the parameters and flags, comparing them to the (Python) addon r.in.pdal which we use a lot: The Python version of Potentially wanted flags: -s Scan data file for extent then exit ... useful for getting an idea about the input LAS/LAZ file without import. Esp. nice to be able to set the computational region properly. However, perhaps it is solved differently in this new C implementation, I didn't check that yet. Desired parameters:
... useful for taking a GRASS raster maps (e.g. orthophoto) or an external file (e.g. DEM GeoTIFF) as the pixel geometry reference for the computational region.
... useful for generating a vector polygon (at least in my area the LAS/LAZ files are not delivered in square tiles but of rather arbitrary shape. |
These are one of the reason this is not finished; implementing these nicely requires some additional thought. But, in other words, I plan them. After GSoC, this PR coming up in my todo list, so there is hope. One issue there is metadata if present versus going through all the points. Going through all the point is how it is in r.in.lidar/r.in.xyz, any bad experience with that?
I'm sure you have heard about computational region, right? :-) If you want that for more than one module, the question is why not for all?
You mean as an output, right? v.in.lidar has that as an input (a mask/filter). Footprint as a vector map would be a vector polygon, no? How would a vector polygon match the raster cells? This sounds like a convex or concave hull. Is that what you mean? I don't know, but PDAL lib may require another run through the cloud to get that. GRASS has these algs, but only in modules, not lib, I think. |
Not a bad experience but certainly not fast. However, in r.in.lidar/r.in.xyz the PDAL API isn't really used and hence it might be all faster in this "pure" PDAL implementation.
I think so, see my line above :-)
Very easy to answer: not for all but only here as we do raster binning here which often depends on further (external) data. For example, we often use orthophotos along with LAS files. The pixel geometry (esp. position) needs to be aligned with the orthophotos in case you want to perform a classification on top of it, using both orthophoto and LiDAR data. However, this may happen in different parts of the data storage infrastructure and having the possibility to "grab" the pixel positioning from an external file (orthophoto, existing DEM) comes very handy.
Yes. The footprint as a vector map would be a vector polygon. Example:
This is the function, based on |
|
Initial functionality test: I was able to successfully generate a DSM from a LAZ point cloud: The result looks cool!
One issue (which I don't know how to address): water surface (river Rhine here) in open.NRW is set to the class 41 which, however, comes with ReturnNumber=0 (synthetic). At time "first, mid, last" returns are supported but not yet "synthetic". |
|
Please take a look at my request to remove ground filtering options from *.in.pdal at #1049 Reason – PDAL has already changed. This functionality could cause us to constantly change module parameters to catch-up with latest developments in PDAL. Support of pipelines could be a way out, but do we need to turn import module into a full filtering module? |
PDAL can work with other formats too and thus I would vote to keep it as r.in.pdal similarly to r.in.gdal. Removing existing liblas based modules for G8 seems like a good idea. I would even not move them to addons as anyone using liblas is on an old system (with old GRASS) anyway. |
Markus, please clarify your expectations. A separate filter to include/exclude points with Withheld/Keypoint/Synthetic flags set? A single point can have more than one flag set – how conflicts should be handled? PS. For testing you can use https://github.com/marisn/grass/tree/wenzeslaus-pdal-binning as it contains some extra functionality. |
No strong opinion here, perhaps an
Cool, I see new "binning" methods in https://github.com/marisn/grass/tree/wenzeslaus-pdal-binning/raster/r.in.pdal |
We have to thing more about how it should be presented to the user as class filter is already implemented and this is some kind of flag based filter.
Unfortunately I am just working on things I miss from the module and not features missing for final inclusion. Still I start to worry that soon my version will have diverged (from this PR) too much for an easy merge and will have to replace version of this PR. |
And thus it should go into an add-on. Markus, please open a feature request for such add-on. I could see this as an option for v.in.pdal, but certainly not in r.in.pdal. |
Here the use case: imagine a bunch of LAS/LAZ files (like many thousand) which partially overlap and are not rectangular as it happens with those provided for our federal state here (and no footprint map provided). With the possibility to write out the footprints of these files, you generate a tile index vector map while importing. |
Markus, there are a lot of useful features that could be implemented in r.in.pdal but the main question remains – should they. r.in.pdal is already quite feature heavy.
And still I think it would be best to have v.pdal.tileset module to implement such functionality to not make r.in.pdal even larger monster. KISS. |
|
Could we close this since it is obsolete? |
|
Replaced by #1200. |
r.in.pdal has most of the features from r.in.lidar and some from v.in.pdal. It uses PDAL streaming API as v.in.pdal and it uses segment lib to access the output raster, so it does not have the custom row cache and repeated looping over input point cloud (which was concern for combination of large output raster and large point cloud).
This is work in progress, but I'm not sure how far should this PR go and what to leave for the future. Features are there. CRS handling error, handling, and metadata printing is not finished. Tests are missing. Testing in general is welcome and needed. I was able to test only the basic functionality so far.
TODO: