Lidar point cloud classification

[ymoisan]

IMPORTANT : This is a duplicate of #2

Lidar point cloud classification

Context

Airborne lidar systems generate data in the form of point clouds. The American Society for Photogrammetry and Remote Sensing (ASPRS) -- which now tags itself as The Imaging and Geospatial Information Society -- has a Lidar division the mission of which is "to provide a forum for collection development and dissemination of information related to the best practices in developing maintaining and operating kinematic laser scanners and associated sensors".

The LAS Working Group maintains the LAS file format standard, which is the default format for storing lidar data points. The latest version of the standard, LAS 1.4, provides the structure of an individual point data record:

Table 1: Point Data Record Format 0

Item	Format	Size	Required
X	long	4 bytes	*
Y	long	4 bytes	*
Z	long	4 bytes	*
Intensity	unsigned short	2 bytes
Return Number	3 bits (bits 0–2)	3 bits	*
Number of Returns (given pulse)	3 bits (bits 3–5)	3 bits	*
Scan Direction Flag	1 bit (bit 6)	1 bit	*
Edge of Flight Line	1 bit (bit 7)	1 bit	*
Classification	unsigned char	1 byte	*
Scan Angle Rank	char	1 byte	*
User Data	unsigned char	1 byte
Point Source ID	unsigned short	2 bytes	*

The classification item in the table above is further specified in the following two tables:

Table 2: Classification Bit definition (field encoding)

Bit	Field Name	Description
0:4	Classification	Standard ASPRS classification from 0 - 31 as defined in the classification table for legacy point formats (see table 3 below)
5	Synthetic	If set then this point was created by a technique other than LIDAR collection such as digitized from a photogrammetric stereo model or by traversing a waveform.
6	Key-point	If set, this point is considered to be a model key-point and thus generally should not be withheld in a thinning algorithm.
7	Withheld	f set, this point should not be included in processing (synonymous with Deleted).

**Table 3: ASPRS Standard LIDAR Point Class Values**

Classification Value (bits 0:4)	Meaning
0	Created, never classified
1	Unclassified
2	Ground
3	Low Vegetation
4	Medium Vegetation
5	High Vegetation
6	Building
7	Low Point (noise)
8	Model Key-point (mass point)
9	Water
10	Reserved for ASPRS Definition
11	Reserved for ASPRS Definition
12	Overlap Points2
13-31	Reserved for ASPRS Definition

Therefore, the format for the classification field is a bit encoding with the lower five bits used for the class value (as shown in table 3 above) and the three high bits used for flags.

Problem

The issue here is that the classification of lidar points is both expensive (typically 30 % of total data acquisition cost) and rather unreliable. We would like to devise a deep learning approach to help us classify data points independently, that is irrespective of class values assigned (or not) by the data provider.

Literature review

There are roughly two types of methods for lidar point cloud classification using DL:

• rasterization of certain point atttributes so that CNNs may be used
• direct manipulation of the point cloud, for example using voxels

The methods we are most interested in are those that have a PyTorch implementation.

CNN

Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network

"With several selected attributes of LiDAR point clouds, our method first creates a group of multi-scale contextual images for each point in the data using interpolation. Taking the contextual images as inputs, a multi-scale convolutional neural network (MCNN) is then designed and trained to learn the deep features of LiDAR points across various scales. A softmax regression classifier (SRC) is finally employed to generate classification results of the data with a combination of the deep features learned from various scales. "

3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks

"The point cloud ... the common solution [for classification] of transforming the data into a 3D voxel grid introduces its own challenges, mainly large memory size ... we propose a novel 3D point cloud representation called 3D Modified Fisher Vectors (3DmFV) ... it combines the discrete structure of a grid with continuous generalization of Fisher vectors ... Using the grid enables us to design a new CNN architecture for point cloud classification and part segmentation."

Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs

Point clouds

Using CNNs implies rasterization of point attribute values, which in turn implies degradation of the level of information contained in the point cloud. Ideally, we would like our algorithms to work directly in the point cloud. The main problem here is the data volume. We should strive to find approaches that minimize data I/O. Things like Entwine may allow us to request point data for training or inference via web services and therefore avoid data duplication.

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation

Point cloud ... due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images.

"This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds and well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective."

Spherical Convolutional Neural Network for 3D Point Clouds

"We propose a neural network for 3D point cloud processing that exploits `spherical' convolution kernels and octree partitioning of space. The proposed metric-based spherical kernels systematically quantize point neighborhoods to identify local geometric structures in data ... The network architecture itself is guided by octree data structuring that takes full advantage of the sparse nature of irregular point clouds. We specify spherical kernels with the help of neurons in each layer that in turn are associated with spatial locations. We exploit this association to avert dynamic kernel generation during network training, that enables efficient learning with high resolution point clouds. We demonstrate the utility of the spherical convolutional neural network for 3D object classification on standard benchmark datasets."

Interactive Visualization of 10M+ 3D Points with New Open-Source Python Package PPTK

"The PPTK viewer is part of a larger effort of developing a Python toolkit for not only visualizing but also processing point data."

[mpelchat04]

This ticket is a duplicate of #2