Add native PCD I/O

- Supports reading and writing ASCII - Supports reading and writing binary - Readers are streamable, while writers are not - Does NOT yet support binary compressed - Removes need for PcdCommon header A note about the PCD writer. Similar to other writers, the PCD writer can output a subset of the dimensions if keep_unspecified is set to false. In that case, only the dimensions specified in the order parameter are written in the output. The dimension type ("F", "I", or "U") and size (in bytes) can also be specified, along with precision in the case of ASCII output. This native driver for PCD will remove a long-standing issue with the previous PCL-backed drivers, which required a conversion to a fixed, single-precision point schema, that resulted in loss of extra dimensions (beyond XYZRGBA) and confusion surrounding subtraction of offsets to avoid loss of precision for certain coordinate systems.
PDAL · Jul 18, 2019 · 858ccfe · 858ccfe
1 parent af7831c
commit 858ccfe
Show file tree

Hide file tree

Showing 10 changed files with 721 additions and 540 deletions.
diff --git a/doc/stages/readers.pcd.rst b/doc/stages/readers.pcd.rst
@@ -1,18 +1,14 @@
 .. _readers.pcd:
 
-******************************************************************************
 readers.pcd
-******************************************************************************
-
+===========
 
 The **PCD Reader** supports reading from `Point Cloud Data (PCD)`_ formatted
 files, which are used by the `Point Cloud Library (PCL)`_.
 
-.. note::
-
-    The `PCD Reader` requires linkage of the `PCL`_ library.
+.. embed::
 
-.. plugin::
+.. streamable::
 
 
 Example
@@ -41,5 +37,4 @@ filename
 
 .. _Point Cloud Data (PCD): http://pointclouds.org/documentation/tutorials/pcd_file_format.php
 .. _Point Cloud Library (PCL): http://pointclouds.org
-.. _PCL: http://pointclouds.org
 
diff --git a/doc/stages/writers.pcd.rst b/doc/stages/writers.pcd.rst
@@ -7,14 +7,11 @@ The **PCD Writer** supports writing to `Point Cloud Data (PCD)`_ formatted
 files, which are used by the `Point Cloud Library (PCL)`_.
 
 By default, compression is not enabled, and the PCD writer will output ASCII
-formatted data. When compression is enabled, the output is PCD's
-binary-compressed format.
+formatted data.
 
-.. plugin::
+.. embed::
 
-.. note::
-
-    The `PCD Writer` requires linkage of the `PCL`_ library.
+.. streamable::
 
 
 Example
@@ -40,22 +37,26 @@ filename
   PCD file to write [Required]
 
 compression
-  Level of PCD compression to use (ascii, binary, compressed)
-  [Default: "ascii"]
+  Level of PCD compression to use (ascii, binary, compressed) [Default:
+  "ascii"]
 
-xyz
-  Write only XYZ dimension? [Default: "false"]
+_`precision`
+  Decimal Precision for output of values. This can be overridden for individual
+  dimensions using the order option. [Default: 3]
 
-subtract_minimum
-  Set origin to minimum of XYZ dimension? [Default: true]
+_`order`
+  Comma-separated list of dimension names in the desired output order. For
+  example "X,Y,Z,Red,Green,Blue". Dimension names can optionally be followed
+  with a colon (':') seperated list of field size, type, and in the case of
+  ASCII, precision. Ex: "X:4:F:2, Y:4:F:2, Z:4:F:3" If no precision is
+  specified the value provided with the precision_ option is used. Size
+  defaults to 8 bytes and type defaults to float. [Default: none]
 
-offset_x, offset_y, offset_z
-  Offset to be subtracted from XYZ position [Default: 0.0]
+keep_unspecified
+  If true, writes all dimensions. Dimensions specified with the order_ option
+  precede those not specified. [Default: **true**]
 
-scale_x, scale_y, scale_z
-  Scale to divide from XYZ dimension [Default: 1.0]
 
 .. _Point Cloud Data (PCD): http://pointclouds.org/documentation/tutorials/pcd_file_format.php
 .. _Point Cloud Library (PCL): http://pointclouds.org
-.. _PCL: http://pointclouds.org
 
diff --git a/io/PcdCommon.hpp b/io/PcdCommon.hpp
diff --git a/io/PcdHeader.cpp b/io/PcdHeader.cpp
@@ -1,37 +1,37 @@
 /******************************************************************************
-* Copyright (c) 2019, Kirk McKelvey (kirkoman@gmail.com)
-*
-* All rights reserved.
-*
-* Redistribution and use in source and binary forms, with or without
-* modification, are permitted provided that the following
-* conditions are met:
-*
-*     * Redistributions of source code must retain the above copyright
-*       notice, this list of conditions and the following disclaimer.
-*     * Redistributions in binary form must reproduce the above copyright
-*       notice, this list of conditions and the following disclaimer in
-*       the documentation and/or other materials provided
-*       with the distribution.
-*     * Neither the name of Hobu, Inc. or Flaxen Geo Consulting nor the
-*       names of its contributors may be used to endorse or promote
-*       products derived from this software without specific prior
-*       written permission.
-*
-* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
-* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
-* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
-* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
-* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
-* OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
-* AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
-* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
-* OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
-* OF SUCH DAMAGE.
-****************************************************************************/
-
+ * Copyright (c) 2019, Kirk McKelvey (kirkoman@gmail.com)
+ * Copyright (c) 2019, Bradley J Chambers (brad.chambers@gmail.com)
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following
+ * conditions are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided
+ *       with the distribution.
+ *     * Neither the name of Hobu, Inc. or Flaxen Geo Consulting nor the
+ *       names of its contributors may be used to endorse or promote
+ *       products derived from this software without specific prior
+ *       written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+ * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
+ * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
+ * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
+ * OF SUCH DAMAGE.
+ ****************************************************************************/
 
 #include "PcdHeader.hpp"
 
@@ -124,7 +124,8 @@ std::istream& operator>>(std::istream& in, PcdDataStorage& compression)
     else
     {
         compression = PcdDataStorage::unknown;
-        throw pdal_error("failed parsing PCD data storage scheme (\"" + s + "\")");
+        throw pdal_error("failed parsing PCD data storage scheme (\"" + s +
+                         "\")");
     }
     return in;
 }
@@ -152,9 +153,11 @@ std::ostream& operator<<(std::ostream& out, PcdDataStorage& compression)
 std::istream& operator>>(std::istream& in, PcdHeader& header)
 {
     std::string line;
+    size_t lines(0);
     while (!in.eof())
     {
         std::getline(in, line);
+        lines++;
         Utils::trim(line);
 
         if (line.empty() || line.substr(0, 1) == "#")
@@ -190,7 +193,8 @@ std::istream& operator>>(std::istream& in, PcdHeader& header)
             while (i != header.m_fields.end() && !line_stream.eof())
                 line_stream >> (i++)->m_size;
             if (i != header.m_fields.end() || !line_stream.eof())
-                throw pdal_error("number of SIZE values does not match number of FIELDS");
+                throw pdal_error(
+                    "number of SIZE values does not match number of FIELDS");
             continue;
         }
 
@@ -200,7 +204,8 @@ std::istream& operator>>(std::istream& in, PcdHeader& header)
             while (i != header.m_fields.end() && !line_stream.eof())
                 line_stream >> (i++)->m_type;
             if (i != header.m_fields.end() || !line_stream.eof())
-                throw pdal_error("number of TYPE values does not match number of FIELDS");
+                throw pdal_error(
+                    "number of TYPE values does not match number of FIELDS");
             continue;
         }
 
@@ -210,7 +215,8 @@ std::istream& operator>>(std::istream& in, PcdHeader& header)
             while (i != header.m_fields.end() && !line_stream.eof())
                 line_stream >> (i++)->m_count;
             if (i != header.m_fields.end() || !line_stream.eof())
-                throw pdal_error("number of COUNT values does not match number of FIELDS");
+                throw pdal_error(
+                    "number of COUNT values does not match number of FIELDS");
             continue;
         }
 
@@ -248,6 +254,7 @@ std::istream& operator>>(std::istream& in, PcdHeader& header)
         {
             line_stream >> header.m_dataStorage;
             header.m_dataOffset = in.tellg();
+            header.m_numLines = lines;
             // data starts immediately following so this needs to be last
             break;
         }
@@ -293,7 +300,8 @@ std::ostream& operator<<(std::ostream& out, PcdHeader& header)
         // TODO: noshowpoint does not seem to work
         out << "VIEWPOINT " << std::fixed << std::noshowpoint;
         out << orig.x() << " " << orig.y() << " " << orig.z() << " ";
-        out << orient.w() << " " << orient.x() << " " << orient.y() << " " << orient.z() << std::endl;
+        out << orient.w() << " " << orient.x() << " " << orient.y() << " "
+            << orient.z() << std::endl;
     }
 
     out << "POINTS " << header.m_pointCount << std::endl;
@@ -303,4 +311,122 @@ std::ostream& operator<<(std::ostream& out, PcdHeader& header)
     return out;
 }
 
+OLeStream& operator<<(OLeStream& out, PcdHeader& header)
+{
+    out.put("VERSION ", 8);
+    switch (header.m_version)
+    {
+    case PcdVersion::PCD_V6:
+        out.put("0.6", 3);
+        break;
+    case PcdVersion::PCD_V7:
+        out.put("0.7", 3);
+        break;
+    case PcdVersion::unknown:
+    default:
+        throw pdal_error("Unrecognized PcdVersion");
+    }
+    out.put("\n", 1);
+
+    out.put("FIELDS", 6);
+    for (auto i : header.m_fields)
+    {
+        out.put(" ", 1);
+        out.put(i.m_label);
+    }
+    out.put("\n", 1);
+
+    out.put("SIZE", 4);
+    for (auto i : header.m_fields)
+    {
+        out.put(" ", 1);
+        std::stringstream ss;
+        ss << i.m_size;
+        out.put(ss.str());
+    }
+    out.put("\n", 1);
+
+    out.put("TYPE", 4);
+    for (auto i : header.m_fields)
+    {
+        out.put(" ", 1);
+        switch (i.m_type)
+        {
+        case PcdFieldType::I:
+            out.put("I", 1);
+            break;
+        case PcdFieldType::U:
+            out.put("U", 1);
+            break;
+        case PcdFieldType::F:
+            out.put("F", 1);
+            break;
+        case PcdFieldType::unknown:
+        default:
+            throw pdal_error("Unrecognized PcdFieldType");
+        }
+    }
+    out.put("\n", 1);
+
+    out.put("COUNT", 5);
+    for (auto i : header.m_fields)
+    {
+        out.put(" ", 1);
+        std::stringstream ss;
+        ss << i.m_count;
+        out.put(ss.str());
+    }
+    out.put("\n", 1);
+
+    out.put("WIDTH ", 6);
+    std::stringstream ss;
+    ss << header.m_width;
+    out.put(ss.str());
+    out.put("\n", 1);
+
+    out.put("HEIGHT ", 7);
+    ss.str(std::string());
+    ss << header.m_height;
+    out.put(ss.str());
+    out.put("\n", 1);
+
+    if (header.m_version == PcdVersion::PCD_V7)
+    {
+        auto& orig = header.m_origin;
+        auto& orient = header.m_orientation;
+        // TODO: noshowpoint does not seem to work
+        out.put("VIEWPOINT ", 10);
+        ss.str(std::string());
+        ss << std::fixed << std::noshowpoint << orig.x() << " " << orig.y()
+           << " " << orig.z() << " " << orient.w() << " " << orient.x() << " "
+           << orient.y() << " " << orient.z() << "\n";
+        out.put(ss.str());
+    }
+
+    out.put("POINTS ", 7);
+    ss.str(std::string());
+    ss << header.m_pointCount;
+    out.put(ss.str());
+    out.put("\n", 1);
+
+    out.put("DATA ", 5);
+    switch (header.m_dataStorage)
+    {
+    case PcdDataStorage::ASCII:
+        out.put("ascii", 5);
+        break;
+    case PcdDataStorage::BINARY:
+        out.put("binary", 6);
+        break;
+    case PcdDataStorage::COMPRESSED:
+        out.put("compressed", 10);
+        break;
+    case PcdDataStorage::unknown:
+    default:
+        throw pdal_error("Unrecognized PcdDataStorage");
+    }
+    out.put("\n", 1);
+
+    return out;
+}
 }