Always register X/Y/Z as doubles (#2932)

* Always create XYZ as doubles. * Fix EPT reader now that XYZ are forced to be doubles (#2926) * Initial commit of DBSCAN clustering * Begin playing around with n-dimensional KDIndex * Finishing KDFlexIndex * Initialize bbox properly * Implicit deletes and move ctor to CPP * Remove the explicit KDNormalIndex as it can be replicated with KDFlexIndex * Move KDFlexIndex from DBSCANFilter to KDIndex * Add note about dimensions in the docs * Default dimensions to X,Y,Z * Clarification on dimensions when setting min_points * Initial commit of DBSCAN clustering * Add a new n-dimensional index called KDFlexIndex to KDIndex * Add a test for the ELM filter (#2869) * Add simple test for ELM filter. * Add a second test. * bump version to 2.1.0 to eliminate confusion with 2.0.0 builds when using master * Python filter doc (#2879) * Add a warning to remember numpy needs to be installed * Use appropriate key for file name * Fix some code indent issues * NitfWrap: Fix compilation error 'pdal::Log::Log(const string&, const string&, bool) is private within this context' (#2877) * Gdal utils doc (#2884) * Update docs for GDALUtils. Fix leaks and such in getPolygons(). * Move dialect declaration. * Register ClusterID dimension in filters.cluster (#2887) * Make ClusterID a built-in dimension. * Make sure cluster dimension gets registered. * Support passing a reference to a .py script to readers.numpy along with function/module/args options that reference a function that returns a Numpy array (#2867) * support passing a reference to a .py script to readers.numpy along with function/module/args options that reference a function that returns a Numpy array * make sure to add nlohman includes * address review comments in #2867 * increment PDAL version to 2.1 to avoid collision * remove unused reference to nlohman * [WIP] Add little description on transformation filter (#2747) * add graphics and first description * fix styling * fix styling * add description for more tranformations * fix wrong order for z-axis rotation, add more description about figures * fix typo * refine text * refine text * Add versionadded * Force X/Y/Z to 64 bit. (#2893) * require C++17 #2890 (#2896) * remove SQLite drivers #2646 (#2897) * remove SQLite drivers #2646 * remove CMake for sqlite * update to next OSX azp image (#2901) * Fix compilation on newer clang. * RDB windows build tweaks (#2909) * tiledb and e57 config * rdb building on windows * rdb win64.bat * move Python to https://github.com/PDAL/python (#2895) * move Python to https://github.com/PDAL/python * adjust azp scripts for python removal * remove travis python config * remove more python cruft * Don't do memcpy (#2910) * Don't do memcpy. * Pair is value-initialized (0'd). No need to memset. * Gcc9.2 fixes (#2913) * GCC 9.2 fixes. * Explicitly delete undefined assignment operators so that we actually generate copy ctors that we need. * Eigen fixes for GCC 9.2. * Implement a Linux build based on Conda with a pinned GDAL dependency (#2911) * add Linux/Conda build * fix job name * conda junk * set uid/gid * set uid:gid * conda user stuff again * more usermod * clobber /opt/conda perms * clobber user to vsts_azpcontainer * use azp conda * fix syntax * remove manual env setting * conda init * LD_FLAGS * no pin * another swing * selectively shut off tiledb tests with BUILD_TILEDB_TESTS * try pinning gdal * use correct GDAL pin * Add reader, now, and file_size keys to pdal info output (#2900) * scratch * add now, file_size, and reader to 'pdal info' output * add debug lint * update test output * output comparison information when we fail a pdal info test * fix up comparison test * Simplify E57 build and fix warnings. (#2914) * Refined wording of transformation filter documentation (#2915) * pdal-config on windows * Windows updates (#2917) * Updates for VS 2017 * Use constexpr function call for Uuid size. * make AzP/Windows use GDAL 3 (#2918) * E57 changes for GCC 9.2. * add AzP/docs build (#2919) * add AzP/docs build * try uploading doc builds to s3 * rejigger variables * remove extraneous ninja command * variable settings * variable quoting * more variable churn * more variable churns * one more time * azp variable churning some more * build PDF too * output URLs * condition doc upload for only PullRequest * yaml is whitespace sensitive * --quiet aws sync * try var subst again * quiet file upload * try quoting var * reenable the rest of CI * Revert #2890 – use C++11 as standard instead of C++17 * Check for empty PointView in ELM filter (#2924) * Add method to EPT's FixedPointLayout to allow it to contain non-double XYZs to match a remote buffer layout. Co-authored-by: chambbj <brad.chambers@gmail.com> Co-authored-by: Andrew Bell <andrew.bell.ia@gmail.com> Co-authored-by: Howard Butler <howard@hobu.co> Co-authored-by: Jules Robichaud-Gagnon <j.robichaudg@gmail.com> Co-authored-by: Nick M Cummins <nickmcummins@gmail.com> Co-authored-by: vvmruder <vvmruder@users.noreply.github.com> Co-authored-by: Connor Manning <connor@hobu.co> Co-authored-by: chambbj <brad.chambers@gmail.com> Co-authored-by: Howard Butler <howard@hobu.co> Co-authored-by: Jules Robichaud-Gagnon <j.robichaudg@gmail.com> Co-authored-by: Nick M Cummins <nickmcummins@gmail.com> Co-authored-by: vvmruder <vvmruder@users.noreply.github.com>
PDAL · Feb 13, 2020 · b40acc7 · b40acc7
1 parent 5f4e56a
commit b40acc7
Show file tree

Hide file tree

Showing 9 changed files with 90 additions and 91 deletions.
diff --git a/io/BpfReader.cpp b/io/BpfReader.cpp
@@ -200,14 +200,9 @@ void BpfReader::addDimensions(PointLayoutPtr layout)
 {
     for (size_t i = 0; i < m_dims.size(); ++i)
     {
-        Dimension::Type type = Dimension::Type::Float;
-
         BpfDimension& dim = m_dims[i];
-        if (dim.m_label == "X" ||
-            dim.m_label == "Y" ||
-            dim.m_label == "Z")
-            type = Dimension::Type::Double;
-        dim.m_id = layout->registerOrAssignDim(dim.m_label, type);
+        dim.m_id = layout->registerOrAssignDim(dim.m_label,
+            Dimension::Type::Float);
     }
 }
 

diff --git a/io/EptReader.cpp b/io/EptReader.cpp
@@ -483,7 +483,7 @@ void EptReader::addDimensions(PointLayoutPtr layout)
             Dimension::interpretationName(coercedType) << std::endl;
 
         layout->registerOrAssignDim(name, coercedType);
-        m_remoteLayout->registerOrAssignDim(name, remoteType);
+        m_remoteLayout->registerOrAssignFixedDim(name, remoteType);
     }
 
     m_remoteLayout->finalize();

diff --git a/io/MemoryViewReader.cpp b/io/MemoryViewReader.cpp
@@ -107,12 +107,7 @@ void MemoryViewReader::pushField(const Field& f)
 void MemoryViewReader::addDimensions(PointLayoutPtr layout)
 {
     for (auto& f : m_fields)
-    {
-        Dimension::Type type = f.m_type;
-        if (f.m_name == "X" || f.m_name == "Y" || f.m_name == "Z")
-            type = Dimension::Type::Double;
-        f.m_id = layout->registerOrAssignDim(f.m_name, type);
-    }
+        f.m_id = layout->registerOrAssignDim(f.m_name, f.m_type);
 }
 
 

diff --git a/io/private/EptSupport.cpp b/io/private/EptSupport.cpp
@@ -168,5 +168,25 @@ void Pool::work()
     }
 }
 
+void FixedPointLayout::registerFixedDim(const Dimension::Id id,
+    const Dimension::Type type)
+{
+    Dimension::Detail dd = m_detail[Utils::toNative(id)];
+    dd.setType(type);
+    update(dd, Dimension::name(id));
+}
+
+Dimension::Id FixedPointLayout::registerOrAssignFixedDim(const std::string name,
+    const Dimension::Type type)
+{
+    Dimension::Id id = Dimension::id(name);
+    if (id != Dimension::Id::Unknown)
+    {
+        registerFixedDim(id, type);
+        return id;
+    }
+    return assignDim(name, type);
+}
+
 } // namespace pdal
 
diff --git a/io/private/EptSupport.hpp b/io/private/EptSupport.hpp
@@ -284,7 +284,15 @@ class FixedPointLayout : public PointLayout
     // The default PointLayout class may reorder dimension entries for packing
     // efficiency.  However if a PointLayout is intended to be mapped to data
     // coming from a remote source, then the dimensions must retain their order.
-    // FixedPointLayout retains the order of dimensions as they are registered.
+    // FixedPointLayout retains the order of dimensions as they are registered,
+    // as well as adding a custom assignment function that makes sure XYZ are
+    // registered as their appropriate remote type rather than always as
+    // doubles.
+public:
+    void registerFixedDim(Dimension::Id id, Dimension::Type type);
+    Dimension::Id registerOrAssignFixedDim(std::string name,
+        Dimension::Type type);
+
 protected:
     PDAL_DLL virtual bool update(
             pdal::Dimension::Detail dimDetail,

diff --git a/pdal/DbReader.cpp b/pdal/DbReader.cpp
@@ -48,12 +48,10 @@ void DbReader::loadSchema(PointLayoutPtr layout, const XMLSchema& schema)
     m_layout = layout;
     m_dims = schema.xmlDims();
 
-    // Override XYZ to doubles and use those going forward
-    // we will apply any scaling set before handing it off
-    // to PDAL.
-    layout->registerDim(Dimension::Id::X);
-    layout->registerDim(Dimension::Id::Y);
-    layout->registerDim(Dimension::Id::Z);
+    // Always register X, Y and Z.  We will apply any scaling set
+    // before handing it off to PDAL.
+    using namespace Dimension;
+    layout->registerDims( {Id::X, Id::Y, Id::Z} );
 
     m_orientation = schema.orientation();
     m_packedPointSize = 0;

diff --git a/pdal/PointLayout.cpp b/pdal/PointLayout.cpp
@@ -83,8 +83,16 @@ void PointLayout::registerDim(Dimension::Id id)
 
 void PointLayout::registerDim(Dimension::Id id, Dimension::Type type)
 {
-    Dimension::Detail dd = m_detail[Utils::toNative(id)];
-    dd.setType(resolveType(type, dd.type()));
+    using namespace Dimension;
+
+    Detail dd = m_detail[Utils::toNative(id)];
+
+    // Force X, Y and Z to always be doubles.
+    if (id == Id::X || id == Id::Y || id == Id::Z)
+        type = Type::Double;
+    else
+        type = resolveType(type, dd.type());
+    dd.setType(type);
     update(dd, Dimension::name(id));
 }
 

diff --git a/plugins/mrsid/io/MrsidReader.cpp b/plugins/mrsid/io/MrsidReader.cpp
@@ -277,45 +277,20 @@ point_count_t MrsidReader::read(PointViewPtr view, point_count_t count)
             if (Utils::iequals(name, "X") &&
                 m_pointInfo.hasChannel(CHANNEL_NAME_X))
             {
-                if (t == Type::Double)
-                {
-                    view->setField<double>( d,
-                                            pointIndex,
-                                            getData<double>(points, CHANNEL_NAME_X, pointIndex));
-                } else if (t == Type::Signed32)
-                {
-                    view->setField<int32_t>(d,
-                                            pointIndex,
-                                            getData<int32_t>(points, CHANNEL_NAME_X, pointIndex));
-                }
-            } else if (Utils::iequals(name, "Y") &&
-                       m_pointInfo.hasChannel(CHANNEL_NAME_Y))
+                view->setField(d, pointIndex,
+                    getData<double>(points, CHANNEL_NAME_X, pointIndex));
+            }
+            else if (Utils::iequals(name, "Y") &&
+                m_pointInfo.hasChannel(CHANNEL_NAME_Y))
             {
-                if (t == Type::Double)
-                {
-                    view->setField<double>( d,
-                                            pointIndex,
-                                            getData<double>(points, CHANNEL_NAME_Y, pointIndex));
-                } else if (t == Type::Signed32)
-                {
-                    view->setField<int32_t>(d,
-                                            pointIndex,
-                                            getData<int32_t>(points, CHANNEL_NAME_Y, pointIndex));
-                }
-            } else if (Utils::iequals(name, "Z") &&
-                       m_pointInfo.hasChannel(CHANNEL_NAME_Z))
+                view->setField(d, pointIndex,
+                    getData<double>(points, CHANNEL_NAME_Y, pointIndex));
+            }
+            else if (Utils::iequals(name, "Z") &&
+                m_pointInfo.hasChannel(CHANNEL_NAME_Z))
             {
-                if (t == Type::Double)
-                {
-                    view->setField<double>( d,
-                                            pointIndex,
-                                            getData<double>(points, CHANNEL_NAME_Z, pointIndex));
-                } else if (t == Type::Signed32)
-                {
-                    view->setField<int32_t>(d,
-                                            pointIndex,
-                                            getData<int32_t>(points, CHANNEL_NAME_Z, pointIndex));
-                }
+                view->setField(d, pointIndex,
+                    getData<double>(points, CHANNEL_NAME_Z, pointIndex));
             }
 #ifdef CHANNEL_NAME_GPSTime
             else if (Utils::iequals(name, "GpsTime") &&

diff --git a/test/unit/PointTableTest.cpp b/test/unit/PointTableTest.cpp
@@ -62,47 +62,47 @@ TEST(PointTable, resolveType)
     EXPECT_EQ(layout->dimType(Id::X), Type::Double);
 
     /// Build as we go.
-    layout->registerDim(Id::Y, Type::Unsigned8);
-    EXPECT_EQ(layout->dimSize(Id::Y), 1u);
-    EXPECT_EQ(layout->dimType(Id::Y), Type::Unsigned8);
+    layout->registerDim(Id::Intensity, Type::Unsigned8);
+    EXPECT_EQ(layout->dimSize(Id::Intensity), 1u);
+    EXPECT_EQ(layout->dimType(Id::Intensity), Type::Unsigned8);
 
-    layout->registerDim(Id::Y, Type::Unsigned8);
-    EXPECT_EQ(layout->dimSize(Id::Y), 1u);
-    EXPECT_EQ(layout->dimType(Id::Y), Type::Unsigned8);
+    layout->registerDim(Id::Intensity, Type::Unsigned8);
+    EXPECT_EQ(layout->dimSize(Id::Intensity), 1u);
+    EXPECT_EQ(layout->dimType(Id::Intensity), Type::Unsigned8);
 
-    layout->registerDim(Id::Y, Type::Signed8);
-    // Signed 8 and Unsigned 8 should yeild signed 16.
-    EXPECT_EQ(layout->dimSize(Id::Y), 2u);
-    EXPECT_EQ(layout->dimType(Id::Y), Type::Signed16);
+    layout->registerDim(Id::Intensity, Type::Signed8);
+    // Signed 8 and Unsigned 8 should yield signed 16.
+    EXPECT_EQ(layout->dimSize(Id::Intensity), 2u);
+    EXPECT_EQ(layout->dimType(Id::Intensity), Type::Signed16);
 
-    layout->registerDim(Id::Y, Type::Signed16);
-    EXPECT_EQ(layout->dimSize(Id::Y), 2u);
-    EXPECT_EQ(layout->dimType(Id::Y), Type::Signed16);
+    layout->registerDim(Id::Intensity, Type::Signed16);
+    EXPECT_EQ(layout->dimSize(Id::Intensity), 2u);
+    EXPECT_EQ(layout->dimType(Id::Intensity), Type::Signed16);
 
-    layout->registerDim(Id::Y, Type::Float);
-    EXPECT_EQ(layout->dimSize(Id::Y), 4u);
-    EXPECT_EQ(layout->dimType(Id::Y), Type::Float);
+    layout->registerDim(Id::Intensity, Type::Float);
+    EXPECT_EQ(layout->dimSize(Id::Intensity), 4u);
+    EXPECT_EQ(layout->dimType(Id::Intensity), Type::Float);
 
-    layout->registerDim(Id::Y, Type::Double);
-    EXPECT_EQ(layout->dimSize(Id::Y), 8u);
-    EXPECT_EQ(layout->dimType(Id::Y), Type::Double);
+    layout->registerDim(Id::Intensity, Type::Double);
+    EXPECT_EQ(layout->dimSize(Id::Intensity), 8u);
+    EXPECT_EQ(layout->dimType(Id::Intensity), Type::Double);
 
     ///
-    layout->registerDim(Id::Z, Type::Unsigned16);
-    EXPECT_EQ(layout->dimSize(Id::Z), 2u);
-    EXPECT_EQ(layout->dimType(Id::Z), Type::Unsigned16);
+    layout->registerDim(Id::Red, Type::Unsigned16);
+    EXPECT_EQ(layout->dimSize(Id::Red), 2u);
+    EXPECT_EQ(layout->dimType(Id::Red), Type::Unsigned16);
 
-    layout->registerDim(Id::Z, Type::Signed8);
-    EXPECT_EQ(layout->dimSize(Id::Z), 4u);
-    EXPECT_EQ(layout->dimType(Id::Z), Type::Signed32);
+    layout->registerDim(Id::Red, Type::Signed8);
+    EXPECT_EQ(layout->dimSize(Id::Red), 4u);
+    EXPECT_EQ(layout->dimType(Id::Red), Type::Signed32);
 
-    layout->registerDim(Id::Z, Type::Signed16);
-    EXPECT_EQ(layout->dimSize(Id::Z), 4u);
-    EXPECT_EQ(layout->dimType(Id::Z), Type::Signed32);
+    layout->registerDim(Id::Red, Type::Signed16);
+    EXPECT_EQ(layout->dimSize(Id::Red), 4u);
+    EXPECT_EQ(layout->dimType(Id::Red), Type::Signed32);
 
-    layout->registerDim(Id::Z, Type::Double);
-    EXPECT_EQ(layout->dimSize(Id::Z), 8u);
-    EXPECT_EQ(layout->dimType(Id::Z), Type::Double);
+    layout->registerDim(Id::Red, Type::Double);
+    EXPECT_EQ(layout->dimSize(Id::Red), 8u);
+    EXPECT_EQ(layout->dimType(Id::Red), Type::Double);
 }
 
 TEST(PointTable, userView)