Add ExpressionFilter for MongoDB-style JSON filtering.

doc/stages/filters.expression.rst

@@ -0,0 +1,138 @@
+.. _filters.expression:
+
+filters.expression
+==================
+
+.. contents::
+
+The expression filter applies query logic to the input point cloud based on a
+MongoDB-style query expression using the point cloud attributes.
+
+.. embed::
+
+.. streamable::
+
+Pipeline Example
+----------------
+
+This example passes through only the points whose Classification is non-zero.
+
+.. code-block:: json
+
+    [
+        "input.las",
+        {
+            "type": "filters.expression",
+            "expression": {
+                "Classification": { "$ne": 0 }
+            }
+        },
+        "filtered.las"
+    ]
+
+Options
+-------
+
+expression
+    A JSON query :ref:`expression` containing a combination of query comparisons
+    and logical operators.
+
+
+.. _expression:
+
+Expression
+--------------------------------------------------------------------------------
+
+A query expression is a combination of comparison and logical operators that
+define a query which can be used to select matching points by their attribute
+values.
+
+Comparison operators
+................................................................................
+
+There are 8 valid query comparison operators:
+    - ``$eq``: Matches values equal to a specified value.
+    - ``$gt``: Matches values greater than a specified value.
+    - ``$gte``: Matches values greater than or equal to a specified value.
+    - ``$lt``: Matches values less than a specified value.
+    - ``$lte``: Matches values less than or equal to a specified value.
+    - ``$ne``: Matches values not equal to a specified value.
+    - ``$in``: Matches any of the values specified in the array.
+    - ``$nin``: Matches none of the values specified in the array.
+
+Comparison operators compare a point cloud attribute with a value or an array
+of values.  For all comparison operators except for ``$in`` and ``$nin``, the
+value must be a number.  For ``$in`` and ``$nin``, the value must be an array
+of numbers.
+
+Comparison operators are applied directly to attribute values, and thus must be
+contained within an attribute selection by which an attribute is selected by its
+name.  For example:
+
+.. code-block:: json
+
+    { "Classification": { "$eq": 2 } }
+
+.. code-block:: json
+
+    { "Intensity": { "$gt": 0 } }
+
+.. code-block:: json
+
+    { "Classification": { "$in": [2, 6, 9] } }
+
+The ``$eq`` comparison operator may be implicitly invoked by setting an
+attribute name directly to a value.
+
+.. code-block:: json
+
+    { "Classification": 2 }
+
+Logical operators
+................................................................................
+
+There are 4 valid logical operators:
+    - ``$and``: Applies a logical **and** on the expressions of the array and
+      returns a match only if all expressions match.
+    - ``$not``: Inverts the value of the single sub-expression.
+    - ``$nor``: Applies a logical **nor** on the expressions of the array and
+      returns a match only if all expressions fail to match.
+    - ``$nor``: Applies a logical **or** on the expressions of the array and
+      returns a match if any of the expressions match.
+
+Logical operators are used to logically combine sub-expressions.  All logical
+operators except for ``$not`` are applied to arrays of expressions.
+``$not`` is applied to a single expression and negates its result.
+
+Logical operators may be applied directly to comparison expressions or may
+contain further nested logical operators.  For example:
+
+.. code-block:: json
+
+    { "$or": [
+        { "Classification": 2 },
+        { "Intensity": { "$gt": 0 } }
+    ] }
+
+.. code-block:: json
+
+    { "$or": [
+        { "Classification": 2 },
+        { "$and": [
+            { "ReturnNumber": { "$gt": 0 } },
+            { "Z": { "$lte": 42 } }
+        ] }
+    ] }
+
+.. code-block:: json
+
+    { "$not": {
+        "$or": [
+            { "Classification": 2 },
+            { "$and": [
+                { "ReturnNumber": { "$gt": 0 } },
+                { "Z": { "$lte": 42 } }
+            ] }
+        ] }
+    }
+

filters/ExpressionFilter.cpp

@@ -0,0 +1,102 @@
+/******************************************************************************
+ * Copyright (c) 2018, Connor Manning (connor@hobu.co)
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following
+ * conditions are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided
+ *       with the distribution.
+ *     * Neither the name of Hobu, Inc. or Flaxen Geo Consulting nor the
+ *       names of its contributors may be used to endorse or promote
+ *       products derived from this software without specific prior
+ *       written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+ * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
+ * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
+ * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
+ * OF SUCH DAMAGE.
+ ****************************************************************************/
+
+#include "ExpressionFilter.hpp"
+
+#include "private/expression/Expression.hpp"
+
+namespace pdal
+{
+
+static const StaticPluginInfo s_info
+{
+    "filters.expression",
+    "Pass only points that pass a logic filter.",
+    "http://pdal.io/stages/filters.logic.html"
+};
+
+CREATE_STATIC_STAGE(ExpressionFilter, s_info);
+
+std::string ExpressionFilter::getName() const
+{
+    return s_info.name;
+}
+
+ExpressionFilter::ExpressionFilter()
+{}
+
+ExpressionFilter::~ExpressionFilter()
+{}
+
+void ExpressionFilter::addArgs(ProgramArgs& args)
+{
+    args.add("expression", "Logical query expression", m_json).setPositional();
+}
+
+void ExpressionFilter::prepared(PointTableRef table)
+{
+    log()->get(LogLevel::Debug) << "Building expression from: " << m_json <<
+        std::endl;
+
+    m_expression = makeUnique<Expression>(*table.layout(), m_json);
+
+    log()->get(LogLevel::Debug) << "Built expression: " << *m_expression <<
+        std::endl;
+}
+
+PointViewSet ExpressionFilter::run(PointViewPtr inView)
+{
+    PointViewSet views;
+    PointViewPtr view(inView->makeNew());
+
+    for (PointId i(0); i < inView->size(); ++i)
+    {
+        PointRef pr(inView->point(i));
+        if (processOne(pr))
+        {
+            view->appendPoint(*inView, i);
+        }
+    }
+
+    views.insert(view);
+    return views;
+}
+
+bool ExpressionFilter::processOne(PointRef& pr)
+{
+    return m_expression->check(pr);
+}
+
+} // namespace pdal
+

filters/ExpressionFilter.hpp

@@ -0,0 +1,66 @@
+/******************************************************************************
+ * Copyright (c) 2018, Connor Manning (connor@hobu.co)
+ *
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following
+ * conditions are met:
+ *
+ *     * Redistributions of source code must retain the above copyright
+ *       notice, this list of conditions and the following disclaimer.
+ *     * Redistributions in binary form must reproduce the above copyright
+ *       notice, this list of conditions and the following disclaimer in
+ *       the documentation and/or other materials provided
+ *       with the distribution.
+ *     * Neither the name of Hobu, Inc. or Flaxen Geo Consulting nor the
+ *       names of its contributors may be used to endorse or promote
+ *       products derived from this software without specific prior
+ *       written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
+ * COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
+ * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
+ * OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
+ * AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY
+ * OF SUCH DAMAGE.
+ ****************************************************************************/
+
+#pragma once
+
+#include <json/json.h>
+
+#include <pdal/Filter.hpp>
+#include <pdal/Streamable.hpp>
+
+namespace pdal
+{
+
+class Expression;
+
+class PDAL_DLL ExpressionFilter : public Filter, public Streamable
+{
+public:
+    ExpressionFilter();
+    ~ExpressionFilter();
+
+    std::string getName() const override;
+    virtual bool processOne(PointRef& point) override;
+
+private:
+    virtual void addArgs(ProgramArgs& args) override;
+    virtual void prepared(PointTableRef table) override;
+    virtual PointViewSet run(PointViewPtr view) override;
+
+    Json::Value m_json;
+    std::unique_ptr<Expression> m_expression;
+};
+
+} // namespace pdal
+