Incorrect filter interpretation

[gurisko]

my environment running ArangoDB

I'm using the latest ArangoDB of the respective release series:

• 2.8
• 3.0
• 3.1
• 3.2
• self-compiled devel branch

Mode:

• Cluster
• Single-Server

Storage-Engine:

• mmfiles
• rocksdb

On this operating system:

• DCOS on
  • AWS
  • Azure
  • own infrastructure
• Linux
  • Debian .deb
  • Ubuntu .deb
  • SUSE .rpm
  • RedHat .rpm
  • Fedora .rpm
  • Gentoo
  • docker - official docker library
  • other:
• Windows, version:
• MacOS, version:

this is an AQL-related issue:

• I'm using graph features

I'm issuing AQL via:

• web interface with this browser: running on this OS:
• arangosh
• this Driver:

I've run db._explain("<my aql query>") and it didn't shed more light on this.
The AQL query in question is:

Query string:
     FOR dev IN vDevice
       FOR site IN vSite
         LET $site = site._id 
         FILTER site._id == dev.ad
           && dev.configReg != null
           && LENGTH(dev.configReg) > 0
           && !(dev.configReg == "0x2102" || dev.configReg == "0x2101" || dev.configReg == "0xf")
         RETURN dev
 

Execution plan:
 Id   NodeType                  Est.   Comment
  1   SingletonNode                1   * ROOT
  3   EnumerateCollectionNode     30     - FOR site IN vSite   /* full collection scan */
  8   IndexNode                  420       - FOR dev IN vDevice   /* hash index scan */
  9   CalculationNode            420         - LET #3 = ((dev.`configReg` == "0x2102") && (LENGTH(dev.`configReg`) > 0))   /* simple expression */   /* collections used: dev : vDevice */
  6   FilterNode                 420         - FILTER #3
  7   ReturnNode                 420         - RETURN dev

Indexes used:
 By   Type   Collection   Unique   Sparse   Selectivity   Fields     Ranges
  8   hash   vDevice      false    false         6.79 %   [ `ad` ]   (site.`_id` == dev.`ad`)

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   remove-unnecessary-calculations
  3   interchange-adjacent-enumerations
  4   use-indexes
  5   remove-filter-covered-by-index

I can forward you dataset via email.

I am aware of how to rewrite the query to make it work.
One odd solution is provided below (5th line of the query):

Query string:
     FOR dev IN vDevice
       FOR site IN vSite
         LET $site = site._id 
         FILTER site._id == dev.ad
         FILTER dev.configReg != null
           && LENGTH(dev.configReg) > 0
           && !(dev.configReg == "0x2102" || dev.configReg == "0x2101" || dev.configReg == "0xf")
         RETURN dev
 

Execution plan:
 Id   NodeType                  Est.   Comment
  1   SingletonNode                1   * ROOT
  2   EnumerateCollectionNode    383     - FOR dev IN vDevice   /* full collection scan */
  7   CalculationNode            383       - LET #5 = (((dev.`configReg` != null) && (LENGTH(dev.`configReg`) > 0)) && ! (((dev.`configReg` == "0x2102") || (dev.`configReg` == "0x2101")) || (dev.`configReg` == "0xf")))   /* simple expression */   /* collections used: dev : vDevice */
  8   FilterNode                 383       - FILTER #5
 10   IndexNode                  383       - FOR site IN vSite   /* primary index scan */
  9   ReturnNode                 383         - RETURN dev

Indexes used:
 By   Type      Collection   Unique   Sparse   Selectivity   Fields       Ranges
 10   primary   vSite        true     false       100.00 %   [ `_key` ]   (site.`_id` == dev.`ad`)

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   move-filters-up
  3   remove-unnecessary-calculations
  4   use-indexes
  5   remove-filter-covered-by-index
  6   remove-unnecessary-calculations-2

[jsteemann]

Potential fix:

diff --git a/arangod/Aql/Ast.cpp b/arangod/Aql/Ast.cpp
index 726bceca4e..d63a89177c 100644
--- a/arangod/Aql/Ast.cpp
+++ b/arangod/Aql/Ast.cpp
@@ -2509,6 +2509,10 @@ AstNode* Ast::optimizeUnaryOperatorArithmetic(AstNode* node) {
 /// the unary NOT operation will be replaced with the result of the operation
 AstNode* Ast::optimizeNotExpression(AstNode* node) {
   TRI_ASSERT(node != nullptr);
+  if (node->type != NODE_TYPE_OPERATOR_UNARY_NOT) {
+    return node;
+  }
+
   TRI_ASSERT(node->type == NODE_TYPE_OPERATOR_UNARY_NOT);
   TRI_ASSERT(node->numMembers() == 1);
 
diff --git a/arangod/Aql/Condition.cpp b/arangod/Aql/Condition.cpp
index a2400265d5..63443abb1b 100644
--- a/arangod/Aql/Condition.cpp
+++ b/arangod/Aql/Condition.cpp
@@ -1386,12 +1386,11 @@ AstNode* Condition::transformNode(AstNode* node) {
         auto negated = transformNode(_ast->createNodeUnaryOperator(
             NODE_TYPE_OPERATOR_UNARY_NOT, sub->getMemberUnchecked(i)));
         auto optimized = _ast->optimizeNotExpression(negated);
-
         newOperator->addMember(optimized);
       }
 
-      return newOperator;
-    }
+      return transformNode(newOperator);
+    } 
 
     node->changeMember(0, transformNode(sub));
   }

I still need to confirm the fix with some test cases.

[jsteemann]

Fixed in 3.2.2

[dothebart]

ArangoDB 3.2.2 containing this bugfix is available for download - thanks for reporting.

Closing.