Query Fails on Upsert with Replace_nth

[arango-db-us3r]

My Environment

• ArangoDB Version: 3.7.8
• Storage Engine: RocksDB
• Deployment Mode: Single Server
• Deployment Strategy: ArangoDB Starter in Docker
• Configuration: No extra configuration
• Infrastructure: own
• Operating System: Ubuntu 20.04
• Total RAM in your machine: 16GB
• Disks in use: SSD
• Used Package: Docker - official Docker library

Component, Query & Data

Affected feature:
AQL query using web interface and python driver

AQL query (if applicable):

let values = [
  [
    "t1",
    0,
    0
  ], [
    "t1",
    1,
    0
  ]
]


for value_set in values
let t = value_set[0]
let index = value_set[1]
let value = value_set[2]


UPSERT {}
INSERT {v: { [t] : []}}
UPDATE { 
    'v' : { [ t ] : REPLACE_NTH(OLD.v[t], index, value, value)}
    
}
IN temp
return [OLD, NEW]

AQL explain and/or profile (if applicable):

Query String (317 chars, cacheable: false):
 let values = [
   [
     "t1",
     0,
     0
   ], [
     "t1",
     1,
     0
   ]
 ]
 for value_set in values
 let t = value_set[0]
 let index = value_set[1]
 let value = value_set[2]
 UPSERT {}
 INSERT {v: { [t] : []}}
 UPDATE { 
     'v' : { [ t ] : REPLACE_NTH(OLD.v[t], index, value, value)}
     
 }
 IN temp
 return [OLD, NEW]

Execution plan:
 Id   NodeType                  Est.   Comment
  1   SingletonNode                1   * ROOT
  2   CalculationNode              1     - LET values = [ [ "t1", 0, 0 ], [ "t1", 1, 0 ] ]   /* json expression */   /* const assignment */
  4   EnumerateListNode            2     - FOR value_set IN values   /* list iteration */
 19   SubqueryStartNode            2       - LET #8 = ( /* subquery begin */
  9   EnumerateCollectionNode      0         - FOR #6 IN temp   /* full collection scan */
 10   LimitNode                    0           - LIMIT 0, 1
 20   SubqueryEndNode              2           - RETURN  #6 ) /* subquery end */
  5   CalculationNode              2       - LET t = value_set[0]   /* simple expression */
  7   CalculationNode              2       - LET value = value_set[2]   /* simple expression */
 14   CalculationNode              2       - LET #13 = { "v" : { [ t ] : [ ] } }   /* simple expression */
 13   CalculationNode              2       - LET $OLD = #8[0]   /* simple expression */
 15   CalculationNode              2       - LET #15 = { "v" : { [ t ] : REPLACE_NTH($OLD.`v`[t], value_set[1], value, value) } }   /* simple expression */
 16   UpsertNode                   2       - UPSERT $OLD INSERT #13 UPDATE #15 IN temp
 17   CalculationNode              2       - LET #17 = [ $OLD, $NEW ]   /* simple expression */
 18   ReturnNode                   2       - RETURN #17

Indexes used:
 By   Name      Type      Collection   Unique   Sparse   Selectivity   Fields       Ranges
 16   primary   primary   temp         true     false       100.00 %   [ `_key` ]   $OLD

Functions used:
 Name          Deterministic   Cacheable   Uses V8
 REPLACE_NTH   true            true        false  

Optimization rules applied:
 Id   RuleName
  1   move-calculations-up
  2   remove-redundant-calculations
  3   remove-unnecessary-calculations
  4   move-calculations-up-2
  5   remove-data-modification-out-variables
  6   move-calculations-down
  7   splice-subqueries

Write query options:
 Option                   Value
 waitForSync              false
 skipDocumentValidation   false
 keepNull                 true
 mergeObjects             true
 ignoreRevs               true
 isRestore                false
 ignoreErrors             false
 ignoreDocumentNotFound   false
 readCompleteInput        false
 consultAqlWriteFilter    false
 exclusive                false
 nullMeansRemove          false

Dataset:
Dataset temp is newly created and empty

Size of your Dataset on disk:
0B

Replication Factor & Number of Shards (Cluster only):
N/A

Steps to reproduce

1. Run query

Problem:
Query does not finish, giving the generic error: Query: Expecting Array slice (while executing) (exception location: /work/ArangoDB/arangod/RestHandler/RestCursorHandler.cpp:320). Please report this error to arangodb.com

Expected result:
Query to finish without error

[jsteemann]

Using latest 3.7, the "Expecting Array slice" exception comes from here:

#0  0x00007ffff7a4a762 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x0000555555af3fef in arangodb::velocypack::Slice::byteSize (this=<optimized out>, start=<optimized out>)
    at /home/jsteemann/Arango37NoAsan/3rdParty/velocypack/include/velocypack/Slice.h:1272
#2  arangodb::velocypack::Slice::byteSize (this=<optimized out>) at /home/jsteemann/Arango37NoAsan/3rdParty/velocypack/include/velocypack/Slice.h:875
#3  arangodb::velocypack::Slice::arrayLength (this=<optimized out>) at /home/jsteemann/Arango37NoAsan/3rdParty/velocypack/include/velocypack/Slice.h:1041
#4  arangodb::velocypack::ArrayIterator::ArrayIterator (this=<optimized out>, slice=...) at /home/jsteemann/Arango37NoAsan/3rdParty/velocypack/include/velocypack/Iterator.h:61
#5  0x00005555560e5f54 in arangodb::aql::Functions::ReplaceNth (expressionContext=<optimized out>, trx=0x7fffac98aa98, parameters=...)
    at /home/jsteemann/Arango37NoAsan/arangod/Aql/Functions.cpp:7350
#6  0x00005555564141c5 in arangodb::aql::Expression::executeSimpleExpressionFCallCxx (this=0x7fffac961260, node=<optimized out>, trx=0x7fffac98aa98, mustDestroy=@0x7fffaf3f8940: false)
    at /home/jsteemann/Arango37NoAsan/arangod/Aql/Expression.cpp:870
#7  0x00005555564132b5 in arangodb::aql::Expression::executeSimpleExpressionObject (this=0x7fffac961260, node=0x7fffac9bf380, trx=0x7fffac98aa98, mustDestroy=@0x7fffaf3f8b10: false)
    at /home/jsteemann/Arango37NoAsan/arangod/Aql/Expression.cpp:732
#8  0x00005555564132b5 in arangodb::aql::Expression::executeSimpleExpressionObject (this=0x7fffac961260, node=0x7fffac9bf100, trx=0x7fffac98aa98, mustDestroy=@0x7fffaf3f8c2f: false)
    at /home/jsteemann/Arango37NoAsan/arangod/Aql/Expression.cpp:732
#9  0x00005555566d0872 in arangodb::aql::CalculationExecutor<(arangodb::aql::CalculationType)0>::doEvaluation (this=0x7fffac98aa98, input=..., output=...)
    at /home/jsteemann/Arango37NoAsan/arangod/Aql/CalculationExecutor.cpp:176

[jsteemann]

Bugfix patch:

diff --git a/arangod/Aql/Functions.cpp b/arangod/Aql/Functions.cpp
index 9ed344028c..c5db143d79 100644
--- a/arangod/Aql/Functions.cpp
+++ b/arangod/Aql/Functions.cpp
@@ -7340,9 +7340,10 @@ AqlValue Functions::ReplaceNth(ExpressionContext* expressionContext,
     THROW_ARANGO_EXCEPTION_PARAMS(TRI_ERROR_QUERY_FUNCTION_ARGUMENT_TYPE_MISMATCH, AFN);
   }
 
-  AqlValueMaterializer materializer(trx);
-  VPackSlice arraySlice = materializer.slice(baseArray, false);
-  VPackSlice replaceValue = materializer.slice(newValue, false);
+  AqlValueMaterializer materializer1(trx);
+  VPackSlice arraySlice = materializer1.slice(baseArray, false);
+  AqlValueMaterializer materializer2(trx);
+  VPackSlice replaceValue = materializer2.slice(newValue, false);
 
   transaction::BuilderLeaser builder(trx);
   builder->openArray();
@@ -7359,6 +7360,7 @@ AqlValue Functions::ReplaceNth(ExpressionContext* expressionContext,
 
   uint64_t pos = length;
   if (replaceOffset >= length) {
+    AqlValueMaterializer materializer(trx);
     VPackSlice paddVpValue = materializer.slice(paddValue, false);
     while (pos < replaceOffset) {
       builder->add(paddVpValue);

[jsteemann]

@arango-db-us3r : Fixed via the above PRs. The bugfix is to supposed to appear in release 3.7.10.

[maxkernbach]

Hi @arango-db-us3r,

Version 3.7.10, which includes a fix for this issue, has been released.

Please let us know whether your query works as expected.

[arango-db-us3r]

Yes, this fix allowed my query to work as expected, thank you.