Projection bug in the 3.8 series for projections that end on the same name for the path

[HarryCordewener]

My Environment

• ArangoDB Version: 3.8.1 (Did not exist in 3.7.2)
• Storage Engine: RocksDB
• Deployment Mode: Single Server
• Deployment Strategy: Manual Start
• Configuration: Standard windows install
• Infrastructure: own
• Operating System: Windows 10
• Total RAM in your machine: Irrelevant
• Disks in use: Irrelevant
• Used Package: https://download.arangodb.com/arangodb38/Community/Windows/ArangoDB3-3.8.1_win64.exe

Component, Query & Data

Affected feature:
AQL (general)

AQL query (if applicable):
The Error:

FOR t in schema
RETURN {a: t.fields.c, b: t.columns.c}

Curious way to avoid the bug:

FOR t in schema
RETURN {a: t.fields.c, b: t.columns.c, c: t}

Proof that this is because of c existing at the end of two projections:

FOR t in schema
RETURN {a: t.fields.d, b: t.columns.c}

AQL explain and/or profile (if applicable):

Query String (54 chars, cacheable: true):
 FOR t in schema
 RETURN {a: t.fields.c, b: t.columns.c}

Execution plan:
 Id   NodeType                  Est.   Comment
  1   SingletonNode                1   * ROOT
  2   EnumerateCollectionNode      2     - FOR t IN schema   /* full collection scan, projections: `columns` */
  3   CalculationNode              2       - LET #1 = { "a" : t.`fields`.`c`, "b" : t.`columns`.`c` }   /* simple expression */   /* collections used: t : schema */
  4   ReturnNode                   2       - RETURN #1

Indexes used:
 none

Optimization rules applied:
 Id   RuleName
  1   reduce-extraction-to-projection

Optimization rules with highest execution times:
 RuleName                                    Duration [s]
 reduce-extraction-to-projection                  0.00001
 use-indexes                                      0.00001
 simplify-conditions                              0.00000
 optimize-count                                   0.00000
 replace-function-with-index                      0.00000

41 rule(s) executed, 1 plan(s) created

Dataset:

INSERT { "fields": { "c": "schema" }, "columns": { "c": "schema" } } INTO schema
INSERT { "fields": { "d": "schema" }, "columns": { "c": "schema" } } INTO schema

Size of your Dataset on disk: Irrelevant

Replication Factor & Number of Shards (Cluster only): Irrelevant

Steps to reproduce

1. Create the data set
2. Run the queries to see the results
3. Note that because there are two projection paths which end in '.c', one of them returns null.
4. Note that when another projection is added, it circumvents this bug.
5. Note that when the projection does not have two items that end in '.c', this bug does not occur.

Problem:
When there are two projection paths ending in, for example, .c, one ends up returning null.

Expected result:
For the projection to return correctly.

[HarryCordewener]

Call out to Kapil Pipaliya on the ArangoDB Community Slack for being the one to actually spot this bug.

[kapilpipaliya]

Thanks for creating the issue. Its also happening on devel branch.

FOR t IN schema
  RETURN { table: t.columns.c, form: t.fields.c }

returns null values for form keys.

[Simran-B]

This seems to be the same problem as in #14672, caused by a bad optimization. As a workaround, you can disable the optimizer rule reduce-extraction-to-projection, e.g. in arangosh:

db._query(`FOR doc IN coll RETURN [doc.foo.attr, doc.bar.attr]`, null, { optimizer: { rules: ["-reduce-extraction-to-projection"] } })

[jsteemann]

Fixed via #14801