Skip to content

HIVE-28408: Support ARRAY field access in CBO - #5577

Merged
ramesh0201 merged 1 commit into
apache:masterfrom
ramesh0201:HIVE-28408
Jan 14, 2025
Merged

ramesh0201 merged 1 commit into
apache:masterfrom
ramesh0201:HIVE-28408

Conversation

@ramesh0201

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

We need to support array field access in the CBO. This is currently already supported in the execution side. Only CBO fails with an error. So we will be by passing the CBO by doing no-op in this case to leave it to hive compiler and execution engine to do rest of the work

Why are the changes needed?

In order to enable CBO for all the queries

Does this PR introduce any user-facing change?

No

Is the change a dependency upgrade?

No

How was this patch tested?

mvn test -Dtest=TestMiniLlapLocalCliDriver -Dqfile=nested_column_pruning.q

Comment thread common/src/java/org/apache/hadoop/hive/conf/HiveConf.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/parse/type/RexNodeExprFactory.java Outdated
Comment thread ql/src/test/results/clientpositive/llap/vector_orc_nested_column_pruning.q.out Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/parse/type/RexNodeExprFactory.java Outdated
Comment thread ql/src/java/org/apache/hadoop/hive/ql/parse/type/RexNodeExprFactory.java Outdated
@zabetak

zabetak commented Jan 6, 2025

Copy link
Copy Markdown
Member

It would be also nice to include a qfile with an EXPLAIN CBO statement that shows that we can treat ARRAY field access in CBO. Possibly the book example that I added under HIVE-28408 is a good fit.

@ramesh0201

Copy link
Copy Markdown
Contributor Author

@zabetak Can you please review the latest patch? I have addressed most of the comments, except the two that have I left a comment above.

@zabetak zabetak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for addressing the comments @ramesh0201 !

@sonarqubecloud

Copy link
Copy Markdown

@ramesh0201
ramesh0201 merged commit 91e0f1a into apache:master Jan 14, 2025
@ramesh0201

Copy link
Copy Markdown
Contributor Author

Thank you for the review @okumin and @zabetak

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants