Skip to content

Conversation

@uncleGen
Copy link

@uncleGen uncleGen commented Feb 8, 2023

What changes were proposed in this pull request?

Query will fail when use collect_list (or collect_set) and disable map-side aggregationg:

Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryMap cannot be cast to java.util.Map
        at org.apache.hadoop.hive.serde2.objectinspector.StandardMapObjectInspector.getMap(StandardMapObjectInspector.java:85)
        at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:437)
        at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:362)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.putIntoCollection(GenericUDAFMkCollectionEvaluator.java:154)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMkCollectionEvaluator.iterate(GenericUDAFMkCollectionEvaluator.java:120)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:192)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:638)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.processAggr(GroupByOperator.java:877)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:721)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:787)

To reproduce this issue:

create table tb1 (a int, b string, c string);
insert into tb1 values (1, "100", "101");
insert into tb1 values (1, "102", "103");
insert into tb1 values (2, "200", "201");
set hive.map.aggr=false;
select a, collect_list(map("b",b,"c",c)) as col1 from tb1 group by a;
select a, collect_set(array(b, c)) as col1 from tb1 group by a;

Why are the changes needed?

Wrong object inspector will be created when use collect_list and disable map-side aggregation.

Does this PR introduce any user-facing change?

No

How was this patch tested?

UT and manual test

@uncleGen uncleGen changed the title [HIVE-27059] Wrong object inspector will be created when use collect_list and disable map-side aggregation HIVE-27059: Wrong object inspector will be created when use collect_list and disable map-side aggregation Feb 8, 2023
@uncleGen uncleGen closed this Feb 8, 2023
@uncleGen uncleGen reopened this Feb 8, 2023
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the dev@hive.apache.org list if the patch is in need of reviews.

@github-actions github-actions bot added the stale label May 22, 2023
@github-actions github-actions bot closed this May 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants