Skip to content
This repository has been archived by the owner on Sep 20, 2022. It is now read-only.

[HIVEMALL-138-2] refactored to_ordered_map & to_ordered_list UDAF #114

Closed
wants to merge 7 commits into from

Conversation

myui
Copy link
Member

@myui myui commented Sep 12, 2017

What changes were proposed in this pull request?

This PR applies refactoring to #108

What type of PR is it?

[Feature | Refactoring]

What is the Jira issue?

https://issues.apache.org/jira/browse/HIVEMALL-138

How was this patch tested?

unit tests, manual tests

How to use this feature?

see #108

Checklist

  • Did you apply source code formatter, i.e., mvn formatter:format, for your commit?

@myui
Copy link
Member Author

myui commented Sep 12, 2017

@takuti applied some refactoring to #108

@myui
Copy link
Member Author

myui commented Sep 12, 2017

FAILED: IllegalArgumentException Size requested for unknown type: hivemall.tools.list.UDAFToOrderedList$UDAFToOrderedListEvaluator$AbstractQueueHandler

failed in Hive v2.3.0 because UDAF estimate() is lacking.

@myui
Copy link
Member Author

myui commented Sep 12, 2017

Ah... It was Hive v2.3.0 bug and not related to estimate().

https://issues.apache.org/jira/browse/HIVE-17406

@myui
Copy link
Member Author

myui commented Sep 12, 2017

oops... I forget that default constructor is mandatory in UDAF.

Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: hivemall.tools.map.UDAFToOrderedMap$DescendingMapEvaluator.<init>()
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:134)
        at org.apache.hadoop.hive.ql.plan.AggregationDesc.getGenericUDAFEvaluator(AggregationDesc.java:92)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:350)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:366)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:556)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:508)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
        at org.apache.hadoop.hive.ql.exec.MapOperator.initializeMapOperator(MapOperator.java:501)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:317)
        ... 15 more
Caused by: java.lang.NoSuchMethodException: hivemall.tools.map.UDAFToOrderedMap$DescendingMapEvaluator.<init>()
        at java.lang.Class.getConstructor0(Class.java:3082)
        at java.lang.Class.getDeclaredConstructor(Class.java:2178)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
        ... 26 more

@asfgit asfgit closed this in 688daa5 Sep 13, 2017
@myui
Copy link
Member Author

myui commented Sep 13, 2017

@takuti reverted some changes by me and merged.

@coveralls
Copy link

coveralls commented Sep 13, 2017

Coverage Status

Coverage increased (+0.4%) to 40.617% when pulling df39bd5 on myui:HIVEMALL-138-2 into 3804789 on apache:master.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants