Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add @Description to UDFs registered in Apache Hive #66

Merged
merged 2 commits into from
Feb 22, 2023

Conversation

okumin
Copy link
Contributor

@okumin okumin commented Feb 21, 2023

I observe many WARN logs when using Hive 4. Please feel free to ask me to update if my description doesn't make sense.

pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,535  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.cpc.UnionSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,538  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.hll.UnionSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,540  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.IntersectSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,541  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.EstimateSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,541  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.ExcludeSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,542  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.UnionSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,542  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.tuple.ArrayOfDoublesSketchToValuesUDTF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.

@okumin okumin force-pushed the udf-descriptions-warning branch from ed1f875 to 53d0249 Compare February 21, 2023 13:00
value = "_FUNC_(firstSketch, secondSketch[, lgK[, seed]]) - Compute the union of the given "
+ "sketches with the given size and seed",
extended = "The return value is a binary blob that can be operated on by other sketch related functions."
+ " The lgK parameter controls the sketch size and rlative error expected from the sketch."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo "rlative"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I fixed all typos. Please let me know if you prefer to have separate PRs.
e3bafbe

$ git grep rlative
src/main/java/org/apache/datasketches/hive/cpc/DataToSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/cpc/UnionSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/cpc/UnionSketchUDF.java:        + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/hll/DataToSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/hll/UnionSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/hll/UnionSketchUDF.java:        + " The lgK parameter controls the sketch size and rlative error expected from the sketch."

value = "_FUNC_(firstSketch, secondSketch[, lgK[, type]]) - Compute the union of the given "
+ "sketches with the given size and seed",
extended = "The return value is a binary blob that can be operated on by other sketch related functions."
+ " The lgK parameter controls the sketch size and rlative error expected from the sketch."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same typo

Copy link
Contributor

@AlexanderSaydakov AlexanderSaydakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build seems to be broken, but I am inclined to ignore that for now since there are no code changes in this request

@AlexanderSaydakov AlexanderSaydakov merged commit 50ef98e into apache:master Feb 22, 2023
@okumin okumin deleted the udf-descriptions-warning branch February 23, 2023 02:41
@okumin
Copy link
Contributor Author

okumin commented Feb 23, 2023

Thanks for your review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants