Skip to content

Add @Description to UDFs registered in Apache Hive#66

Merged
AlexanderSaydakov merged 2 commits into
apache:masterfrom
okumin:udf-descriptions-warning
Feb 22, 2023
Merged

Add @Description to UDFs registered in Apache Hive#66
AlexanderSaydakov merged 2 commits into
apache:masterfrom
okumin:udf-descriptions-warning

Conversation

@okumin
Copy link
Copy Markdown
Contributor

@okumin okumin commented Feb 21, 2023

I observe many WARN logs when using Hive 4. Please feel free to ask me to update if my description doesn't make sense.

pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,535  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.cpc.UnionSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,538  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.hll.UnionSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,540  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.IntersectSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,541  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.EstimateSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,541  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.ExcludeSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,542  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.theta.UnionSketchUDF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.
pod/hive-hiveserver2-84dbfbc99c-d7sll: 2023-02-19T11:05:51,542  WARN [main] exec.FunctionRegistry: UDF Class org.apache.hive.org.apache.datasketches.hive.tuple.ArrayOfDoublesSketchToValuesUDTF does not have description. Please annotate the class with the org.apache.hadoop.hive.ql.exec.Description annotation and provide the description of the function.

@okumin okumin force-pushed the udf-descriptions-warning branch from ed1f875 to 53d0249 Compare February 21, 2023 13:00
value = "_FUNC_(firstSketch, secondSketch[, lgK[, seed]]) - Compute the union of the given "
+ "sketches with the given size and seed",
extended = "The return value is a binary blob that can be operated on by other sketch related functions."
+ " The lgK parameter controls the sketch size and rlative error expected from the sketch."
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo "rlative"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I fixed all typos. Please let me know if you prefer to have separate PRs.
e3bafbe

$ git grep rlative
src/main/java/org/apache/datasketches/hive/cpc/DataToSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/cpc/UnionSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/cpc/UnionSketchUDF.java:        + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/hll/DataToSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/hll/UnionSketchUDAF.java:    + " The lgK parameter controls the sketch size and rlative error expected from the sketch."
src/main/java/org/apache/datasketches/hive/hll/UnionSketchUDF.java:        + " The lgK parameter controls the sketch size and rlative error expected from the sketch."

value = "_FUNC_(firstSketch, secondSketch[, lgK[, type]]) - Compute the union of the given "
+ "sketches with the given size and seed",
extended = "The return value is a binary blob that can be operated on by other sketch related functions."
+ " The lgK parameter controls the sketch size and rlative error expected from the sketch."
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same typo

Copy link
Copy Markdown
Contributor

@AlexanderSaydakov AlexanderSaydakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build seems to be broken, but I am inclined to ignore that for now since there are no code changes in this request

@AlexanderSaydakov AlexanderSaydakov merged commit 50ef98e into apache:master Feb 22, 2023
@okumin okumin deleted the udf-descriptions-warning branch February 23, 2023 02:41
@okumin
Copy link
Copy Markdown
Contributor Author

okumin commented Feb 23, 2023

Thanks for your review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants