fix: NPE when udf metrics enabled #5960
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fixes: confluentinc#5890 Fixes an NPE thrown when the second instance of a UDAF is initialized. The NPE was being thrown due to a bug in the code that failed to initialise the sensor fields in the `UdafAggregateFunction` class if the sensor already existed. The commit also refactors the code used to add function invocation metrics such that all three function types: UDF, UDTF and UDAF, use the same function to register metrics. Unit tests added to ensure this single code path doesn't throw an NPE or return null on the subsequent invocation. Also added `ksql.udf.collect.metrics` metric to the server config file, so users are more likely to be able to find it and try it out. Moved UDTF functions out of the `ksql-udf` group and into `ksql-udtf` group. Having them in the former was another bug.
agavra
approved these changes
Aug 6, 2020
Comment on lines
74
to
79
| final Sensor existing = metrics.getSensor(sensorName); | ||
| if (existing != null) { | ||
| return existing; | ||
| } | ||
|
|
||
| final Sensor newSensor = metrics.sensor(sensorName); |
There was a problem hiding this comment.
there's a race here that is possible here where two calls for the same metrics.getSensor(sensorName) don't find the sensor and they both end up registering the extra metrics. I don't know of a good way around it...
There was a problem hiding this comment.
Yeah I know, but I was just fixing the NPE and being lazy ;). Fixed it.
| @@ -67,7 +68,8 @@ public void loadUdtfFromClass( | |||
| } | |||
| final String functionName = udtfDescriptionAnnotation.name(); | |||
| final String sensorName = "ksql-udtf-" + functionName; | |||
| FunctionLoaderUtils.addSensor(sensorName, functionName, metrics); | |||
|
|
|||
| FunctionMetrics.initInvocationSensor(metrics, sensorName, "ksql-udtf", functionName + " udtf"); | |||
There was a problem hiding this comment.
if anyone was relying on the old metrics, their alerts for udtfs will break with this change. probably ok as I'd be shocked if anyone was actually writing their own UDTFs and using metrics on them at this point but something to call out 😉
There was a problem hiding this comment.
Yeah, I'm aware, but ... meh. turning metrics on caused an NPE, so likely no one is using them, and better to change it now.
There was a problem hiding this comment.
As mentioned in the description, I see this as a bug fix.
sarwarbhuiyan
pushed a commit
to sarwarbhuiyan/ksql
that referenced
this pull request
Sep 4, 2020
* fix: NPE when udf metrics enabled fixes: confluentinc#5890 Fixes an NPE thrown when the second instance of a UDAF is initialized. The NPE was being thrown due to a bug in the code that failed to initialise the sensor fields in the `UdafAggregateFunction` class if the sensor already existed. The commit also refactors the code used to add function invocation metrics such that all three function types: UDF, UDTF and UDAF, use the same function to register metrics. Unit tests added to ensure this single code path doesn't throw an NPE or return null on the subsequent invocation. Also added `ksql.udf.collect.metrics` metric to the server config file, so users are more likely to be able to find it and try it out. Moved UDTF functions out of the `ksql-udf` group and into `ksql-udtf` group. Having them in the former was another bug. Co-authored-by: Andy Coates <big-andy-coates@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
fixes: #5890
Fixes an NPE thrown when the second instance of a UDAF is initialized. The NPE was being thrown due to a bug in the code that failed to initialise the sensor fields in the
UdafAggregateFunctionclass if the sensor already existed.The commit also refactors the code used to add function invocation metrics such that all three function types: UDF, UDTF and UDAF, use the same function to register metrics.
Unit tests added to ensure this single code path doesn't throw an NPE or return null on the subsequent invocation.
Also added
ksql.udf.collect.metricsmetric to the server config file, so users are more likely to be able to find it and try it out.Moved UDTF functions out of the
ksql-udfgroup and intoksql-udtfgroup. Having them in the former was another bug.Testing done
Added unit tests plus manual validation that metric names didn't change with new code, (except intentional move of UDTF metrics into own group).
Reviewer checklist