Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix task context fails with non string property value #16612

Merged
merged 2 commits into from
Jun 16, 2024

Conversation

maytasm
Copy link
Contributor

@maytasm maytasm commented Jun 15, 2024

Fix task context fails with non string property value

Description

This is a regression caused by #14239
The above PR introduces a new method addSystemProperty with String type in the method definition for the property value. (Previously, we were using StringUtils.format which takes in Object as argument). The problem is that task.getContextValue(propName) has a return type which is a generic type parameter ContextValueType. This means the method can return a value of any type specified at the time of invocation. When the value of the context property is not a String, Java is still trying to call the addSystemProperty with String type causing the task to fail.
i.e. if we have something like

  "context": {
    "druid.indexer.fork.property.druid.processing.numThreads": 4
  }

we'll get the error:

java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
	at org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:319) [druid-indexing-service-31.0.0-SNAPSHOT.jar:31.0.0-SNAPSHOT]
	at org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:171) [druid-indexing-service-31.0.0-SNAPSHOT.jar:31.0.0-SNAPSHOT]
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131) [guava-32.0.1-jre.jar:?]
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:75) [guava-32.0.1-jre.jar:?]
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82) [guava-32.0.1-jre.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_322]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_322]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_322]

This PR fix the issue by casting the return value from task.getContextValue(propName) to Object and then use addSystemProperty with Object type for the value parameter.

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@kfaraz
Copy link
Contributor

kfaraz commented Jun 15, 2024

@maytasm , this PR https://github.com/apache/druid/pull/16281/files also fixes the same issue.
Maybe we can just merge that existing PR instead? The original author has not responded on that PR, so we can proceed with this one.

@@ -924,6 +924,11 @@ public CommandListBuilder addSystemProperty(String property, boolean value)
return addSystemProperty(property, String.valueOf(value));
}

public CommandListBuilder addSystemProperty(String property, Object value)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new method feels unnecessary.
Maybe we can use the approach used in #16281 instead.

Since the original author has not responded on that PR, we can proceed with this one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@maytasm
Copy link
Contributor Author

maytasm commented Jun 15, 2024

@maytasm , this PR https://github.com/apache/druid/pull/16281/files also fixes the same issue. Maybe we can just merge that existing PR instead? The original author has not responded on that PR, so we can proceed with this one.

SGTM. Closing

@maytasm maytasm closed this Jun 15, 2024
@maytasm maytasm reopened this Jun 15, 2024
@maytasm
Copy link
Contributor Author

maytasm commented Jun 15, 2024

@kfaraz Oh. I misread your comment. Will update this PR to use the same approach used in #16281

@maytasm
Copy link
Contributor Author

maytasm commented Jun 15, 2024

@kfaraz Updated the PR with the approach used in #16281 and clean up some stuff. Can you please review? Thanks!

Copy link
Contributor

@kfaraz kfaraz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@kfaraz kfaraz merged commit d6c7d86 into apache:master Jun 16, 2024
87 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants