KAFKA-8550: Fix plugin loading of aliased converters in Connect #6959
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@kkonstantine @rayokota could one or both of you take a look at this when you have a chance? |
|
retest this please |
|
@C0urante, it looks like the test failure is related to this PR: |
|
Thanks for the catch, @rhauch. Turns out the |
|
@rhauch it looks like the test failure on Scala 2.11 is unrelated and probably an environmental issue: |
|
Huh. I saw two green builds and one red yesterday, but now I see three red builds. I don't have access to the build reports on Jenkins so I can't see what caused them. @rhauch any thoughts on how to proceed? |
rhauch
pushed a commit
that referenced
this pull request
Aug 11, 2019
Connector validation fails if an alias is used for the converter since the validation for that is done via `ConfigDef.validateAll(...)`, which in turn invokes `Class.forName(...)` on the alias. Even though the class is successfully loaded by the DelegatingClassLoader, some Java implementations will refuse to return a class from `Class.forName(...)` whose name differs from the argument provided. This commit alters `ConfigDef.parseType(...)` to first invoke `ClassLoader.loadClass(...)` on the class using our class loader in order to get a handle on the actual class object to be loaded, then invoke `Class.forName(...)` with the fully-qualified class name of the to-be-loaded class and return the result. The invocation of `Class.forName(...)` is necessary in order to allow static initialization to take place; simply calling `ClassLoader.loadClass(...)` is insufficient. Also corrected a unit test that relied upon the old behavior. Author: Chris Egerton <chrise@confluent.io> Reviewers: Robert Yokota <rayokota@gmail.com>, Randall Hauch <rhauch@gmail.com>
rhauch
pushed a commit
that referenced
this pull request
Aug 11, 2019
Connector validation fails if an alias is used for the converter since the validation for that is done via `ConfigDef.validateAll(...)`, which in turn invokes `Class.forName(...)` on the alias. Even though the class is successfully loaded by the DelegatingClassLoader, some Java implementations will refuse to return a class from `Class.forName(...)` whose name differs from the argument provided. This commit alters `ConfigDef.parseType(...)` to first invoke `ClassLoader.loadClass(...)` on the class using our class loader in order to get a handle on the actual class object to be loaded, then invoke `Class.forName(...)` with the fully-qualified class name of the to-be-loaded class and return the result. The invocation of `Class.forName(...)` is necessary in order to allow static initialization to take place; simply calling `ClassLoader.loadClass(...)` is insufficient. Also corrected a unit test that relied upon the old behavior. Author: Chris Egerton <chrise@confluent.io> Reviewers: Robert Yokota <rayokota@gmail.com>, Randall Hauch <rhauch@gmail.com>
rhauch
pushed a commit
that referenced
this pull request
Aug 11, 2019
Connector validation fails if an alias is used for the converter since the validation for that is done via `ConfigDef.validateAll(...)`, which in turn invokes `Class.forName(...)` on the alias. Even though the class is successfully loaded by the DelegatingClassLoader, some Java implementations will refuse to return a class from `Class.forName(...)` whose name differs from the argument provided. This commit alters `ConfigDef.parseType(...)` to first invoke `ClassLoader.loadClass(...)` on the class using our class loader in order to get a handle on the actual class object to be loaded, then invoke `Class.forName(...)` with the fully-qualified class name of the to-be-loaded class and return the result. The invocation of `Class.forName(...)` is necessary in order to allow static initialization to take place; simply calling `ClassLoader.loadClass(...)` is insufficient. Also corrected a unit test that relied upon the old behavior. Author: Chris Egerton <chrise@confluent.io> Reviewers: Robert Yokota <rayokota@gmail.com>, Randall Hauch <rhauch@gmail.com>
rhauch
pushed a commit
that referenced
this pull request
Aug 11, 2019
Connector validation fails if an alias is used for the converter since the validation for that is done via `ConfigDef.validateAll(...)`, which in turn invokes `Class.forName(...)` on the alias. Even though the class is successfully loaded by the DelegatingClassLoader, some Java implementations will refuse to return a class from `Class.forName(...)` whose name differs from the argument provided. This commit alters `ConfigDef.parseType(...)` to first invoke `ClassLoader.loadClass(...)` on the class using our class loader in order to get a handle on the actual class object to be loaded, then invoke `Class.forName(...)` with the fully-qualified class name of the to-be-loaded class and return the result. The invocation of `Class.forName(...)` is necessary in order to allow static initialization to take place; simply calling `ClassLoader.loadClass(...)` is insufficient. Also corrected a unit test that relied upon the old behavior. Author: Chris Egerton <chrise@confluent.io> Reviewers: Robert Yokota <rayokota@gmail.com>, Randall Hauch <rhauch@gmail.com>
rhauch
pushed a commit
that referenced
this pull request
Aug 11, 2019
Connector validation fails if an alias is used for the converter since the validation for that is done via `ConfigDef.validateAll(...)`, which in turn invokes `Class.forName(...)` on the alias. Even though the class is successfully loaded by the DelegatingClassLoader, some Java implementations will refuse to return a class from `Class.forName(...)` whose name differs from the argument provided. This commit alters `ConfigDef.parseType(...)` to first invoke `ClassLoader.loadClass(...)` on the class using our class loader in order to get a handle on the actual class object to be loaded, then invoke `Class.forName(...)` with the fully-qualified class name of the to-be-loaded class and return the result. The invocation of `Class.forName(...)` is necessary in order to allow static initialization to take place; simply calling `ClassLoader.loadClass(...)` is insufficient. Also corrected a unit test that relied upon the old behavior. Author: Chris Egerton <chrise@confluent.io> Reviewers: Robert Yokota <rayokota@gmail.com>, Randall Hauch <rhauch@gmail.com>
rhauch
pushed a commit
that referenced
this pull request
Aug 11, 2019
Connector validation fails if an alias is used for the converter since the validation for that is done via `ConfigDef.validateAll(...)`, which in turn invokes `Class.forName(...)` on the alias. Even though the class is successfully loaded by the DelegatingClassLoader, some Java implementations will refuse to return a class from `Class.forName(...)` whose name differs from the argument provided. This commit alters `ConfigDef.parseType(...)` to first invoke `ClassLoader.loadClass(...)` on the class using our class loader in order to get a handle on the actual class object to be loaded, then invoke `Class.forName(...)` with the fully-qualified class name of the to-be-loaded class and return the result. The invocation of `Class.forName(...)` is necessary in order to allow static initialization to take place; simply calling `ClassLoader.loadClass(...)` is insufficient. Also corrected a unit test that relied upon the old behavior. Author: Chris Egerton <chrise@confluent.io> Reviewers: Robert Yokota <rayokota@gmail.com>, Randall Hauch <rhauch@gmail.com>
|
The failures were likely unrelated. I ran a build locally with multiple JDKs, and the only failures I had were unrelated. |
|
restest this please |
xiowu0
pushed a commit
to linkedin/kafka
that referenced
this pull request
Aug 22, 2019
…converters in Connect (apache#6959) TICKET = KAFKA-8550 LI_DESCRIPTION = EXIT_CRITERIA = HASH [04cf8fb] ORIGINAL_DESCRIPTION = Connector validation fails if an alias is used for the converter since the validation for that is done via `ConfigDef.validateAll(...)`, which in turn invokes `Class.forName(...)` on the alias. Even though the class is successfully loaded by the DelegatingClassLoader, some Java implementations will refuse to return a class from `Class.forName(...)` whose name differs from the argument provided. This commit alters `ConfigDef.parseType(...)` to first invoke `ClassLoader.loadClass(...)` on the class using our class loader in order to get a handle on the actual class object to be loaded, then invoke `Class.forName(...)` with the fully-qualified class name of the to-be-loaded class and return the result. The invocation of `Class.forName(...)` is necessary in order to allow static initialization to take place; simply calling `ClassLoader.loadClass(...)` is insufficient. Also corrected a unit test that relied upon the old behavior. Author: Chris Egerton <chrise@confluent.io> Reviewers: Robert Yokota <rayokota@gmail.com>, Randall Hauch <rhauch@gmail.com> (cherry picked from commit 04cf8fb)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Jira
Summary of issue: connector validation fails if an alias is used for the converter since the validation for that is done via
ConfigDef.validateAll(...), which in turn invokesClass.forName(...)on the alias. Even though the class is successfully loaded by theDelegatingClassLoader, some Java implementations will refuse to return a class fromClass.forName(...)whose name differs than the argument provided.Summary of fix: alter
ConfigDef.parseType(...)to first invokeClassLoader.loadClass(...)on the class in order to get a handle on the actual class object to be loaded, then invokeClass.forName(...)with the fully-qualified class name of the to-be-loaded class and return the result. The invocation ofClass.forName(...)is necessary in order to allow static initialization to take place; simply callingClassLoader.loadClass(...)is insufficient.Summary of testing: tested manually on trunk. Added unit test to
ConfigDefTestthat simulates the plugin-aliasing behavior of theDelegatingClassLoaderand then invokesConfigDef.parseTypeon an aliased class; this test fails on the current trunk but succeeds with the changes proposed here.This should be backported at least to 2.0; it's likely the issue goes back further than that but since it's been around for so long and has yet to be noted by anyone else it doesn't seem worth the effort to backport that much further if there are any significant merge conflicts.
Committer Checklist (excluded from commit message)