Adding the PropertyNamingStrategies from jackson for fixing hadoop ingestion#14671
Conversation
|
Just to add a bit more colour here. Hadoop has support for classloader isolation that follows a child-first class loading mechanism. There are jackson classes present on Hadoop server and there are jackson classes present in the libs that druid uploads. The aws sdk that is part of druid lib, wants to load PropertyNamingStrategies but it is not available in the child classloader so that class gets loaded through the parent classloader. The parent classloader would also load the PropertyNamingStrategy referenced by the PropertyNamingStrategies. All is well so far. But then the child classloader needs to load PropertyNamingStrategy later which it is able to load. But because it has also been loaded through the parent classloader, we get a linkage error. Somewhat explained here - http://frankkieviet.blogspot.com/2009/03/javalanglinkageerror-loader-constraint.html |
893c60e to
a86b692
Compare
|
@pranavbhole - code inspection jobs are failing. |
6ac3d23 to
922388a
Compare
922388a to
cf2d63f
Compare
|
@abhishekagarwal87 pushed the fix. |
…gestion
Fixes #XXXX.
Description
We found that aws jdk core is using the jackson version 2.12.7 and it is not compatible with older jackson esp inside hadoop container as it loads the classes from given extension. We found that s3 extension is loading S3 required jars and instantiating the PropertyNamingStrategies from newer jackson and failing.
Creating the PR to pull this class into druid repo itself to fix the hadoop ingestion and make it compatible with newer jackson.
Fixed the bug ...
Renamed the class ...
Added a forbidden-apis entry ...
Release note
Key changed/added classes in this PR
MyFooOurBarTheirBazThis PR has: