Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[SPARK-5569] [STREAMING] fix ObjectInputStreamWithLoader for supporting load array classes. #8955
When use Kafka DirectStream API to create checkpoint and restore saved checkpoint when restart,
The reason for this error is that ObjectInputStreamWithLoader extends the ObjectInputStream class and override its resolveClass method. But Instead of Using Class.forName(desc,false,loader), Spark uses loader.loadClass(desc) to instance the class, which do not works with array class.
details of the difference between Class.forName and loader.loadClass can be found here.