Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fixed avro schema decode error in functions #6662

Merged
merged 60 commits into from
Apr 5, 2020
Merged

Fixed avro schema decode error in functions #6662

merged 60 commits into from
Apr 5, 2020

Conversation

gaoran10
Copy link
Contributor

@gaoran10 gaoran10 commented Apr 3, 2020 •
edited
Loading

Fixes #5503
From #6445

Motivation

In functions, it will encounter ClassCastException when using the Avro schema for topics.

Exception in thread "main" java.lang.ClassCastException: org.apache.pulsar.shade.org.apache.avro.generic.GenericData$Record cannot be cast to io.streamnative.KeyValueSchemaTest$Foo2
	at io.streamnative.KeyValueSchemaTest.testConsumerByPythonProduce(KeyValueSchemaTest.java:412)
	at io.streamnative.KeyValueSchemaTest.main(KeyValueSchemaTest.java:305)

Modifications

In functions, when using Avro schema specific the ClassLoader for ReflectDatumReader.

Add integration test testAvroSchemaFunction in class PulsarFunctionsTest.

@gaoran10 gaoran10 mentioned this pull request Apr 3, 2020
Closed
@gaoran10
Copy link
Contributor Author

gaoran10 commented Apr 3, 2020

/pulsarbot run-failure-checks

4 similar comments
@gaoran10
Copy link
Contributor Author

gaoran10 commented Apr 3, 2020

/pulsarbot run-failure-checks

@jiazhai
Copy link
Member

jiazhai commented Apr 4, 2020

/pulsarbot run-failure-checks

@gaoran10
Copy link
Contributor Author

gaoran10 commented Apr 4, 2020

/pulsarbot run-failure-checks

@codelipenghui
Copy link
Contributor

/pulsarbot run-failure-checks

@tuteng tuteng added area/client type/bug The PR fixed a bug or issue reported a bug labels Apr 4, 2020
@tuteng tuteng added this to the 2.6.0 milestone Apr 4, 2020
@yjshen
Copy link
Member

yjshen commented Apr 4, 2020

[ERROR] An error occurred while instantiating class org.apache.pulsar.functions.worker.request.ServiceRequestManagerTest: Cannot create a new instance of test class class org.apache.pulsar.functions.worker.request.ServiceRequestManagerTest
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
[ERROR]
[ERROR] An error occurred while instantiating class org.apache.pulsar.functions.worker.request.ServiceRequestManagerTest: Cannot create a new instance of test class class org.apache.pulsar.functions.worker.request.ServiceRequestManagerTest
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:657)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:283)
[ERROR] at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:246)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1161)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1002)
[ERROR] at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:848)
[ERROR] at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)

@gaoran10 gaoran10 force-pushed the issue/function-avro-schema-decode-error branch from 2997045 to 5a90f71 Compare April 4, 2020 12:12
@jiazhai
Copy link
Member

jiazhai commented Apr 4, 2020

/pulsarbot run-failure-checks

@gaoran10
Copy link
Contributor Author

gaoran10 commented Apr 4, 2020

/pulsarbot run-failure-checks

@codelipenghui codelipenghui merged commit 52ae182 into apache:master Apr 5, 2020
tuteng pushed a commit that referenced this pull request Apr 6, 2020
@gaoran10 @tuteng
Fixed avro schema decode error in functions (#6662)
0d16f77 
Fixes #5503
From #6445

In functions, it will encounter ```ClassCastException``` when using the Avro schema for topics.

```
Exception in thread "main" java.lang.ClassCastException: org.apache.pulsar.shade.org.apache.avro.generic.GenericData$Record cannot be cast to io.streamnative.KeyValueSchemaTest$Foo2
	at io.streamnative.KeyValueSchemaTest.testConsumerByPythonProduce(KeyValueSchemaTest.java:412)
	at io.streamnative.KeyValueSchemaTest.main(KeyValueSchemaTest.java:305)
```

In functions, when using Avro schema specific the ClassLoader for ReflectDatumReader.

Add integration test ```testAvroSchemaFunction``` in class ```PulsarFunctionsTest```.

(cherry picked from commit 52ae182)

Handle conflict
tuteng pushed a commit that referenced this pull request Apr 13, 2020
@gaoran10 @tuteng
Fixed avro schema decode error in functions (#6662)
d60eebf 
Fixes #5503
From #6445

In functions, it will encounter ```ClassCastException``` when using the Avro schema for topics.

```
Exception in thread "main" java.lang.ClassCastException: org.apache.pulsar.shade.org.apache.avro.generic.GenericData$Record cannot be cast to io.streamnative.KeyValueSchemaTest$Foo2
	at io.streamnative.KeyValueSchemaTest.testConsumerByPythonProduce(KeyValueSchemaTest.java:412)
	at io.streamnative.KeyValueSchemaTest.main(KeyValueSchemaTest.java:305)
```

In functions, when using Avro schema specific the ClassLoader for ReflectDatumReader.

Add integration test ```testAvroSchemaFunction``` in class ```PulsarFunctionsTest```.

(cherry picked from commit 52ae182)

Handle conflict
@gaoran10 gaoran10 deleted the issue/function-avro-schema-decode-error branch April 25, 2020 05:17
jiazhai pushed a commit to jiazhai/pulsar that referenced this pull request May 18, 2020
@gaoran10 @jiazhai
Fixed avro schema decode error in functions (apache#6662)
ff57921 
Fixes apache#5503
From apache#6445

In functions, it will encounter ```ClassCastException``` when using the Avro schema for topics.

```
Exception in thread "main" java.lang.ClassCastException: org.apache.pulsar.shade.org.apache.avro.generic.GenericData$Record cannot be cast to io.streamnative.KeyValueSchemaTest$Foo2
	at io.streamnative.KeyValueSchemaTest.testConsumerByPythonProduce(KeyValueSchemaTest.java:412)
	at io.streamnative.KeyValueSchemaTest.main(KeyValueSchemaTest.java:305)
```

In functions, when using Avro schema specific the ClassLoader for ReflectDatumReader.

Add integration test ```testAvroSchemaFunction``` in class ```PulsarFunctionsTest```.
(cherry picked from commit 52ae182)
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Aug 24, 2020
@gaoran10 @huangdx0726
Fixed avro schema decode error in functions (apache#6662)
c631f44 
Fixes apache#5503 
From apache#6445 

# Motivation
In functions, it will encounter ```ClassCastException``` when using the Avro schema for topics.

```
Exception in thread "main" java.lang.ClassCastException: org.apache.pulsar.shade.org.apache.avro.generic.GenericData$Record cannot be cast to io.streamnative.KeyValueSchemaTest$Foo2
	at io.streamnative.KeyValueSchemaTest.testConsumerByPythonProduce(KeyValueSchemaTest.java:412)
	at io.streamnative.KeyValueSchemaTest.main(KeyValueSchemaTest.java:305)
```

# Modifications
In functions, when using Avro schema specific the ClassLoader for ReflectDatumReader.

Add integration test ```testAvroSchemaFunction``` in class ```PulsarFunctionsTest```.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jiazhai jiazhai jiazhai approved these changes

@codelipenghui codelipenghui codelipenghui approved these changes

@sijie sijie Awaiting requested review from sijie

Assignees

@gaoran10 gaoran10

Labels
area/client release/2.5.1 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Milestone
2.6.0
Development

Successfully merging this pull request may close these issues.

[Schema]GenericData$Record cannot be cast
6 participants
@gaoran10 @jiazhai @codelipenghui @yjshen @tuteng @sijie