Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix: Cast exception occurs if function/source/sink type is ByteBuffer #11611

Merged
merged 1 commit into from
Aug 10, 2021
Merged

Fix: Cast exception occurs if function/source/sink type is ByteBuffer #11611

merged 1 commit into from
Aug 10, 2021

Conversation

jerrypeng
Copy link
Contributor

Motivation

If a Pulsar Function/Source/Sink is of type ByteBuffer, an cast exception will occur:

6:44:00.226 INFO  [external-repl-prop/io/PulsarSink-test-0] o.a.p.f.i.JavaInstanceRunnable@357 - Encountered exception in sink write: 
java.lang.ClassCastException: class [B cannot be cast to class java.nio.ByteBuffer ([B and java.nio.ByteBuffer are in module java.base of loader 'bootstrap')
	at org.apache.pulsar.functions.worker.PulsarFunctionLocalRunTest$StatsNullSink.write(PulsarFunctionLocalRunTest.java:1027) ~[test-classes/:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:355) [classes/:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.handleResult(JavaInstanceRunnable.java:338) [classes/:?]
	at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:283) [classes/:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]

The is because if the function is of type ByteBuffer, the schema to used in the consumer is BYTES, which is not correct

Modifications

Use the correct ByteBuffer schema when the type of the function/source/sink is ByteBuffer

Verifying this change

Added a test

Documentation

No changes necessary

For committer

For this PR, do we need to update docs?

No

@jerrypeng jerrypeng added type/bug The PR fixed a bug or issue reported a bug area/function area/connector labels Aug 10, 2021
@jerrypeng jerrypeng added this to the 2.9.0 milestone Aug 10, 2021
@jerrypeng jerrypeng self-assigned this Aug 10, 2021
@jerrypeng jerrypeng force-pushed the fix_pulsar_functions_byte_buffer branch from 1c1464e to 766381f Compare August 10, 2021 00:04
@jerrypeng jerrypeng merged commit b7e027b into apache:master Aug 10, 2021
hangc0276 pushed a commit that referenced this pull request Aug 12, 2021
@jerrypeng @hangc0276
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
33c9fb7 
…#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit b7e027b)
@hangc0276 hangc0276 added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Aug 12, 2021
jerrypeng added a commit to jerrypeng/incubator-pulsar that referenced this pull request Aug 18, 2021
@jerrypeng
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
c3a5b8b 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
jerrypeng added a commit to jerrypeng/incubator-pulsar that referenced this pull request Nov 4, 2021
@jerrypeng
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
b0b7465 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
dlg99 pushed a commit to dlg99/pulsar that referenced this pull request Mar 4, 2022
@jerrypeng @dlg99
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
fb8e6e9 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit b7e027b)
eolivelli pushed a commit to datastax/pulsar that referenced this pull request Mar 7, 2022
@jerrypeng @eolivelli
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
9b23e26 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
(cherry picked from commit b7e027b)
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
@jerrypeng @ciaocloud
Fix: Cast exception occurs if function/source/sink type is ByteBuffer (…
4b896c1 
…apache#11611)

Co-authored-by: Jerry Peng <jerryp@splunk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@merlimat merlimat merlimat approved these changes

@cckellogg cckellogg cckellogg approved these changes

@srkukarni srkukarni Awaiting requested review from srkukarni

@aahmed-se aahmed-se Awaiting requested review from aahmed-se

Assignees

@jerrypeng jerrypeng

Labels
area/connector area/function cherry-picked/branch-2.8 Archived: 2.8 is end of life release/2.8.1 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Milestone
2.9.0
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@jerrypeng @merlimat @cckellogg @hangc0276