-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Shade pulsar-functions-runtime instead of pulsar-functions-worker #1351
Conversation
/cc @srkukarni @jerrypeng |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
<groupId>${project.groupId}</groupId> | ||
<artifactId>pulsar-common</artifactId> | ||
<version>${project.version}</version> | ||
<exclusions> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to exclude checksum here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
because bookkeeper-server-shaded jar include the circe-checksum when shading guava. so circe-checksum is effectively available in pulsar-broker when including bookkeeper-server-shaded
jar. so excluding pulsar-common to make the behavior more deterministic.
that said, we probably don't have to exclude it though. because in the classpath we have both shaded guava and non-shaded guava.
merged to latest master since the new |
- pulsar-functions-worker-shaded was removed already by apache#1351
- pulsar-functions-worker-shaded was removed already by #1351
Motivation
pulsar-functions-runtime
is required by worker/broker and also it would be packaged into a fat jar to used by process runtime to invoke function executions.so
pulsar-functions-runtime
notpulsar-functions-worker
.Modifications
runtime-shade
: package all the dependencies that use netty and protobuf and shade netty and protobuf, it also exclude the shaded client dependency. so that it can be integrated into worker and be used as part of worker.runtime-all
: this is the module for packagingruntime-shade
and a non-shade pulsar-client to produce a fat jar for java process runtime.pulsar-client-tools
andpulsar-broker
to usepulsar-functions-worker
since worker module now is using ashaded
runtime jar, there will be no protobuf/netty conflicts.At the same time, changed using bookkeeper/test-jar to bookkeeper-server-tests-shaded, so it won't pollute the classpath.
Result