Skip to content

Use pulsar-client-all jar #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 14, 2022
Merged

Use pulsar-client-all jar #114

merged 1 commit into from
Sep 14, 2022

Conversation

onobc
Copy link
Collaborator

@onobc onobc commented Sep 13, 2022

@onobc onobc merged commit 5b549c3 into spring-projects:main Sep 14, 2022
@onobc onobc deleted the cbono-use-pulsar-client-all branch September 14, 2022 05:03
@nandorholozsnyak
Copy link

Hello there,
I know it has been a while, I'm just curious why it was done in this way to include the pulsar-client-all and not the other modules independently?

We have just updated to Spring Boot 3.3.3 (3.3.4) while using the Spring Cloud GCP's SecretManager integration, and in the App bootstrap phase we identified a really nasty VerifyError.

The problem is that while the GCP dependencies were constantly upgrading the protobuf-java dependency version (namely in the latest is the 3.25.5), the pulsar-client-all is a uber jar (so it puts the protobuf-java classes into the jar) and puts a super old protobuf-java (3.22.3) and there are incompatibilities.

For example in the pulsar-client-all (I guess it's 3.22.3):

public class MapField<K, V> implements MutabilityOracle {
...

And in the 3.25.5 protobuf-java which is used by the Spring Boot GCP dependencies (Secret):

public class MapField<K, V> extends MapFieldReflectionAccessor implements MutabilityOracle {
....

What is the solution?

The question again: Why was it better to include the -all module instead of the modules independently like it was before this PR.

Error:

13:12:14.412 [main] ERROR org.springframework.boot.SpringApplication -- Application run failed
java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    com/google/cloud/secretmanager/v1/Secret.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @40: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @40
    flags: { }
    locals: { 'com/google/cloud/secretmanager/v1/Secret', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0000 0000 0032 0000 0003 0000 0004
    0000010: 0000 0023 0000 000b 0000 0028 0000 000d
    0000020: 0000 002d 2ab7 0005 b02a b700 03b0 2ab7
    0000030: 0002 b0bb 001d 59bb 001e 59b7 001f 1220
    0000040: b600 211b b600 22b6 0023 b700 24bf     
  Stackmap Table:
    same_frame(@36)
    same_frame(@41)
    same_frame(@46)
    same_frame(@51)

	at com.google.cloud.secretmanager.v1.stub.GrpcSecretManagerServiceStub.<clinit>(GrpcSecretManagerServiceStub.java:82)
	at com.google.cloud.secretmanager.v1.stub.SecretManagerServiceStubSettings.createStub(SecretManagerServiceStubSettings.java:359)
	at com.google.cloud.secretmanager.v1.SecretManagerServiceClient.<init>(SecretManagerServiceClient.java:455)
	at com.google.cloud.secretmanager.v1.SecretManagerServiceClient.create(SecretManagerServiceClient.java:437)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.createSecretManagerClient(SecretManagerConfigDataLocationResolver.java:120)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.lambda$registerSecretManagerBeans$0(SecretManagerConfigDataLocationResolver.java:75)
	at org.springframework.boot.BootstrapRegistry$InstanceSupplier.lambda$from$1(BootstrapRegistry.java:160)
	at org.springframework.boot.DefaultBootstrapContext.getInstance(DefaultBootstrapContext.java:119)
	at org.springframework.boot.DefaultBootstrapContext.getOrElseThrow(DefaultBootstrapContext.java:111)
	at org.springframework.boot.DefaultBootstrapContext.get(DefaultBootstrapContext.java:88)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.createSecretManagerTemplate(SecretManagerConfigDataLocationResolver.java:132)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.registerSecretManagerBeans(SecretManagerConfigDataLocationResolver.java:85)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.resolve(SecretManagerConfigDataLocationResolver.java:60)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.lambda$resolve$1(ConfigDataLocationResolvers.java:102)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.resolve(ConfigDataLocationResolvers.java:113)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.resolve(ConfigDataLocationResolvers.java:102)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.resolve(ConfigDataLocationResolvers.java:94)
	at org.springframework.boot.context.config.ConfigDataImporter.resolve(ConfigDataImporter.java:106)
	at org.springframework.boot.context.config.ConfigDataImporter.resolve(ConfigDataImporter.java:98)
	at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:86)
	at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:121)
	at org.springframework.boot.context.config.ConfigDataEnvironment.processInitial(ConfigDataEnvironment.java:245)
	at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:232)
	at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:96)
	at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:89)
	at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEnvironmentPreparedEvent(EnvironmentPostProcessorApplicationListener.java:109)
	at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEvent(EnvironmentPostProcessorApplicationListener.java:94)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:185)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:178)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:156)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:138)
	at org.springframework.boot.context.event.EventPublishingRunListener.multicastInitialEvent(EventPublishingRunListener.java:136)
	at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:81)
	at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:64)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
	at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:63)
	at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:370)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:330)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1363)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1352)
	at com.example.Application.main(Application.java:20)
Exception in thread "main" java.lang.IllegalStateException: java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    com/google/cloud/secretmanager/v1/Secret.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @40: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @40
    flags: { }
    locals: { 'com/google/cloud/secretmanager/v1/Secret', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0000 0000 0032 0000 0003 0000 0004
    0000010: 0000 0023 0000 000b 0000 0028 0000 000d
    0000020: 0000 002d 2ab7 0005 b02a b700 03b0 2ab7
    0000030: 0002 b0bb 001d 59bb 001e 59b7 001f 1220
    0000040: b600 211b b600 22b6 0023 b700 24bf     
  Stackmap Table:
    same_frame(@36)
    same_frame(@41)
    same_frame(@46)
    same_frame(@51)

	at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:825)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:345)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1363)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1352)
	at com.example.Application.main(Application.java:20)
Caused by: java.lang.VerifyError: Bad return type
Exception Details:
  Location:
    com/google/cloud/secretmanager/v1/Secret.internalGetMapFieldReflection(I)Lcom/google/protobuf/MapFieldReflectionAccessor; @40: areturn
  Reason:
    Type 'com/google/protobuf/MapField' (current frame, stack[0]) is not assignable to 'com/google/protobuf/MapFieldReflectionAccessor' (from method signature)
  Current Frame:
    bci: @40
    flags: { }
    locals: { 'com/google/cloud/secretmanager/v1/Secret', integer }
    stack: { 'com/google/protobuf/MapField' }
  Bytecode:
    0000000: 1bab 0000 0000 0032 0000 0003 0000 0004
    0000010: 0000 0023 0000 000b 0000 0028 0000 000d
    0000020: 0000 002d 2ab7 0005 b02a b700 03b0 2ab7
    0000030: 0002 b0bb 001d 59bb 001e 59b7 001f 1220
    0000040: b600 211b b600 22b6 0023 b700 24bf     
  Stackmap Table:
    same_frame(@36)
    same_frame(@41)
    same_frame(@46)
    same_frame(@51)

	at com.google.cloud.secretmanager.v1.stub.GrpcSecretManagerServiceStub.<clinit>(GrpcSecretManagerServiceStub.java:82)
	at com.google.cloud.secretmanager.v1.stub.SecretManagerServiceStubSettings.createStub(SecretManagerServiceStubSettings.java:359)
	at com.google.cloud.secretmanager.v1.SecretManagerServiceClient.<init>(SecretManagerServiceClient.java:455)
	at com.google.cloud.secretmanager.v1.SecretManagerServiceClient.create(SecretManagerServiceClient.java:437)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.createSecretManagerClient(SecretManagerConfigDataLocationResolver.java:120)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.lambda$registerSecretManagerBeans$0(SecretManagerConfigDataLocationResolver.java:75)
	at org.springframework.boot.BootstrapRegistry$InstanceSupplier.lambda$from$1(BootstrapRegistry.java:160)
	at org.springframework.boot.DefaultBootstrapContext.getInstance(DefaultBootstrapContext.java:119)
	at org.springframework.boot.DefaultBootstrapContext.getOrElseThrow(DefaultBootstrapContext.java:111)
	at org.springframework.boot.DefaultBootstrapContext.get(DefaultBootstrapContext.java:88)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.createSecretManagerTemplate(SecretManagerConfigDataLocationResolver.java:132)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.registerSecretManagerBeans(SecretManagerConfigDataLocationResolver.java:85)
	at com.google.cloud.spring.autoconfigure.secretmanager.SecretManagerConfigDataLocationResolver.resolve(SecretManagerConfigDataLocationResolver.java:60)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.lambda$resolve$1(ConfigDataLocationResolvers.java:102)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.resolve(ConfigDataLocationResolvers.java:113)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.resolve(ConfigDataLocationResolvers.java:102)
	at org.springframework.boot.context.config.ConfigDataLocationResolvers.resolve(ConfigDataLocationResolvers.java:94)
	at org.springframework.boot.context.config.ConfigDataImporter.resolve(ConfigDataImporter.java:106)
	at org.springframework.boot.context.config.ConfigDataImporter.resolve(ConfigDataImporter.java:98)
	at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:86)
	at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:121)
	at org.springframework.boot.context.config.ConfigDataEnvironment.processInitial(ConfigDataEnvironment.java:245)
	at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:232)
	at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:96)
	at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:89)
	at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEnvironmentPreparedEvent(EnvironmentPostProcessorApplicationListener.java:109)
	at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEvent(EnvironmentPostProcessorApplicationListener.java:94)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:185)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:178)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:156)
	at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:138)
	at org.springframework.boot.context.event.EventPublishingRunListener.multicastInitialEvent(EventPublishingRunListener.java:136)
	at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:81)
	at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:64)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
	at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
	at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:63)
	at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:370)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:330)
	... 3 more

@onobc onobc mentioned this pull request Oct 8, 2024
Closed
@onobc
Copy link
Collaborator Author

onobc commented Oct 8, 2024

Hi @nandorholozsnyak,

First of all, let me raise my hand for creating a PR w/ absolutely no description nor motivation for the change. My bad. Let me rectify that now...

The need for this change arose when we moved the auto-configuration portion of the project into Spring Boot proper.
Spring Boot is strict on the libraries it supports not having duplicate classes. Because there are duplicate classes in the individual libs we moved to the single jar to avoid the duplication of classes.

More details on the Pulsar client libs here: https://pulsar.apache.org/docs/3.3.x/client-libraries-java-setup/#step-1-install-java-client-library

So... I did not realize (until now) that protobuf is not being shaded in the current Pulsar client jars (single or all jar). Because of this, even if you use the individual libs you would run into the same issue. Regardless, we would like to improve this situation. I have created and issue to track/communicate on this topic going forward. #876

Thank you very much for taking the time to bring this up again. Much appreciated.

@nandorholozsnyak
Copy link

Hello there @onobc thank you so much, appreciate it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@onobc @nandorholozsnyak