[Task]: Java SDK, make BigTableIO compatible with BigTable library 2.71.x #37359
Description
What needs to happen?
Hello dear team,
It appears there's a breaking change when using Apache Beam 2.70 with Bigtable 2.71.0, specifically impacting BigtableIO. The system fails to start under this configuration.
However, it functions correctly with Bigtable 2.70.1.
The current version of Bigtable integrated into BigtableIO seems to be 2.68.0.
Exception in thread "main" java.lang.IllegalStateException: java.lang.NoSuchMethodError: 'com.google.api.gax.core.FixedExecutorProvider com.google.api.gax.core.FixedExecutorProvider.create(java.util.concurrent.ScheduledExecutorService, boolean)' at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:823) at org.springframework.boot.SpringApplication.run(SpringApplication.java:328) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1361) at org.springframework.boot.SpringApplication.run(SpringApplication.java:1350) at de.bonprix.package.app.main(App.java:13) Caused by: java.lang.NoSuchMethodError: 'com.google.api.gax.core.FixedExecutorProvider com.google.api.gax.core.FixedExecutorProvider.create(java.util.concurrent.ScheduledExecutorService, boolean)' at com.google.cloud.bigtable.data.v2.stub.BigtableClientContext.create(BigtableClientContext.java:88) at com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub.createBigtableClientContext(EnhancedBigtableStub.java:224) at com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub.create(EnhancedBigtableStub.java:205) at com.google.cloud.bigtable.data.v2.BigtableDataClient.create(BigtableDataClient.java:179) at org.apache.beam.sdk.io.gcp.bigtable.BigtableServiceFactory.checkTableExists(BigtableServiceFactory.java:204) at org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$WriteWithResults.validateTableExists(BigtableIO.java:1289) at org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$WriteWithResults.validate(BigtableIO.java:1265) at org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$Write.validate(BigtableIO.java:1174) at org.apache.beam.sdk.transforms.PTransform.validate(PTransform.java:180) at org.apache.beam.sdk.Pipeline$ValidateVisitor.enterCompositeTransform(Pipeline.java:673) at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:581) at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:585) at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$500(TransformHierarchy.java:240) at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:214) at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:477) at org.apache.beam.sdk.Pipeline.validate(Pipeline.java:609) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:323) at org.apache.beam.sdk.Pipeline.run(Pipeline.java:310) at de.bonprix.package.apprunner.run(AppRunner.java:226) at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:788) at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:82) at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60) at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:86) at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:796) at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:787) at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:772) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772) at org.springframework.boot.SpringApplication.run(SpringApplication.java:325) ... 3 more
Issue Priority
Priority: 2 (default / most normal work should be filed as P2)
Issue Components
- Component: Python SDK
- Component: Java SDK
- Component: Go SDK
- Component: Typescript SDK
- Component: IO connector
- Component: Beam YAML
- Component: Beam examples
- Component: Beam playground
- Component: Beam katas
- Component: Website
- Component: Infrastructure
- Component: Spark Runner
- Component: Flink Runner
- Component: Samza Runner
- Component: Twister2 Runner
- Component: Hazelcast Jet Runner
- Component: Google Cloud Dataflow Runner