spanner.connection.it.ITTransactionRetryTest: testAbortWithExceptionOnSelect failed #685
Assignees
Labels
api: spanner
Issues related to the googleapis/java-spanner API.
priority: p1
Important issue which blocks shipping the next release. Will be fixed prior to next release.
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Comments
flaky-bot
bot
added
buildcop: issue
priority: p1
product-auto-label
bot
added
the
api: spanner
|
Looks like this issue is flaky. I'm going to leave this open and stop commenting. A human should fix and close this. When run at the same commit (d093089), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge). |
olavloite
added a commit
that referenced
this issue
Dec 8, 2020
Transaction retries in the Connection API / JDBC driver could fail if the following happened: 1. The initial transaction contains a statement that returns an error that does not invalidate the transaction, such as for example a "Table not found" error, and that error is caught and handled by the application code. 2. The retry attempt tries to execute the failed statement to verify that the statement still returns the same error. If however the transaction that is used by the retry has been aborted immediately before the execution of this statement, the statement will now return Aborted instead of the original error. That would be seen as a different error than the initial error and would fail the retry attempt. When the above happens, the Aborted error in the retry should be propagated and the retry attempt should be restarted. Fixes #685
thiagotnunes
pushed a commit
that referenced
this issue
Dec 9, 2020
…688) Transaction retries in the Connection API / JDBC driver could fail if the following happened: 1. The initial transaction contains a statement that returns an error that does not invalidate the transaction, such as for example a "Table not found" error, and that error is caught and handled by the application code. 2. The retry attempt tries to execute the failed statement to verify that the statement still returns the same error. If however the transaction that is used by the retry has been aborted immediately before the execution of this statement, the statement will now return Aborted instead of the original error. That would be seen as a different error than the initial error and would fail the retry attempt. When the above happens, the Aborted error in the retry should be propagated and the retry attempt should be restarted. Fixes #685
thiagotnunes
pushed a commit
that referenced
this issue
May 6, 2021
…688) Transaction retries in the Connection API / JDBC driver could fail if the following happened: 1. The initial transaction contains a statement that returns an error that does not invalidate the transaction, such as for example a "Table not found" error, and that error is caught and handled by the application code. 2. The retry attempt tries to execute the failed statement to verify that the statement still returns the same error. If however the transaction that is used by the retry has been aborted immediately before the execution of this statement, the statement will now return Aborted instead of the original error. That would be seen as a different error than the initial error and would fail the retry attempt. When the above happens, the Aborted error in the retry should be propagated and the retry attempt should be restarted. Fixes #685
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This test failed!
To configure my behavior, see the Build Cop Bot documentation.
If I'm commenting on this issue too often, add the
buildcop: quietlabel andI will stop commenting.
commit: d093089
buildURL: Build Status, Sponge
status: failed
Test output
com.google.cloud.spanner.AbortedDueToConcurrentModificationException: The transaction was aborted and could not be retried due to a database error during the retry at com.google.cloud.spanner.SpannerExceptionFactory.newAbortedDueToConcurrentModificationException(SpannerExceptionFactory.java:137) at com.google.cloud.spanner.connection.FailedQuery.retry(FailedQuery.java:78) at com.google.cloud.spanner.connection.ReadWriteTransaction.handleAborted(ReadWriteTransaction.java:742) at com.google.cloud.spanner.connection.ReadWriteTransaction.runWithRetry(ReadWriteTransaction.java:641) at com.google.cloud.spanner.connection.ReadWriteTransaction$2.call(ReadWriteTransaction.java:302) at com.google.cloud.spanner.connection.ReadWriteTransaction$2.call(ReadWriteTransaction.java:299) at io.grpc.Context$2.call(Context.java:596) at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69) at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: com.google.cloud.spanner.connection.AbstractBaseUnitOfWork$SpannerAsyncExecutionException: Execution failed for statement: SELECT * FROM TEST at com.google.cloud.spanner.connection.AbstractBaseUnitOfWork.executeStatementAsync(AbstractBaseUnitOfWork.java:226) at com.google.cloud.spanner.connection.ReadWriteTransaction.executeQueryAsync(ReadWriteTransaction.java:297) at com.google.cloud.spanner.connection.ConnectionImpl.internalExecuteQuery(ConnectionImpl.java:903) at com.google.cloud.spanner.connection.ConnectionImpl.parseAndExecuteQuery(ConnectionImpl.java:763) at com.google.cloud.spanner.connection.ConnectionImpl.executeQuery(ConnectionImpl.java:731) at com.google.cloud.spanner.connection.it.ITTransactionRetryTest.testAbortWithExceptionOnSelect(ITTransactionRetryTest.java:1083) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:87) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) Caused by: com.google.cloud.spanner.AbortedException: ABORTED: Aborted due to failed initial statement at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:235) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:168) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:108) at com.google.cloud.spanner.TransactionRunnerImpl$TransactionContextImpl.getTransactionSelector(TransactionRunnerImpl.java:445) at com.google.cloud.spanner.AbstractReadContext.getExecuteSqlRequestBuilder(AbstractReadContext.java:571) at com.google.cloud.spanner.AbstractReadContext$1.startStream(AbstractReadContext.java:623) at com.google.cloud.spanner.AbstractResultSet$ResumableStreamIterator.computeNext(AbstractResultSet.java:1020) at com.google.cloud.spanner.AbstractResultSet$ResumableStreamIterator.computeNext(AbstractResultSet.java:898) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at com.google.cloud.spanner.AbstractResultSet$GrpcValueIterator.ensureReady(AbstractResultSet.java:260) at com.google.cloud.spanner.AbstractResultSet$GrpcValueIterator.getMetadata(AbstractResultSet.java:236) at com.google.cloud.spanner.AbstractResultSet$GrpcResultSet.next(AbstractResultSet.java:118) at com.google.cloud.spanner.ForwardingResultSet.next(ForwardingResultSet.java:54) at com.google.cloud.spanner.SessionPool$SessionPoolResultSet.next(SessionPool.java:563) at com.google.cloud.spanner.connection.DirectExecuteResultSet.(DirectExecuteResultSet.java:62) at com.google.cloud.spanner.connection.DirectExecuteResultSet.ofResultSet(DirectExecuteResultSet.java:56) at com.google.cloud.spanner.connection.ReadWriteTransaction$2$1.call(ReadWriteTransaction.java:313) at com.google.cloud.spanner.connection.ReadWriteTransaction$2$1.call(ReadWriteTransaction.java:303) at com.google.cloud.spanner.connection.ReadWriteTransaction.runWithRetry(ReadWriteTransaction.java:639) ... 9 more Caused by: com.google.cloud.spanner.AbortedException: ABORTED: Aborted due to failed initial statement at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:235) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:59) at com.google.cloud.spanner.TransactionRunnerImpl$TransactionContextImpl.onError(TransactionRunnerImpl.java:476) at com.google.cloud.spanner.AbstractResultSet$GrpcResultSet.yieldError(AbstractResultSet.java:154) at com.google.cloud.spanner.AbstractResultSet$GrpcResultSet.next(AbstractResultSet.java:130) ... 16 more Caused by: com.google.cloud.spanner.SpannerException: INVALID_ARGUMENT: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Table not found: FOO [at 1:15] SELECT * FROM FOO ^ - Statement: 'SELECT * FROM FOO' at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:259) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:168) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:108) at com.google.cloud.spanner.AbstractResultSet$GrpcStreamIterator.computeNext(AbstractResultSet.java:838) at com.google.cloud.spanner.AbstractResultSet$GrpcStreamIterator.computeNext(AbstractResultSet.java:769) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at com.google.cloud.spanner.AbstractResultSet$ResumableStreamIterator.computeNext(AbstractResultSet.java:1029) at com.google.cloud.spanner.AbstractResultSet$ResumableStreamIterator.computeNext(AbstractResultSet.java:898) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at com.google.cloud.spanner.AbstractResultSet$GrpcValueIterator.ensureReady(AbstractResultSet.java:260) at com.google.cloud.spanner.AbstractResultSet$GrpcValueIterator.getMetadata(AbstractResultSet.java:236) at com.google.cloud.spanner.AbstractResultSet$GrpcResultSet.next(AbstractResultSet.java:118) ... 16 more Suppressed: com.google.cloud.spanner.connection.AbstractBaseUnitOfWork$SpannerAsyncExecutionException: Execution failed for statement: SELECT * FROM FOO at com.google.cloud.spanner.connection.AbstractBaseUnitOfWork.executeStatementAsync(AbstractBaseUnitOfWork.java:226) at com.google.cloud.spanner.connection.ReadWriteTransaction.executeQueryAsync(ReadWriteTransaction.java:297) at com.google.cloud.spanner.connection.ConnectionImpl.internalExecuteQuery(ConnectionImpl.java:903) at com.google.cloud.spanner.connection.ConnectionImpl.parseAndExecuteQuery(ConnectionImpl.java:763) at com.google.cloud.spanner.connection.ConnectionImpl.executeQuery(ConnectionImpl.java:731) at com.google.cloud.spanner.connection.it.ITTransactionRetryTest.testAbortWithExceptionOnSelect(ITTransactionRetryTest.java:1073) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:27) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:87) at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) Caused by: com.google.cloud.spanner.SpannerException: INVALID_ARGUMENT: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Table not found: FOO [at 1:15] SELECT * FROM FOO ^ - Statement: 'SELECT * FROM FOO' at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:259) at com.google.cloud.spanner.AbstractResultSet$GrpcStreamIterator$ConsumerImpl.onError(AbstractResultSet.java:874) at com.google.cloud.spanner.spi.v1.GapicSpannerRpc$SpannerResponseObserver.onError(GapicSpannerRpc.java:1686) at com.google.api.gax.tracing.TracedResponseObserver.onError(TracedResponseObserver.java:103) at com.google.api.gax.rpc.Watchdog$WatchdogStream.onErrorImpl(Watchdog.java:284) at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:86) at com.google.api.gax.grpc.ExceptionResponseObserver.onErrorImpl(ExceptionResponseObserver.java:84) at com.google.api.gax.rpc.StateCheckingResponseObserver.onError(StateCheckingResponseObserver.java:86) at com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onClose(GrpcDirectStreamController.java:149) at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose(SpannerErrorInterceptor.java:100) at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:617) at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:803) at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:782) at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ... 3 more Caused by: com.google.cloud.spanner.SpannerException: INVALID_ARGUMENT: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Table not found: FOO [at 1:15] SELECT * FROM FOO ^ at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted(SpannerExceptionFactory.java:259) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:59) at com.google.cloud.spanner.SpannerExceptionFactory.fromApiException(SpannerExceptionFactory.java:274) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:172) at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException(SpannerExceptionFactory.java:108) ... 24 more Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Table not found: FOO [at 1:15] SELECT * FROM FOO ^ at io.grpc.Status.asRuntimeException(Status.java:533) ... 18 moreThe text was updated successfully, but these errors were encountered: