Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: convert silent rollbacks into exception if application sends commit command #1729

Merged
merged 1 commit into from Mar 6, 2020

Conversation

vlsi
Copy link
Member

@vlsi vlsi commented Mar 6, 2020

Hopefully, the server would fix the error in the future.

See discussion in pgsql-hackers: https://www.postgresql.org/message-id/b9fb50dc-0f6e-15fb-6555-8ddb86f4aa71%40postgresfriends.org

fixes #1697

For documentation purposes: the feature will be enabled by default.
It can be disabled by setting raiseExceptionOnSilentRollback=false connection property.

Sample failure:

org.postgresql.util.PSQLException: The database returned ROLLBACK, so the transaction cannot be committed. Transaction failure cause is <<ERROR: savepoint "jdbc_savepoint_1" does not exist>>
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2191)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:316)
	at org.postgresql.jdbc.PgConnection.executeTransactionCommand(PgConnection.java:829)
	at org.postgresql.jdbc.PgConnection.commit(PgConnection.java:851)
	at org.postgresql.jdbc.PgConnection.setAutoCommit(PgConnection.java:793)
	at org.postgresql.test.jdbc3.Jdbc3SavepointTest.tearDown(Jdbc3SavepointTest.java:40)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
	at java.util.Iterator.forEachRemaining(Iterator.java:116)
	at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
	at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:82)
	at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:73)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:412)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: savepoint "jdbc_savepoint_1" does not exist
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2562)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2297)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:316)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:448)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:369)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:310)
	at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:296)
	at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:273)
	at org.postgresql.jdbc.PgConnection.execSQLUpdate(PgConnection.java:479)
	at org.postgresql.jdbc.PgConnection.rollback(PgConnection.java:1741)
	at org.postgresql.test.jdbc3.Jdbc3SavepointTest.testRollingBackToInvalidSavepointFails(Jdbc3SavepointTest.java:216)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	... 61 more

@vlsi vlsi force-pushed the raise_exception_on_silent_commit branch from aa21fc6 to f03b463 Compare Mar 6, 2020
@vlsi vlsi mentioned this pull request Mar 6, 2020
2 tasks
@vlsi vlsi force-pushed the raise_exception_on_silent_commit branch 11 times, most recently from 5c6cc50 to 1258986 Compare Mar 6, 2020
@vlsi vlsi force-pushed the raise_exception_on_silent_commit branch 5 times, most recently from e85aad1 to cef17fb Compare Mar 6, 2020
@codecov-io
Copy link

codecov-io commented Mar 6, 2020

Codecov Report

Merging #1729 into master will decrease coverage by 3.05%.
The diff coverage is 82.14%.

@@             Coverage Diff              @@
##             master    #1729      +/-   ##
============================================
- Coverage     69.44%   66.38%   -3.06%     
- Complexity     4201     4238      +37     
============================================
  Files           187      187              
  Lines         17279    17803     +524     
  Branches       2872     3054     +182     
============================================
- Hits          11999    11819     -180     
- Misses         3993     4657     +664     
- Partials       1287     1327      +40

@vlsi vlsi force-pushed the raise_exception_on_silent_commit branch from cef17fb to 8abbd4f Compare Mar 6, 2020
@vlsi vlsi merged commit 13decbb into pgjdbc:master Mar 6, 2020
0 of 2 checks passed
vlsi added a commit that referenced this issue Mar 7, 2020
…mit command (#1729)

The feature is enabled by default.
It can be disabled by setting raiseExceptionOnSilentRollback=false connection property.

See discussion in pgsql-hackers: https://www.postgresql.org/message-id/b9fb50dc-0f6e-15fb-6555-8ddb86f4aa71%40postgresfriends.org

fixes #1697
vlsi added a commit that referenced this issue Mar 7, 2020
…mit command (#1729)

The feature is enabled by default.
It can be disabled by setting raiseExceptionOnSilentRollback=false connection property.

See discussion in pgsql-hackers: https://www.postgresql.org/message-id/b9fb50dc-0f6e-15fb-6555-8ddb86f4aa71%40postgresfriends.org

fixes #1697
vlsi added a commit that referenced this issue Mar 7, 2020
…mit command (#1729)

The feature is enabled by default.
It can be disabled by setting raiseExceptionOnSilentRollback=false connection property.

See discussion in pgsql-hackers: https://www.postgresql.org/message-id/b9fb50dc-0f6e-15fb-6555-8ddb86f4aa71%40postgresfriends.org

fixes #1697
@sehrope
Copy link
Member

sehrope commented Mar 7, 2020

Similar to the PR from the other day, isn't it a bit rushed to merge this in immediately? I'm not opposed to fixing this but it's a change, enabled by default, that changes transaction commit behavior. It's probably a good idea but let's at least have some review period.

For example, what's the idea behind checking the SQL in the looksLikeXYZ(...) methods? Is that checking against the user's SQL? What if there's comments like /* Not a ROLLBACK */ COMMIT?

@vlsi
Copy link
Member Author

vlsi commented Mar 7, 2020

We need to release 42.2.11, and it is not clear if 42.3.0 will have -jre6 branch.
So it makes sense to include rollback WA into 42.2.11 for those who are still using Java 6

What if there's comments like /* Not a ROLLBACK */ COMMIT?

There are tests in the PR, and this SQL is present in tests.

@davecramer
Copy link
Member

davecramer commented Mar 7, 2020

Ya I'd say we want to release. If its' broken we can fix it. Historically we've been paralyzed by lack of willingness to commit anything. Unfortunately we do not get a lot of real world testing of snapshots so the only way to test it is to release it.

@sehrope
Copy link
Member

sehrope commented Mar 8, 2020

If you guys say so then okay. Going forward, standardizing on some standard review period would be nice. I don't know what the right answer is but it's got to be more than 24-hours for a user facing change. The thing that jumped out at me was the parsing of user's SQL commands as that has a tendency of leading to a long term baggage.

I will admit that the thread about "commit" actually being a rollback might be the most surprising thing I've seen ever seen with PG. It's just plain weird. I bet most of us have never run into anything like it either as any sane code would bubble up exception. But man is that weird behavior.

Though even this won't solve the original use case in the issue that spawned this thread as I think a "malicious" user that receives the Connection could do some work, fail to commit, issue a rollback, start a new transaction, then hand back the connection to the caller which things that everything is dandy. Short of actually wrapping Connection and downstream classes in proxies, I doubt it's possible to handle every edge case as they'd never know.

@sehrope
Copy link
Member

sehrope commented Mar 8, 2020

Oh and regarding the minor style issue with sorting the properties. I've got a PR to fix that and add a sort-check test. I'll push that in a min too.

@vlsi
Copy link
Member Author

vlsi commented Mar 8, 2020

Connection could do some work, fail to commit, issue a rollback, start a new transaction, then hand back the connection to the caller which things that everything is dandy.

@sehrope , can you please clarify the test case?

@sehrope
Copy link
Member

sehrope commented Mar 8, 2020

@vlsi I mean a generic situation where the some framework code creates a connection and then hands it off to other user code. IIUC, that's what the user in the original issue was saying could happen:

Say you have some generic framework code that wraps arbitrary user work in transaction, potentially with other work it wants to include in that transaction:

void doWithConnection(Callable<Connection> task) {
    Connection conn = ds.createConnection();
    try {
        conn.setAutoCommit(false);
        // Do our pre-work to get things ready
        doSomethingInThisTransaction(conn);
        // Do the arbitrary user work that we have no control over
        task.call(conn);
        // User work finished successfully so do our post-work
        doSomethingElseInThisTransaction(conn);
        // All work finished successfully so commit...
        conn.commit();
    } catch (Exception e) {
        // User work failed so roll back...
        conn.rollback();
        throw e;
    }
}

If the invoker of the framework invokes it with a task that performs its own commits or rollbacks, there's no guarantee to the framework that the rest of the code is executing in the same transactions:

doWithConnection(conn -> {
    try {
        // Commit the framework transaction
        conn.commit();
    } catch (Exception e) {
        // Ignore error and rollback
        conn.rollback();
    }
    try {
        // Do our own thing
        Statement stmt = conn.createStatement();
        stmt.executeQuery("INSERT ... ");
        // Commit it
        conn.commit();
    } catch (Exception) {        
        // Again ignore the error
        conn.rollback();
    }
    // Pretend we're still in the original transaction
    conn.setAutoCommit(false);
});

Without the framework code wrapping every JDBC interface in some kind of proxy and preventing transaction management commands, I don't see any way of totally stopping this.

In most cases this is a non-issue as any sane user code would be bubble up the exceptions and cause the original transaction to fail. But it sounds like the original issue creator has run into a situation where the other code cannot be trusted and thus could muck with the transaction state.

I think it may be possible for the framework to detect it's a new transaction by tracking something like the transaction XID, but once the client is handed off to the the user task, nothing short of a proxy on every interface is going to stop the arbitrary user code from being able to mess with the transaction or create its own transactions.

Basically I'm saying if you can't trust the rest of your code to bubble up errors then things can can broken without you noticing anyway. That's why I don't see this break being a big deal in practice. Any sane user code would not run into it.

@haumacher
Copy link

haumacher commented Mar 9, 2020

@sehrope Just to clarify the original issue. We observed user code catching the exception (actually it was a single boundary test case of our framework), but not user code intentionally breaking the transaction into pieces using commit()/rollback(). Such code I would consider hostile, and I do not need security against hostile code, only safety against sloppy exception handling in user code.

@vlsi
Copy link
Member Author

vlsi commented Mar 9, 2020

@sehrope , thanks for the cases.
The cases you mention are typically solved with two approaches:

  1. Wrapping the connection with a proxy that forbids application-level Connection#commit() calls. Of course, it can't prevent application-level executeQuery("..;commit;"), but it does have some protection and it does catch bugs
  2. The use of XA connection pool. When using XA transactions, the database forbids explicit use of commit and rollback. If application issues executeQuery("commit") then database would reject it. The only way to proceed with XA is to use prepare transaction 'xid'.... If an application is doing that "by mistake", then nothing can help.

yogthos pushed a commit to yogthos/migratus that referenced this issue Mar 15, 2020
…lbacks into exceptions on commit (pgjdbc/pgjdbc#1729) (#180)

Co-authored-by: Erik Strömberg <erik.stromberg@gmail.com>
davecramer added a commit to davecramer/pgjdbc that referenced this issue Mar 30, 2020
…ends commit command (pgjdbc#1729)"

This reverts commit adcb194.
we still want to do this but it is a breaking change and we will introduce this change in 42.3.0
davecramer added a commit that referenced this issue Mar 30, 2020
…ends commit command (#1729)" (#1746)

This reverts commit adcb194.
we still want to do this but it is a breaking change and we will introduce this change in 42.3.0
randomnicode added a commit to randomnicode/airsonic-advanced that referenced this issue Apr 6, 2020
davecramer pushed a commit to davecramer/pgjdbc that referenced this issue Jul 5, 2021
…mit command (pgjdbc#1729)

The feature is enabled by default.
It can be disabled by setting raiseExceptionOnSilentRollback=false connection property.

See discussion in pgsql-hackers: https://www.postgresql.org/message-id/b9fb50dc-0f6e-15fb-6555-8ddb86f4aa71%40postgresfriends.org

fixes pgjdbc#1697
davecramer added a commit to davecramer/pgjdbc that referenced this issue Jul 5, 2021
…ends commit command (pgjdbc#1729)" (pgjdbc#1746)

This reverts commit adcb194.
we still want to do this but it is a breaking change and we will introduce this change in 42.3.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants