Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Omit BEGIN/COMMIT statements for empty transactions #32647
If a transaction is opened and closed without any queries being run, we can safely omit the
This implementation buffers transactions inside the transaction manager and materializes them the next time the connection is used. For this to work, the adapter needs to guard all connection use with a call to
This test failure was legitimate: https://travis-ci.org/rails/rails/jobs/368924286#L1171-L1175
Executing a prepared statement doesn't touch the connection object (at least on the SQLite adapter), but does require the connection's transaction state to be up to date.
I've fixed the problem by materializing transactions in
added a commit
this pull request
Apr 23, 2018
Oracle enhanced adapter needs some update to support this pull request. Here are CI results for this pull request https://travis-ci.org/yahonda/oracle-enhanced/builds/370051142 .
I am still having some jet lags due after a long flight, I'll take a look at this result in detail tomorrow.
A few thoughts from an initial read-through:
My gut feeling is that this "should" be closer to the transaction manager rather than the connection -- particularly the deferral stack, though the connection of course needs to be responsible for triggering the materialize.
Failing that, I also wonder about putting the functionality on the abstract connection itself rather than a proxy. I don't mind the code complexity needed for more thorough bookkeeping, but the proxy feels architecturally heavyweight.
Side-thought that would probably just distract us to pursue right now, but I'll mention anyway: this has some interesting parallels (deferring work on the connection until we need to run a real statement, and then potentially giving that some special handling) with #28200 and friends.
Finally, the big one, because it's about behaviour rather than implementation: I believe it's important that savepoints don't materialize the transaction -- that they too get deferred. To me, the ideal version of this change allows us to create transactions more freely, which means we can default
Thanks for the detailed feedback (as always) @matthewd!
I originally wrote a patch that tracked deferred transactions in the transaction manager and materialised them from the adapter, just as you suggested. I wanted to explore the proxy idea instead because I was concerned about the maintenance burden of guarding all connection usage, but if that's an acceptable tradeoff to make for a simpler design, I'll dig up the other implementation and switch to it.
You make a good point that once we have a hook that checks some property of the connection every time it's used and are committed to maintaining that invariant, #28200 seems tantalisingly within reach.
I agree that savepoints should be in scope here, as well as "fixture" transactions; the latter is actually what motivated me to work this issue (10 connection pools * 30,000 tests = lots of empty transactions). Adding support for them breaks dozens of tests that assert that an exact set of queries are executed though, so I wanted to get feedback on the core of the idea before I fixed them all up.
I've pushed an alternative implementation that guards all connection use in the adapters instead of wrapping the raw connection in a proxy. The previous implementation is available here.
This version defers savepoints as well as transactions, and is also enabled for "fixture" transactions.
I definitely prefer this approach. Thanks for preserving the previous one for comparison, too.
@sgrif are you more/less/equally inclined toward this version? It spreads out the
materialize_transactions calls, but to places we already need to care about for other similar reasons (logging, connection locking, future #28200)... that aside, the logic feels more semantically contained to me.
I also think this makes my other future nice-to-have more accessible: if we're three unmaterialized transactions deep when we run a statement, we only need to send a single BEGIN. (If that transaction ends up rolled back, we send a ROLLBACK. If it ends up committed, we send nothing, and mark its parent as having been materialized.) But that's for another time.
@kamipo I think this is valuable for two use cases: 1) bulk updates, where many/most records aren't actually changed: each
Basically, I think single empty transactions are likely quite rare, and also uninteresting (a small fixed cost on an unusual request is not much of a concern) -- but there are reasonable-enough patterns that can lead to lots of empty transactions, and while the caller could work around them, it's something we're in a better position to solve.
The two specific use cases I know of are 1) ManageIQ, which carried a custom patch since Rails 2.2 (they lost it during an upgrade, but last I heard it's still an ongoing source of pain), and 2) @jeremy's point in #17937 (comment) about not-actual-writes forcing a switch to the primary DB (likely to become a wider complaint as we improve built-in support for such configurations).
On Thu, Aug 23, 2018 at 03:10 Matthew Draper ***@***.***> wrote: Sorry I left this sitting for ages, and thanks again!