Instrument Active Record transactions

[ipc103]

Motivation / Background

This Pull Request adds events to track when Active Record-managed transactions occur. Tracking Active Record-managed transactions seems to be a common need, but there's currently not a great way to do it. Here's a few examples we've seen:

• GitHub has custom transaction tracking that monkey patches the Active Record TransactionManager and RealTransaction. We use the tracking to prevent opening a transaction to one database cluster inside a transaction to a different database cluster, and to report slow transactions (we get slow transaction data directly from MySQL as well, but it's still helpful to report from the application with backtraces to help track them down).
• https://github.com/palkan/isolator tracks transactions to prevent non-atomic interactions like external network calls inside a transaction. The gem works by subscribing to sql.active_record, then piecing together the transactions by looking for BEGIN, COMMIT, SAVEPOINT, etc., but this is unreliable: - Corrupt transaction counts with Rails 7.1 restarting savepoint transactions palkan/isolator#65 - Corrupt transaction counts when connection closes unexpectedly palkan/isolator#64
• It looks like GitLab patches TransactionManager and RealTransaction to track nested savepoints. See Subtransactions tracking/preventions palkan/isolator#46

Detail

This PR adds a new transaction.active_record event that should provide a more reliable solution for these various use cases. It includes the connection in the payload (useful, for example, in differentiating transactions to different databases),

Additional information

• This instrumentation needs to start and finish at fairly specific times:
  • start on materialize
  • finish after committing or rolling back, but before the after_commit or after_rollback callbacks
  • finish and start again when the transaction restarts (at least for real transactions—we've done it for savepoints as well but I'm not certain we should)
  • ensure it finishes if commit and rollback fail (e.g. if the connection goes away) To make all that work, this commit uses the lower-level #build-handle API instead of #instrument.
• If this change gets merged we're also planning to add details about what type of transaction it is (savepoint or real) and what the outcome is (commit, rollback, restarted, errored).

Checklist

Before submitting the PR make sure the following are checked:

• This Pull Request is related to one change. Changes that are unrelated should be opened in separate PRs.
• Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
• Tests are added or updated if you fix a bug or add a feature.
• CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.

[fxn]

For practical uses, a pity the transaction or some identifier or something is not available in the payload. I am pondering to add a (lazy) UUID to transactions to be able to group queries in logs.

[zzak]

I also felt weird the whole connection object was made available, but I don't have a clear reason as to why.
But I guess the transaction is not a public API. 🤔

[fxn]

Right, in 7.2 it is going to be. There is now ActiveRecord::Transaction, current_transaction, transaction calls receive a transaction object as block parameter, etc.

I have the transaction UUID in place, and will look at this event now.

[fxn]

PR passing the transaction object here #51955.

[composerinteralia]

I also felt weird the whole connection object was made available, but I don't have a clear reason as to why.

GitHub mainly uses it to differentiate transactions by connection (e.g. using the #connection_class), which gives us a way to detect and prevent nesting transactions to different databases. It looks like the isolator gem is also using the connection for similar purposes.

FWIW sql.active_record also passes the whole connection.