Join GitHub today
Raptor v2 #11254
Thanks, @electrum. After I worked on a few projects, I have to say this is the most elegant codebase.
I mostly read the transaction part, and got some questions. You guys may have discussed before. I don't have the big picture; some questions may not be relevant. So please help me warm up :)
First question, it seems we build a Transaction layer on top of Mysql Transaction layer? For ex, DatabaseMetadataWriter.writeMaster calls masterDbi.useTransaction.
If so, I think we can just use Mysql transaction for both writing and reading, right? A benefit is we may get rid of table "active_commit", and "chunksAdded", "chunksRemoved" inside Transaction.java, since mysql has internal logic for MVCC.
Another question is related to above. I think we are doing MVCC. For ex, table “schemata”, one row’s life is bound by the interval [start_commit_id, end_commit_id]. To clarify, we don’t update, we only insert and delete.
But I think last TransactionID which successful == TRUE, is just last CommitID. We only write into table “schemata” when commits, so may use the committed TransactionID as the CommitID.
I also added some inline comments
There are several reasons why we need to build our own transaction layer:
A commit ID keeps track of committed data. It is allocated sequentially at commit time with the invariant that commits are sequential. This has two important properties:
A transaction ID is used to keep track of chunks that are being created so that they can be cleaned up if the transaction fails.
3 times, most recently
Aug 22, 2018
5 times, most recently
Sep 18, 2018
OrcStorageManager. Looks great.