Balance Module

[WhoSoup]

The goal is to have a way of handling balances that can be easily rolled back in case of errors. My first thought was to implement it similar to the grader module in the following way (using an interface for illustration purposes only. we'll never have two versions of balances so a simple struct would suffice)

type Balance interface {
	NewBlock() Balance
	SetRate(rates []opr.AssetUint)
	AddCoinbase(coinbase Transaction)
	AddTransaction(t Transaction)
	Finalize() (successful, pending, failed []Transaction)
}

The idea is to have a Balance block at a specific height and to build on top of that one for the next block. NewBlock() would deep-copy the Balance and return a new one for height + 1.

To start with, you load the most recent Balance (for height - 1) into memory, then run NewBlock(). You set the rates derived from grading for that height, add the burns and grading rewards as special coinbase Transactions, then add all of the Transactions (which are an umbrella for both transfers and conversions). Transactions know their own height and number of tries left.

The Balance struct would have the following fields:

• balance map[factom.FAAdress]uint64: self-explanatory
• pool []Transaction: a slice of all transactions
• rates map[uint32]map[string]uint64 (or something): basically the Balance block would be capable of tracking rates of multiple heights. only heights for height - maximum number of tries would need to be tracked

Once done, you call Finalize(). This does:

1. Add the coinbase Transactions first (these can never fail)
2. Go through the pool and apply all the transactions. If they succeed or have no more tries left, they are removed, otherwise they stay in the pool. Repeat until there are no possible transactions left.
3. Return a copy of the transactions. successful include coinbase, pending are ones that remain in the pool, failed are ones that have no more tries left (they'll have an error message associated why they failed) This is mostly for application feedback / console notifications.

This would allow us to do the logic of conversions and transfers in one cycle. A conversion can just check if rates exist for its height+1, and will remain in the pool if it doesn't so the next block can handle it.

The Balances struct should be persistable and for a basic "build-as-you-go" state, we only need to keep the most recent one around. However if we persist all of the Balance blocks, it means we can roll back the state to whatever position we desire and calculate metrics like volume and supply for a specific height on-demand (though it would still make more sense to have that in a dedicated table).

Thoughts on this approach?

The downside is that we're not really leveraging the use of SQL's transactions to do anything. We'd be keeping the entire balance book in memory rather than having the state just exist in the tables, and applying individual transactions as sql queries and rolling back if necessary.

[WhoSoup]

In comparison, the current approach is:

1. Start an SQL Transaction
2. Apply burns to DB
3. Apply coinbase rewards to DB
4. Retry any leftover transactions from the previous block (ie conversions that couldn't take place before)
5. Try all new transactions
6. Store all failed/pending transactions in the database
   7a. If there are any errors up to this point, roll back the transaction
   7b. otherwise, execute the SQL Transaction

This will work fine for our purposes but there will not be away to roll back to a previous height. To do that we'll always have to rebuild from genesis.

[Emyrk]

Are you thinking the block for each height would be stored as something like json for each height?

[WhoSoup]

Are you thinking the block for each height would be stored as something like json for each height?

Not json no, but yeah. keeping the entire adress=>balance map probably doesn't make a lot of sense. This would make more sense for an in-memory balance module but since we're building on SQL, it makes a lot more sense to stick with the current approach.

[Emyrk]

I agree with the in-memory approach. I think with the sql, just keeping each block as a tx and applying is enough. If we have a corrupted db/issue, resetting the db is the safest bet vs trying to find the last safe sync point anyway.

[Emyrk]

@WhoSoup Can we close this for now? Since we are going the sql tx approach?