LIP: 0002 Title: Change to byte based block size limit Authors: Iker Alustiza <email@example.com> Nazar Hussain <firstname.lastname@example.org> Discussions-To: https://lists.lisk.io/pipermail/lips_lists.lisk.io/2018-November/000006.html Status: Draft Type: Standards Track Module: Blocks Created: 2018-08-17 Updated: 2018-09-27
This LIP proposes to replace the literal 25 transactions per block size limit with a byte based block size limit.
This LIP is licensed under the GNU General Public License, version 3.
As the Lisk protocol evolves and gains popularity, a clear increase in usage and activity in the network is expected. To cater for this increased demand a more flexible and higher throughput is required. The current maximum of 25 transactions per block imposes a limit that is based on a literal number of transactions per block. This limitation method also conflicts with the varying byte size of each transaction type that can be processed within any given block. We therefore propose a byte based block size limit.
We propose to set a maximum block size of 15KB (kilobytes). This size will maintain the yearly blockchain growth below 50GB, even when blocks are always at full capacity, and the active delegates do not miss any blocks. The summary of the numerical implications are as follows:
- Maximum block size: 15KB.
- Blockchain size will grow a maximum of 47.3GB per year.
- According to the transaction sizes given in the protocol documentation, each transaction type will use the following space in a block:
|Tx type||Block usage (%)|
|0 - Transfer (basic)||0.78%|
|0 - Transfer (2nd signature)||1.2%|
|0 - Transfer (2nd + data field)||1.63%|
|1 - Second Signature||1%|
|2 - Delegate||0.91%|
|3 - Vote (33 votes)||15.08%|
|4 - Multisignature (16 signatures)||8.15%|
|5 - Dapp||1.66%|
|6 - InTransfer||1.33%|
|7 - OutTransfer||1.33%|
Given the previous numbers stated, each block will be able to accommodate:
votesand [ 8
votesand [ 48
votesand [ 88
tx(a) is a type
0 - Transfer (basic) transaction,
tx(b) is a type
0 - Transfer (2nd signature) transaction, and
tx(c) is a type
0 - Transfer (2nd + data field) transaction. A type
0 - Transfer (2nd signature) transaction and a
0 - Transfer (basic + data field) transaction are almost the same size, therefore only one is considered. Here votes are assumed to be a transaction containing 33 votes.
- 1000 users will require at least 27 minutes (166 blocks) to change a third of their votes (33 votes per vote transaction). They will require approximately 90 minutes to change all of their votes.
To get this data, the size of the block header (112 bytes) is ignored, which can imply an extra 300MB of yearly blockchain growth. We have excluded other transaction types from the previous table because they are either one-time transactions per account (delegate registration, second signature registration, etc), or seldom used transaction types (dapp registration, in/out transfer). In the latter case, depending on future development of these transaction types, they can become more common, but the size is assumed to be equivalent to a type
0 - Transfer (basic) transaction.
Moreover, we can compare the maximum transaction throughput per day for current and proposed implementations:
Throughput Comparison (Assuming Full Blocks)
|Tx type||Tx per day (current)||Tx per day (after)||Change|
|0 - Transfer (basic)||216,000||1,080,000||+500%|
|0 - Transfer (2nd + data field)||216,000||589,090||+272%|
|1 - Second signature||216,000||869,798||+402%|
|2 - Delegate||216,000||644,776||+298%|
|3 - Vote||216,000||55,717||-387%|
|4 - Multisignature||216,000||105,968||+49%|
Transaction Ratio Per Block Comparison
To support the previous table, it is also interesting to take note of historical data from the Lisk mainnet:
|Tx Type||Tx Count (Current Avg)||Ratio||Proposed Capacity|
|0 - Transfer||2.04184768||95.422%||65|
|1 - Second signature||0.00438445||0.205%||0.21|
|2 - Delegate||0.00200557||0.094%||0.07|
|3 - Vote||0.09154353||4.278%||0.28|
|4 - Multisignature||0.00001629||0.001%||0.00012264|
|5 - Dapp||0.00000376||0%||0.0|
|6 - InTransfer||0.00000000||0%||0.0|
|7 - OutTransfer||0.00000251||0%||0.0|
Where we can see that type
0 - Transfer transactions account for more than 95% of all transactions in the blockchain. This implies that, on average and assuming full blocks, 65 balance transactions with 2nd signature and full data field will be included in each block (proposed capacity) in the future.
We believe that given these numbers, 15KB is the most adequate choice in terms of network requirements (node bandwidth, blockchain growth, etc) vs network capabilities (transaction per second, votes per hour, etc). Moreover, it could be easily extended in the future if the network usage requires it.
General Implementation Logic
The payload of a block must not weigh more than 15KB. If a delegate generates a bigger block, it should be deemed invalid.
When filling a new block with unconfirmed transactions from the pool, the node should optimize the usage of the available space in the block versus available transactions in the pool. For example, when there is less than 2KB available for the next block (a vote transaction is 2.3KB), the delegate should fill the remaining space using smaller transactions, even though the next transaction in the queue in terms of fee or relative age would be a vote transaction. In this LIP, we propose a basic algorithm to cater for this scenario (please refer to 3. of Implementation Details. However, it is for each delegate to decide whether to implement a different algorithm should they believe it to be more optimal.
When a delegate generates a block, it should fetch the unconfirmed transactions from the pool and verify their size again.
Maintain a static counter for size and update it on every call of the following methods
remove<Queue>Transaction. For example,
Or call in realtime the
getBytesfunction on each transaction within the queue.
getListItem(index = 0)to
logic.transactionPoolwhich will return a transaction within the list at a given index. By default, it should return the transaction at the top of the list.
logic.transactionPoolusing the proposed algorithm.
modules.transactions.getUnconfirmedTransactionListwithout a limit to get all the unconfirmed transactions. As those transactions should always be approximately equal to 15KB.
Remove the redundant
numberOfTransactionscheck from blocks verification for historical data, as it is already included in block's signature.
Remove the redundant
payloadLengthcheck from blocks verification for historical data (up to 1048576KB), as it is already included in block's signature.
maxPayloadLengthchecks up to 15KB for recent blocks.
Algorithm for logic.transactionPool#fillPool
algorithm fillPool input: No output: No Set unconfirmedSize to logic.transactionPool.getListByteSize('Unconfirmed') Set newTransactions to empty array While unconfirmedSize < 15KB do Call pickTransactionFromList('Multisignature', 15KB - unconfirmedSize) Push transaction to newTransactions Array Set unconfirmedSize = unconfirmedSize + transaction.getBytes() While unconfirmedSize < 15KB do Call pickTransactionFromList('Queued', 15KB - unconfirmedSize) Push transaction to newTransactions Array Set unconfirmedSize = unconfirmedSize + transaction.getBytes() Call applyUnconfirmedList(newTransactions) return procedure pickTransactionFromList input: listName - Name of the list from which to pick transaction input: maxAllowedByteSize - Maximum byte size of transaction to pick input: skipIndex - Skip the top N transactions default to 0 output: selectedTransaction Set count to logic.transactionPool.count<listName>() If count <= skipIndex return null Set selectedTransaction to getListItem(listName, skipIndex) If selectedTransaction.getBytes() < maxAllowedByteSize Call logic.transactionPool.remove<listName>Transaction(selectedTransaction.id) return selectedTransaction Else Call pickTransactionForPool(listName, maxAllowedByteSize, skipIndex + 1)
For completeness, we add here a related implementation to this proposal which consists of refactoring the
transactionPool for multisignature transactions:
- Only pending multisignature transactions should stay in the
- As soon as the multisignature transaction is ready it should be pushed to
- In the previous step, we only look into the
queuedlist to fill the pool while generating blocks.
- This proposal will cause a hard fork in the network.
- Once implemented, blocks with more than 25 transactions will be valid, and blocks with more than 7 voting transactions will be invalid.
- The proposed changes will not imply any change in the blocks schema. It will contain the same fields as the current implementation.
- There is no need to add any exception validation logic, as the
numberOfTransactionproperty is already part of the block signature. So validating previous blocks will work before.