Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
SSTORE/SLOAD for byte arrays #97
NOTE: this is split off from #86, and updated to incorporate what was agreed on the dev call on 2016.04.25.
Code for SSTOREBYTES
BASE_COST = 2500 BYTE_STORAGE_COST = 250 BYTE_CHANGE_COST = 40 MIN_COST = 2500 ... elif op == 'SSTOREBYTES': key, mstart, msize = stk.pop(), stk.pop(), stk.pop() if not mem_extend(mem, compustate, op, mstart, msize): return vm_exception('OOG EXTENDING MEMORY') prev_adjbyte_count = len(ext.get_storage_data(msg.to, key)) if prev_adjbyte_count >= 0: prev_adjbyte_count += 32 post_adjbyte_count = msize + (32 if msize else 0) gas_cost = BASE_COST + BYTE_STORAGE_COST * (post_adjbyte_count - prev_adjbyte_count) + BYTE_CHANGE_COST * post_adjbyte_count gas_payment = max(MIN_COST, gas_cost) refund = gas_payment - gas_cost if compustate.gas < gas_payment: return vm_exception('OUT OF GAS') compustate.gas -= gas_payment ext.set_storage_data(msg.to, ''.join(map(chr, mem[mstart: mstart + msize])))
This allows contracts to store data more efficiently by reducing the number of trie/DB writes and reads needed for contract execution. Recent measurements show that trie and DB writes are a primary source of overhead, and serenity tests show a >2x speed improvement in some cases if code is written well. The canonical examples include:
Additionally, compiler logic may be simplified as storing data in sequential slots is no longer required. This does come at the cost of requiring updates to blockchain tools that currently assume that all storage values are either empty or exactly 32 bytes long.
The rationale behind adding a 32 byte cost to nonempty storage slots is that even a one-byte storage value will necessarily have overhead including key/value storage size costs, merkle tree branches/leaves, etc; additionally, an incentive is required to encourage saving a constant 320-byte chunk in a single key rather than splitting it up among 10 keys.
referenced this issue
Apr 26, 2016
Possible implementation concern. Leveldb doesn't handle large values very well. To get around this, break large values into chunks and append nonce to the key. Since leveldb sorts key lexicographical it very efficient to stream the chunks. Leveldb has lg and lt operations, so the query needed here would be
If you look at the bytes per second from that benchmark, you get 779k writes/sec * 100 bytes/write = 77.9 mbps for small values and 1100 writes/sec * 100000 bytes/write = 110 mbps for large values. So it doesn't seem to be fatal or even that problematic.
If this is adopted and should be usable from Solidity, it will add another layer of complexity. This "store as single blob" has to be an optional feature for structs and arrays, because otherwise, elements of the structs and arrays are not accessible as lvalues anymore.
Furthermore, the fact that storage size is dynamic will need some additional handling: If sloadbytes does not write to the full range of memory, we always have to check the size beforehand and optionally even throw if the size is not what we expect.
Finally, as already commented on the original issue, I think that having two "types" of storage slots will add similar complexity, although I think that this is not too big an issue.
Are there any other rationales behind this EIP besides optimization?