EIP 104 (Serenity): Unlimited storage key or value size

[vbuterin]

Instead of storage keys and values being 32 bytes, they have unlimited size. We could have opcodes:

• SSTORE(key_mstart, key_msz, value_mstart, value_msz)
• SLOAD(key_mstart, key_msz, value_output_mstart, value_output_maxsize)

SLOAD can have gas cost equal to 40 + len(key) + len(min(value_output_maxsize, size_of_actual_output))

The gas cost for SSTORE can be:

def compute_gascost(key_length, value_length, prev_value_length):
    return 2500 + 10 * (key_length + value_length) \
        + (5000 + 40 * (key_length + value_length) if value_length else 0) \
        - (5000 + 40 * (key_length + prev_value_length) if prev_value_length else 0)

Where key_length is the length of the key, value_length is the length of the value, and prev_value_length is the length of the previous value at that key. If the function returns a value less than 2500, the difference is instead accounted for as a refund (eg. if compute_gascost returns 1300, that's a 2500 cost plus a 1200 refund, if it returns -700 that's a 2500 cost plus a 3200 refund, etc).

Motivations

• Works well with a possible EVM 2.0 architecture which is based on 64 bit integers and where larger values are treated indiscriminately as byte slices, allowing easier use of RSA-based crypto and large-sized elliptic curves on the same footing as 256-bit ECC. If we go this route, there is no reason why the 32-byte length needs to be privileged
• Works well with an architecture where code is part of storage (eg. code = storage at key index '' [empty string])
• Allows for quick "shutdown and reboot" functionality where a VM can quickly stop computation in mid-step and save its entire memory to a standardized index in one step, and then quickly reload from there the next time a transaction is called. This is useful for (i) dapps that need to sometimes perform large batch operations that may take up more than the gaslimit, eg. a crowdfunding or auction contract refunding very many participants, (ii) security deposit based auditable computation markets, (iii) generally more efficiency by batching variable lookups, requiring fewer moderately expensive trie operations.

[chriseth]

Downsides I can see here:

1. every single SLOAD and SSTORE requires memory access - but they are expensive anyway
2. it might be worthwhile to also provide SSIZE which returns the size of the value at a given key
3. it might be more complex to implement

My main concerns are that we lose some kind of type safety: If your contract assumes that you only store 16 bytes at a key, but it actually stores more, then loading from it will truncate the value. Note that our high-level-language already handle storing variably-sized types quite efficiently.

[vbuterin]

If your contract assumes that you only store 16 bytes at a key, but it actually stores more, then loading from it will truncate the value.

Can't this be easily handled at the HLL level? If you are dealing with a 16-byte data type, then your code should not try storing more than 16 bytes there. I'll accept the idea of SSIZE though.

Regarding efficiency concerns, I actually think that this will increase efficiency far more than it will decrease it. Every SLOAD/SSTORE requires trie operations and in the case of SSTORE that includes hashing, and so if you use solidity's current trick for storing large data types in storage you will end up losing a lot of time on multiple trie reorganizations and particularly on the MLOADs, JUMPs, etc involved in processing the loop; all this is much greater than the requirement to do an extra MLOAD and PUSH.

[wanderer]

Less trie lookups is a big win.

[chriseth]

Sure, caring about using the same size for data can be handled at a higher level, but it adds some level of danger nevertheless.

I agree about efficiency on the trie level for storing e.g. long strings or multiple array elements at the same time, but I'm not sure if this is one of the main use cases of storage. Initially, you might store a long array, but most of the time you probably read and modify single elements. And if you store large arrays (including byte arrays and strings) in a single slot, you have to read and write the whole slot if you only change a single element. This gets more and more expensive the longer the array is, so a higher level language might determine a length where the array is chopped into smaller pieces, at which point we again arrive at the situation we currently are in (especially there will always be loops when copying and slot calculations when accessing) just with larger constants and more complex gas calculations.

I am not really opposed to this change - not forcing a word size is a good thing - I just think that it is not that simple.

[VoR0220]

I am in favor of this.

[axic]

@vbuterin:

SLOAD(key_mstart, key_msz, value_output_mstart, value_output_maxsize)

If there's a limit of the read size, perhaps it is useful to also include a start offset:

SLOAD(key_mstart, key_msz, value_output_mstart, value_output_offset, value_output_maxsize)

And in that case it probably would make sense adding that to SSTORE too, which then probably provides an aid for the issue raised by @chriseth.

[axic]

@vbuterin is #97 obsoleting this one?

[github-actions]

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

[github-actions]

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.