Update storage docs for ink! v4 #137
Conversation
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
Signed-off-by: Cyrill Leutwiler <bigcyrill@hotmail.com>
There was a problem hiding this comment.
Hi! There is a similar documentation for the new ink! storage organization available. It goes into details on how the pre-ink! 4 storage worked, comparing the changes with the new version. Perhaps we could merge some parts of it into this PR?
|
|
||
| ::: | ||
|
|
||
| ## Manual vs. Automatic Key generation |
There was a problem hiding this comment.
There is a slightly more detailed explanation on how compile-time storage keys work at the mentioned document
|
|
||
| Beware, this page is no longer up to date for 4.0! | ||
| ::: | ||
| ## Using custom types on storage |
There was a problem hiding this comment.
Perhaps we could merge the "Pre-ink! v4.0.0-beta storage" chapter here, to provide information about how new traits differ from the old traits?
| similar to traditional hash tables and comparable to the `mapping` type Solidity offers. | ||
| As a core ingredient to the ink! language, its main advantage is being simple and | ||
| lightweight. However, it does not provide any high-level functionality, such as iteration | ||
| or automatic clean-up. Smart contract authors will need to implement any high level |
There was a problem hiding this comment.
How do you mean "automatic clean-up" here? I'm asking because we remove the storage entry on calls of Mapping::remove and Mapping::take, which is my understanding of an automatic clean-up.
There was a problem hiding this comment.
Hmm yes true, I overlooked this as I just took it on from the existing version. Need to go over this again
There was a problem hiding this comment.
It also now came to my attention that here we provide iteration as something that can be implement but later I say it is not advisable 🐒
There was a problem hiding this comment.
I made an effort to document how our storage works to a full extend, please have a look again
versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md
Outdated
Show resolved
Hide resolved
|
|
||
| Using `ManualKey` instead of `AutoKey` might be especially desireable for upgradable | ||
| contracts, as using `AutoKey` might result in a different storage key for the same field | ||
| in a newer version of the contract. Which will break the contract after an upgrade! |
There was a problem hiding this comment.
Just took a look and we actually don't use ManualKey in our ink/examples/upgradeable-contracts/ yet.
Hi @ivan770 I am aware of this example being worked on, however I thought that the documentation should mainly describe how the storage works in the current version. My plan was to link this example somewhere once it gets merged. What do you think? |
Co-authored-by: Michael Müller <mich@elmueller.net>
…yout.md Co-authored-by: Michael Müller <mich@elmueller.net>
versioned_docs/version-4.0.0-alpha.1/datastructures/storage-layout.md
Outdated
Show resolved
Hide resolved
…yout.md Co-authored-by: Michael Müller <mich@elmueller.net>
…-metadata.md Co-authored-by: Michael Müller <mich@elmueller.net>
…yout.md Co-authored-by: Michael Müller <mich@elmueller.net>
…-metadata.md Co-authored-by: Michael Müller <mich@elmueller.net>
…yout.md Co-authored-by: Michael Müller <mich@elmueller.net>
…-metadata.md Co-authored-by: Michael Müller <mich@elmueller.net>
…yout.md Co-authored-by: Michael Müller <mich@elmueller.net>
…-metadata.md Co-authored-by: Michael Müller <mich@elmueller.net>
…-metadata.md Co-authored-by: Michael Müller <mich@elmueller.net>
| Any custom type wanting to be compatible with ink! storage must implement the | ||
| [`Storable`](https://docs.rs/ink_storage_traits/4.0.0-beta/ink_storage_traits/trait.Storable.html) | ||
| trait, so it can be SCALE | ||
| [`encoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Encode.html) | ||
| and | ||
| [`decoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Decode.html). |
There was a problem hiding this comment.
I'd leave the simple derive example for now and do a follow-up with the manual implementation. This PR is already big enough as is
|
|
||
| A basic example of a custom struct is shown below: | ||
| Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for you. If there is no need to implement any special behaviour, the above code example |
| ``` | ||
|
|
||
| ### Calcualte the `PrefixedStorageKey` from the child trie ID | ||
| A [`PrefixedStorageKey`](https://docs.rs/sp-storage/10.0.0/sp_storage/struct.PrefixedStorageKey.html) |
There was a problem hiding this comment.
Since we linked to the docs earlier we can get rid of this link
versioned_docs/version-4.0.0-alpha.1/datastructures/storage-in-metadata.md
Outdated
Show resolved
Hide resolved
| We observe that the storage layout is represented as a tree, where tangible storage values end up | ||
| inside a `leaf`. Because of `Packed` encoding, leafs can share the same storage key, and | ||
| in order to reach them you'd need fetch and decode the whole storage cell under this key. | ||
|
|
There was a problem hiding this comment.
Maybe not for this PR, but I'd love to see a little ASCII diagram showing the storage layout. E.g
Root
/ \
Struct Leaf
/ \
Leaf Leaf
I'd find it helpful for better visualizing the storage
There was a problem hiding this comment.
Good idea! I think our illustrator is already working on some kind of storage layout visualization
| "root": { | ||
| "layout": { | ||
| "leaf": { | ||
| "key": "0xb1f4904e", | ||
| "ty": 2 | ||
| } |
There was a problem hiding this comment.
Not sure in how much detail we want to talk about this in the docs, but there are more keys than just root and leaf available to describe items in storage. E.g we can have a root key that contains an array or enum layout
There was a problem hiding this comment.
Hmm yeah, I think it is a good point but I failed to bring up an example covering everything that is also small and easy to digest...
| On the other hand, this can get problematic if we're storing a large `ink::prelude::vec::Vec` | ||
| in the contract storage but provide messages that do not need to read and write from this | ||
| `ink::prelude::vec::Vec`. In that scenario, each and every contract message bears runtime | ||
| overhead by dealing with that `ink::prelude::vec::Vec`, regardless whether they access it or | ||
| not. This results in extra gas costs. To solve this problem we need to turn our storage | ||
| into a non-packed layout somehow. |
There was a problem hiding this comment.
I'd only use the full import path once and then refer to it as Vec in later parts where its clear from the context that we're referring to the same Vec
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
| Any custom type wanting to be compatible with ink! storage must implement the | ||
| [`Storable`](https://docs.rs/ink_storage_traits/4.0.0-beta/ink_storage_traits/trait.Storable.html) | ||
| trait, so it can be SCALE | ||
| [`encoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Encode.html) | ||
| and | ||
| [`decoded`](https://docs.rs/parity-scale-codec/3.2.2/parity_scale_codec/trait.Decode.html). |
There was a problem hiding this comment.
Yeah, I didn't mean to describe it fully here, only to mention that it has 3 variants of the implementation=)
I guess in the future we need to add something like "More detailed about it [here](And here link to Artem's&Ivan's documentation)"
| Circumventing this restriction by storing populated keys inside an `ink_prelude::Vec` might | ||
| not always be advisable: As accessing a storage cell is relatively expensive, this might | ||
| result in very high gas costs for large mappings. |
There was a problem hiding this comment.
I don't like it here, because it is a big topic. If you use Vec, and you have a lot of keys, it will break your contract, it will not fit into the decode buffer. We don't need to propose a solution to the iterator problem.
Also, we plan to support iterating over elements of Mapping, because of blake2_128_concat, in the future.
I would prefer to keep only the line "you can't iterate over the mapping". Or "you can't iterate over the mapping, but it is possible to implement this behaviour in another way [](link to the article/section/discussion about it)"
versioned_docs/version-4.0.0-alpha.1/datastructures/overview.md
Outdated
Show resolved
Hide resolved
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: xermicus <cyrill@parity.io>
| or to serve as the base key for calculating the actual keys needed to access values in | ||
| non-`Packed` fields (such as `Mapping`s). | ||
| Layouts under a root key can contain `leaf`s directly, but also `struct`, `enum` or `array` | ||
| layout, which are used |
There was a problem hiding this comment.
The sentence cut-off here is strange.
There was a problem hiding this comment.
This wasn't supposed to go in 🙈
Signed-off-by: xermicus <cyrill@parity.io>
|
|
||
| A basic example of a custom struct is shown below: | ||
| Even better: there is a macro `#[ink::storage_item]`, which derives all necessary traits for |
There was a problem hiding this comment.
Can you please add some notes about Packed/Non-Packed values because it is important to understand the context when this is useful to use?
There was a problem hiding this comment.
I'm not sure I understand, there's a whole chapter Packed vs Non-Packed layout in the storage layout docs explaining just that?
There was a problem hiding this comment.
True, so just add a few sentences that this attribute is preferred to be used for non-packed values:
From docs
If the type is non-packed it is best to rely on automatic storage key calculation via ink::storage_item
There was a problem hiding this comment.
Ah I see, thanks. Unfortunately the macro docs do not say why that it is the case, so I don't know what to write here. However I added this to be a link to the docs of the macro
There was a problem hiding this comment.
The #[ink::storage_item] macro is responsible for the storage key calculation for the non-Packed type with AutoKey. Without it, the key for non-Packed fields will be zero. I would say that the usage of this macro is mandatory if you don't plan to use ManualKey.
If you want to have your own implementation of the ink! storage traits for the non-Packed type, it still needs to use this macro only for the key calculation with #[ink::storage_item(derive = false)].
If your type can be Packed, then you need to avoid usage of this macro and derive scale::Encode and scale::Decode.
There was a problem hiding this comment.
Thanks @xgreenx, I added it as a note
Signed-off-by: xermicus <cyrill@parity.io>
Signed-off-by: xermicus <cyrill@parity.io>
|
Absolute legend. |
Update the storage docs for our upcoming version. The docs should give smart contract developers a better conceptual understanding and hint at some implications for contract development.
@xgreenx you're welcome to have a look as well