From 8313b3f2124def61fd7fdd7f0fa33bdc0582ea2a Mon Sep 17 00:00:00 2001 From: Francisco Date: Sat, 22 Jul 2023 18:54:25 +0200 Subject: [PATCH 01/17] Update eip-7201.md --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 6c9ae4ce440fa..8cacf9e832b09 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -28,7 +28,7 @@ A _namespace_ consists of a set of variables that are placed contiguous in the s A _namespace id_ is a string that uniquely identifies a namespace in a contract. It should not contain any whitespace characters. -The storage location for a namespace is defined as `ns_loc(id: string) = keccak256(uint256(keccak256(id)) - 1)`. +The storage location for a namespace is defined as `ns_loc(id: string) = align256(keccak256(uint256(align256(keccak256(id))) - 1))`, where `align256(x: bytes32) = x & ~0xff`. A Solidity contract using namespaced storage can annotate a struct with the NatSpec tag `@custom:storage-location erc7201:` to identify it as a namespace with id ``. _(Note: The Solidity compiler includes this annotation in the AST since v0.8.20, so this is recommended as the minimum compiler version when using this pattern.)_ From c6de07164fb767a4fa640620b153993e1c99074e Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Wed, 23 Aug 2023 16:22:36 -0300 Subject: [PATCH 02/17] simplify --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 8cacf9e832b09..7285d90be98cc 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -28,7 +28,7 @@ A _namespace_ consists of a set of variables that are placed contiguous in the s A _namespace id_ is a string that uniquely identifies a namespace in a contract. It should not contain any whitespace characters. -The storage location for a namespace is defined as `ns_loc(id: string) = align256(keccak256(uint256(align256(keccak256(id))) - 1))`, where `align256(x: bytes32) = x & ~0xff`. +The storage location for a namespace is defined as `ns_loc(id: string) = keccak256(uint256(keccak256(id)) - 1) & ~0xff`. A Solidity contract using namespaced storage can annotate a struct with the NatSpec tag `@custom:storage-location erc7201:` to identify it as a namespace with id ``. _(Note: The Solidity compiler includes this annotation in the AST since v0.8.20, so this is recommended as the minimum compiler version when using this pattern.)_ From 4205806f51358e782604c4eabab2565e474a3bed Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Wed, 23 Aug 2023 20:51:57 -0300 Subject: [PATCH 03/17] align to 256 --- EIPS/eip-7201.md | 15 +++++---------- 1 file changed, 5 insertions(+), 10 deletions(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 7285d90be98cc..488c66ecf4394 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -28,20 +28,15 @@ A _namespace_ consists of a set of variables that are placed contiguous in the s A _namespace id_ is a string that uniquely identifies a namespace in a contract. It should not contain any whitespace characters. -The storage location for a namespace is defined as `ns_loc(id: string) = keccak256(uint256(keccak256(id)) - 1) & ~0xff`. +The storage location for a namespace is defined as `ns_loc(id: string) = keccak256(keccak256(id) - 1) & ~0xff`. A Solidity contract using namespaced storage can annotate a struct with the NatSpec tag `@custom:storage-location erc7201:` to identify it as a namespace with id ``. _(Note: The Solidity compiler includes this annotation in the AST since v0.8.20, so this is recommended as the minimum compiler version when using this pattern.)_ ## Rationale -A requirement for the location is that it shouldn't overlap with any storage location that would be used by the standard Solidity layout. This is in case namespaced storage is used alongside standard linear storage, either deliberately or accidentally. +A requirement for the location is that it shouldn't overlap with any storage location that would be used by the standard Solidity layout. This is in case namespaced storage is used alongside standard linear storage, either deliberately or accidentally. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard linear storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. -First, note that a namespace may be larger than a single storage slot, so a variable in a namespace will be placed in a slot `ns_loc(id) + k`. If we assume collision resistance of Keccak-256, the chosen `ns_loc` function has the desired property with very high probability, because the cases in which a Solidity variable receives a location of the form `keccak256(x)` are: - -1. Dynamic Arrays: - 1. If the array is at the top level and is assigned to storage slot `n`, the location of the `k`th item in the array will be `keccak256(n) + k`, but `n` will be a number much smaller than `uint256(keccak256(id)) - 1`. - 2. If the array is within another array or mapping, it will be in some location `keccak256(x) + j`, and the `k`th item will be at `keccak256(keccak256(x) + j) + k`. For this to equal `ns_loc(x) + k` we would need `j = -1`, but `j` will always be a non-negative number in standard Solidity layout. -2. Mappings: The value for key `q` in a mapping will be at location `keccak256(h(q) . x)` where `x` is the location of the mapping itself, and `h` is as defined in the Solidity documentation (section "Layout of State Variables in Storage"). Note that `h(q)` can be any number of bytes. If it is a non-zero number of bytes, it is distinct from any `ns_loc(id) + k`. If it is zero bytes, it can be that `ns_loc(id) = keccak256(x)` if `x = keccak256(id) - 1`, but we know that `x` is the location of the mapping and (as mentioned for arrays above) a variable will be at `keccak256(y) + j` for a non-negative number `j`. +Additionally, namespace locations are aligned to 256 as a potential optimization, in anticipation of gas schedule changes after the Verkle state tree migration, which may cause groups of 256 storage slots to become warm all at once. ### Naming @@ -63,9 +58,9 @@ contract Example { uint256 y; } - // keccak256(abi.encode(uint256(keccak256("example.main")) - 1)); + // keccak256(abi.encode(uint256(keccak256("example.main")) - 1)) & ~bytes32(uint256(0xff)); bytes32 private constant MAIN_STORAGE_LOCATION = - 0x183a6125c38840424c4a85fa12bab2ab606c4b6d0e7cc73c0c06ba5300eab5da; + 0x183a6125c38840424c4a85fa12bab2ab606c4b6d0e7cc73c0c06ba5300eab500; function _getMainStorage() private pure returns (MainStorage storage $) { assembly { From 0a146be2b494b7cb6395abb2bd7447308bde82ea Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Wed, 23 Aug 2023 20:56:25 -0300 Subject: [PATCH 04/17] add note about structs outside contracts --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 488c66ecf4394..6a1c935a37713 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -30,7 +30,7 @@ A _namespace id_ is a string that uniquely identifies a namespace in a contract. The storage location for a namespace is defined as `ns_loc(id: string) = keccak256(keccak256(id) - 1) & ~0xff`. -A Solidity contract using namespaced storage can annotate a struct with the NatSpec tag `@custom:storage-location erc7201:` to identify it as a namespace with id ``. _(Note: The Solidity compiler includes this annotation in the AST since v0.8.20, so this is recommended as the minimum compiler version when using this pattern.)_ +A Solidity contract using namespaced storage can annotate a struct with the NatSpec tag `@custom:storage-location erc7201:` to identify it as a namespace with id ``. _(Note: The Solidity compiler includes this annotation in the AST since v0.8.20, so this is recommended as the minimum compiler version when using this pattern.)_ Structs with this annotation found outside of contracts are not considered to be namespaces for any contract in the source code. ## Rationale From 66433218174c4229f1b3df236c343788b72edc5c Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Wed, 23 Aug 2023 21:00:40 -0300 Subject: [PATCH 05/17] move to last call --- EIPS/eip-7201.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 6a1c935a37713..7e77cd4d18e5d 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -4,10 +4,11 @@ title: Namespaced Storage Layout description: A formula for the storage location of structs in the namespaced storage pattern. author: Francisco Giordano (@frangio), Hadrien Croubois (@Amxx), Ernesto García (@ernestognw), Eric Lau (@ericglau) discussions-to: https://ethereum-magicians.org/t/eip-7201-namespaced-storage-layout/14796 -status: Draft +status: Last Call type: Standards Track category: ERC created: 2023-06-20 +last-call-deadline: 2023-09-15 --- ## Abstract From 514e230bb6d94b0917388fd63e17bc65dd07d15c Mon Sep 17 00:00:00 2001 From: Hadrien Croubois Date: Thu, 24 Aug 2023 11:48:08 +0200 Subject: [PATCH 06/17] remove mention of linearity --- EIPS/eip-7201.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 7e77cd4d18e5d..34afd2839a326 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -13,19 +13,19 @@ last-call-deadline: 2023-09-15 ## Abstract -We define a formula to derive a location in storage, that is suitable for structs of arbitrary size, from an identifier. The formula is chosen to be safe against collisions with the standard Solidity storage layout. We define a convention to document this location in Solidity source code. +We define a formula to derive a location in storage, that is suitable for structs of arbitrary size, from an identifier. The formula is chosen to be safe against collisions with the storage layouts used by Solidity and Vyper. We define a convention to document this location in Solidity source code. ## Motivation -The default Solidity storage layout is linear: all state variables are assigned sequential locations starting from storage slot 0. This is sufficient for most contracts. However, various design patterns used in smart contract development can benefit from a non-linear storage layout. One example is a modular design where using `DELEGATECALL` a contract executes code from multiple contracts, all of which share the same storage space, and which have to carefully coordinate on how to use it. Another example is upgradeable contracts, where it can be difficult to add state variables in an upgrade given that they may affect the assigned storage location for the preexisting variables. +Smart contract languages such as Solidity and Vyper rely of tree-shaped storage layout. This tree starts at slot 0 and is composed of sequential chunks for consecutive variables. Hashes are used to ensure the chunks containing values of mappings and dynamic arrays do not collide. This is sufficient for most contracts. However, various design patterns used in smart contract development can benefit from a different storage layout. One example is a modular design where using `DELEGATECALL` a contract executes code from multiple contracts, all of which share the same storage space, and which have to carefully coordinate on how to use it. Another example is upgradeable contracts, where it can be difficult to add state variables in an upgrade given that they may affect the assigned storage location for the preexisting variables. -In a non-linear storage layout, instead of being assigned sequential locations starting from slot 0, state variables are spread out across the storage space, usually at pseudorandom locations obtained by hashing. Each value may be placed in an entirely different location, but more frequently values that are used together are put in a Solidity struct and co-located in storage. +Rather than using this default storage layout, specific state variables are sometimes spread out across the storage space, usually at pseudorandom locations obtained by hashing. Each value may be placed in an entirely different location, but more frequently values that are used together are put in a Solidity struct and co-located in storage. These pseudorandom locations can be the root of new storage trees that follow the same rules as the default one. Providing that this pseudorandom root is constructed so that it is not part of the default tree, this should result in the definition of independent spaces that do not collide with one another or with the default one. -These storage usage patterns are invisible to the Solidity compiler because they are not represented as Solidity state variables. Smart contract tools like static analyzers or blockchain explorers often need to know the storage location of contract data. Standardizing the location for non-linear storage layouts will allow these tools to correctly interpret contracts where these design patterns are used. +These storage usage patterns are invisible to the Solidity and Vyper compilers because they are not represented as Solidity state variables. Smart contract tools like static analyzers or blockchain explorers often need to know the storage location of contract data. Standardizing the location for storage layouts will allow these tools to correctly interpret contracts where these design patterns are used. ## Specification -A _namespace_ consists of a set of variables that are placed contiguous in the storage layout of a contract. It should be implemented as a struct. +A _namespace_ consists of a set of variables that are placed contiguous in the storage layout of a contract. It should be implemented as a struct. Namespaces can contain dynamic arrays and mappings for which the values would not be stored contiguously but would be derived following the sames rules as the default storage layout. A _namespace id_ is a string that uniquely identifies a namespace in a contract. It should not contain any whitespace characters. @@ -35,7 +35,7 @@ A Solidity contract using namespaced storage can annotate a struct with the NatS ## Rationale -A requirement for the location is that it shouldn't overlap with any storage location that would be used by the standard Solidity layout. This is in case namespaced storage is used alongside standard linear storage, either deliberately or accidentally. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard linear storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. +A requirement for the location is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper, nor should it be part of the storage tree derived from any other namespace. This is so that multiple namespaced storages can be used alongside each other and alongside the standard storage layout, either deliberately or accidentally. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. Additionally, namespace locations are aligned to 256 as a potential optimization, in anticipation of gas schedule changes after the Verkle state tree migration, which may cause groups of 256 storage slots to become warm all at once. From 2508f5c7f5bdd1d59482f3c44c38a41771701ddc Mon Sep 17 00:00:00 2001 From: Hadrien Croubois Date: Thu, 24 Aug 2023 12:08:02 +0200 Subject: [PATCH 07/17] adding a grammar definition to the Rationale section --- EIPS/eip-7201.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 34afd2839a326..3b63f7e56dbf9 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -35,7 +35,19 @@ A Solidity contract using namespaced storage can annotate a struct with the NatS ## Rationale -A requirement for the location is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper, nor should it be part of the storage tree derived from any other namespace. This is so that multiple namespaced storages can be used alongside each other and alongside the standard storage layout, either deliberately or accidentally. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. +The tree-shaped storage layout used by Solidity and Vyper follows the following grammar (with root=0): + +$ +\begin{align} +L_{root} := & \; root && \nonumber \text{--- root of the tree} \\ + & |\ L_{root} + k && \nonumber \text{--- consecutive values, structures and fixed size arrays} \\ + & |\ keccak256(L_{root}) && \nonumber \text{--- dynamic arrays} \\ + & |\ keccak256(H(q) \oplus L_{root}) && \nonumber \text{--- mappings (Solidity)} \\ + & |\ keccak256(L_{root} \oplus H(q)) && \nonumber \text{--- mappings (Vyper)} \\ +\end{align} +$ + +A requirement for the root is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper (root = 0), nor should it be part of the storage tree derived from any other namespace (another root). This is so that multiple namespaced storages can be used alongside each other and alongside the standard storage layout, either deliberately or accidentally. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. Additionally, namespace locations are aligned to 256 as a potential optimization, in anticipation of gas schedule changes after the Verkle state tree migration, which may cause groups of 256 storage slots to become warm all at once. From 6b00dbd6444fbda8657368a974bea496c640fb67 Mon Sep 17 00:00:00 2001 From: Hadrien Croubois Date: Thu, 24 Aug 2023 12:11:19 +0200 Subject: [PATCH 08/17] texttt{keccak256} --- EIPS/eip-7201.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 3b63f7e56dbf9..a9f3eb7c641f7 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -39,11 +39,11 @@ The tree-shaped storage layout used by Solidity and Vyper follows the following $ \begin{align} -L_{root} := & \; root && \nonumber \text{--- root of the tree} \\ - & |\ L_{root} + k && \nonumber \text{--- consecutive values, structures and fixed size arrays} \\ - & |\ keccak256(L_{root}) && \nonumber \text{--- dynamic arrays} \\ - & |\ keccak256(H(q) \oplus L_{root}) && \nonumber \text{--- mappings (Solidity)} \\ - & |\ keccak256(L_{root} \oplus H(q)) && \nonumber \text{--- mappings (Vyper)} \\ +L_{root} := & \; root && \nonumber \text{--- root of the tree} \\ + & |\ L_{root} + k && \nonumber \text{--- consecutive values, structures and fixed size arrays} \\ + & |\ \texttt{keccak256}(L_{root}) && \nonumber \text{--- dynamic arrays} \\ + & |\ \texttt{keccak256}(H(q) \oplus L_{root}) && \nonumber \text{--- mappings (Solidity)} \\ + & |\ \texttt{keccak256}(L_{root} \oplus H(q)) && \nonumber \text{--- mappings (Vyper)} \\ \end{align} $ From 9061f1e65ff591d8562a52dbc3a984add268d509 Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 15:17:30 -0300 Subject: [PATCH 09/17] edit motivation --- EIPS/eip-7201.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index a9f3eb7c641f7..9a3cab1009061 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -17,9 +17,9 @@ We define a formula to derive a location in storage, that is suitable for struct ## Motivation -Smart contract languages such as Solidity and Vyper rely of tree-shaped storage layout. This tree starts at slot 0 and is composed of sequential chunks for consecutive variables. Hashes are used to ensure the chunks containing values of mappings and dynamic arrays do not collide. This is sufficient for most contracts. However, various design patterns used in smart contract development can benefit from a different storage layout. One example is a modular design where using `DELEGATECALL` a contract executes code from multiple contracts, all of which share the same storage space, and which have to carefully coordinate on how to use it. Another example is upgradeable contracts, where it can be difficult to add state variables in an upgrade given that they may affect the assigned storage location for the preexisting variables. +Smart contract languages such as Solidity and Vyper rely on tree-shaped storage layout. This tree starts at slot 0 and is composed of sequential chunks for consecutive variables. Hashes are used to ensure the chunks containing values of mappings and dynamic arrays do not collide. This is sufficient for most contracts. However, it presents a challenge for various design patterns used in smart contract development. One example is a modular design where using `DELEGATECALL` a contract executes code from multiple contracts, all of which share the same storage space, and which have to carefully coordinate on how to use it. Another example is upgradeable contracts, where it can be difficult to add state variables in an upgrade given that they may affect the assigned storage location for the preexisting variables. -Rather than using this default storage layout, specific state variables are sometimes spread out across the storage space, usually at pseudorandom locations obtained by hashing. Each value may be placed in an entirely different location, but more frequently values that are used together are put in a Solidity struct and co-located in storage. These pseudorandom locations can be the root of new storage trees that follow the same rules as the default one. Providing that this pseudorandom root is constructed so that it is not part of the default tree, this should result in the definition of independent spaces that do not collide with one another or with the default one. +Rather than using this default storage layout, these patterns can benefit from laying out state variables spread out across the storage space, usually at pseudorandom locations obtained by hashing. Each value may be placed in an entirely different location, but more frequently values that are used together are put in a Solidity struct and co-located in storage. These pseudorandom locations can be the root of new storage trees that follow the same rules as the default one. Provided that this pseudorandom root is constructed so that it is not part of the default tree, this should result in the definition of independent spaces that do not collide with one another or with the default one. These storage usage patterns are invisible to the Solidity and Vyper compilers because they are not represented as Solidity state variables. Smart contract tools like static analyzers or blockchain explorers often need to know the storage location of contract data. Standardizing the location for storage layouts will allow these tools to correctly interpret contracts where these design patterns are used. From c90cb6b67bd67a60c6e8fb247ab1856ea0f2e02e Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 15:21:32 -0300 Subject: [PATCH 10/17] update specification --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 9a3cab1009061..4aac4005e98d6 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -25,7 +25,7 @@ These storage usage patterns are invisible to the Solidity and Vyper compilers b ## Specification -A _namespace_ consists of a set of variables that are placed contiguous in the storage layout of a contract. It should be implemented as a struct. Namespaces can contain dynamic arrays and mappings for which the values would not be stored contiguously but would be derived following the sames rules as the default storage layout. +A _namespace_ consists of a set of ordered variables, some of which may be dynamic arrays or mappings, with its values laid out following the same rules as the default storage layout but rooted in some location that is not necessarily slot 0. It should be implemented as a struct. A _namespace id_ is a string that uniquely identifies a namespace in a contract. It should not contain any whitespace characters. From 9becbc1959dc2be98a8cc7ff72e72f024e2d5206 Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 15:25:41 -0300 Subject: [PATCH 11/17] update rationale --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 4aac4005e98d6..8a74f4c444765 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -47,7 +47,7 @@ L_{root} := & \; root && \nonumber \text{--- \end{align} $ -A requirement for the root is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper (root = 0), nor should it be part of the storage tree derived from any other namespace (another root). This is so that multiple namespaced storages can be used alongside each other and alongside the standard storage layout, either deliberately or accidentally. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. +A requirement for the root is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper (root = 0), nor should it be part of the storage tree derived from any other namespace (another root). This is so that multiple namespaces may be used alongside each other and alongside the standard storage layout, either deliberately or accidentally, without colliding. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. Additionally, namespace locations are aligned to 256 as a potential optimization, in anticipation of gas schedule changes after the Verkle state tree migration, which may cause groups of 256 storage slots to become warm all at once. From cffe40ad07fa3c64667a37911d2c8cc02bfbb6b1 Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 16:30:00 -0300 Subject: [PATCH 12/17] fix latex --- EIPS/eip-7201.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 8a74f4c444765..0a96d38948d1c 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -37,15 +37,7 @@ A Solidity contract using namespaced storage can annotate a struct with the NatS The tree-shaped storage layout used by Solidity and Vyper follows the following grammar (with root=0): -$ -\begin{align} -L_{root} := & \; root && \nonumber \text{--- root of the tree} \\ - & |\ L_{root} + k && \nonumber \text{--- consecutive values, structures and fixed size arrays} \\ - & |\ \texttt{keccak256}(L_{root}) && \nonumber \text{--- dynamic arrays} \\ - & |\ \texttt{keccak256}(H(q) \oplus L_{root}) && \nonumber \text{--- mappings (Solidity)} \\ - & |\ \texttt{keccak256}(L_{root} \oplus H(q)) && \nonumber \text{--- mappings (Vyper)} \\ -\end{align} -$ +$L_{root} := \mathit{root} \mid L_{root} + k \mid \texttt{keccak256}(L_{root}) \mid \texttt{keccak256}(H(q) \oplus L_{root}) \mid \texttt{keccak256}(L_{root} \oplus H(q))$ A requirement for the root is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper (root = 0), nor should it be part of the storage tree derived from any other namespace (another root). This is so that multiple namespaces may be used alongside each other and alongside the standard storage layout, either deliberately or accidentally, without colliding. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. From 3fd417bf4f3f5f1aafc8c3e7f557f6b2d8d2dfc8 Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 16:33:04 -0300 Subject: [PATCH 13/17] reorder frontmatter --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 0a96d38948d1c..565fea7b20b92 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -5,10 +5,10 @@ description: A formula for the storage location of structs in the namespaced sto author: Francisco Giordano (@frangio), Hadrien Croubois (@Amxx), Ernesto García (@ernestognw), Eric Lau (@ericglau) discussions-to: https://ethereum-magicians.org/t/eip-7201-namespaced-storage-layout/14796 status: Last Call +last-call-deadline: 2023-09-15 type: Standards Track category: ERC created: 2023-06-20 -last-call-deadline: 2023-09-15 --- ## Abstract From 49935ce34bfffd4bb72fccee65174fb7304b88e3 Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 16:43:12 -0300 Subject: [PATCH 14/17] simplify phrasing --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 565fea7b20b92..32fa68699e8e4 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -19,7 +19,7 @@ We define a formula to derive a location in storage, that is suitable for struct Smart contract languages such as Solidity and Vyper rely on tree-shaped storage layout. This tree starts at slot 0 and is composed of sequential chunks for consecutive variables. Hashes are used to ensure the chunks containing values of mappings and dynamic arrays do not collide. This is sufficient for most contracts. However, it presents a challenge for various design patterns used in smart contract development. One example is a modular design where using `DELEGATECALL` a contract executes code from multiple contracts, all of which share the same storage space, and which have to carefully coordinate on how to use it. Another example is upgradeable contracts, where it can be difficult to add state variables in an upgrade given that they may affect the assigned storage location for the preexisting variables. -Rather than using this default storage layout, these patterns can benefit from laying out state variables spread out across the storage space, usually at pseudorandom locations obtained by hashing. Each value may be placed in an entirely different location, but more frequently values that are used together are put in a Solidity struct and co-located in storage. These pseudorandom locations can be the root of new storage trees that follow the same rules as the default one. Provided that this pseudorandom root is constructed so that it is not part of the default tree, this should result in the definition of independent spaces that do not collide with one another or with the default one. +Rather than using this default storage layout, these patterns can benefit from laying out state variables across the storage space, usually at pseudorandom locations obtained by hashing. Each value may be placed in an entirely different location, but more frequently values that are used together are put in a Solidity struct and co-located in storage. These pseudorandom locations can be the root of new storage trees that follow the same rules as the default one. Provided that this pseudorandom root is constructed so that it is not part of the default tree, this should result in the definition of independent spaces that do not collide with one another or with the default one. These storage usage patterns are invisible to the Solidity and Vyper compilers because they are not represented as Solidity state variables. Smart contract tools like static analyzers or blockchain explorers often need to know the storage location of contract data. Standardizing the location for storage layouts will allow these tools to correctly interpret contracts where these design patterns are used. From bbd2353712c3f01c3f48631f942350164f893e32 Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 17:48:58 -0300 Subject: [PATCH 15/17] use n/k variables and fix vyper location --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 32fa68699e8e4..2a2733caa516d 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -37,7 +37,7 @@ A Solidity contract using namespaced storage can annotate a struct with the NatS The tree-shaped storage layout used by Solidity and Vyper follows the following grammar (with root=0): -$L_{root} := \mathit{root} \mid L_{root} + k \mid \texttt{keccak256}(L_{root}) \mid \texttt{keccak256}(H(q) \oplus L_{root}) \mid \texttt{keccak256}(L_{root} \oplus H(q))$ +$L_{root} := \mathit{root} \mid L_{root} + n \mid \texttt{keccak256}(L_{root}) \mid \texttt{keccak256}(H(k) \oplus L_{root}) \mid \texttt{keccak256}(L_{root} \oplus k)$ A requirement for the root is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper (root = 0), nor should it be part of the storage tree derived from any other namespace (another root). This is so that multiple namespaces may be used alongside each other and alongside the standard storage layout, either deliberately or accidentally, without colliding. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. From 3356f307100dd9054420ce6c7227da2b9b491ea2 Mon Sep 17 00:00:00 2001 From: Francisco Giordano Date: Thu, 24 Aug 2023 18:08:26 -0300 Subject: [PATCH 16/17] fix vyper formula --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index 2a2733caa516d..b64c241a7f83b 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -37,7 +37,7 @@ A Solidity contract using namespaced storage can annotate a struct with the NatS The tree-shaped storage layout used by Solidity and Vyper follows the following grammar (with root=0): -$L_{root} := \mathit{root} \mid L_{root} + n \mid \texttt{keccak256}(L_{root}) \mid \texttt{keccak256}(H(k) \oplus L_{root}) \mid \texttt{keccak256}(L_{root} \oplus k)$ +$L_{root} := \mathit{root} \mid L_{root} + n \mid \texttt{keccak256}(L_{root}) \mid \texttt{keccak256}(H(k) \oplus L_{root}) \mid \texttt{keccak256}(L_{root} \oplus H(k))$ A requirement for the root is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper (root = 0), nor should it be part of the storage tree derived from any other namespace (another root). This is so that multiple namespaces may be used alongside each other and alongside the standard storage layout, either deliberately or accidentally, without colliding. The term `keccak256(id) - 1` in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into `keccak256(id) + n`, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large. From 26a017bc95ebb222230b4b60ff93644be5254996 Mon Sep 17 00:00:00 2001 From: Francisco Date: Thu, 7 Sep 2023 15:03:26 -0300 Subject: [PATCH 17/17] Update eip-7201.md --- EIPS/eip-7201.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7201.md b/EIPS/eip-7201.md index b64c241a7f83b..5888bf8cfdd85 100644 --- a/EIPS/eip-7201.md +++ b/EIPS/eip-7201.md @@ -5,7 +5,7 @@ description: A formula for the storage location of structs in the namespaced sto author: Francisco Giordano (@frangio), Hadrien Croubois (@Amxx), Ernesto García (@ernestognw), Eric Lau (@ericglau) discussions-to: https://ethereum-magicians.org/t/eip-7201-namespaced-storage-layout/14796 status: Last Call -last-call-deadline: 2023-09-15 +last-call-deadline: 2023-10-01 type: Standards Track category: ERC created: 2023-06-20