New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eth_getProof #1186

Open
simon-jentzsch opened this Issue Jun 28, 2018 · 12 comments

Comments

Projects
None yet
5 participants
@simon-jentzsch
Contributor

simon-jentzsch commented Jun 28, 2018


eip: 1186
title: eth_getProof
author: Simon Jentzsch simon.jentzsch@slock.it, Christoph Jentzsch christoph.jentzsch@slock.it
discussions-to: simon.jentzsch@slock.it
status: Draft
type: Standards Track (Core, Networking, Interface, ERC)
category : Interface
created: 2018-06-24

Simple Summary

One of the great features of Ethereum is the fact, that you can verify all data of the state. But in order to allow verification of accounts outside the client, we need an additional function delivering us the required proof. These proofs are important to secure Layer2-Technologies.

Abstract

Ethereum uses MerkleTrees to store the state of accounts and their storage. This allows verification of each value by simply creating a MerkleProof. But currently, the eth-Module in the RPC-Interface does not give you access to these proofs. This EIP suggests an additional RPC-Method, which creates MerkleProofs for Accounts and Storage-Values.

Combined with a stateRoot (from the blockheader) it enables offline verification of any account or storage-value. This allows especially IOT-Devices or even mobile apps which are not able to run a light client to verify responses from an untrusted source only given a trusted blockhash.

Motivation

In order to create a MerkleProof access to the full state db is required. The current RPC-Methods allow an application to access single values (eth_getBalance,eth_getTransactionCount,eth_getStorageAt,eth_getCode), but it is impossible to read the data needed for a MerkleProof through the standard RPC-Interface. (There are implementations using leveldb and accessing the data via filesystems, but this can not be used for production systems since it requires the client to be stopped first - See https://github.com/zmitton/eth-proof)

Today MerkleProofs are already used internally. For example, the Light Client Protocol supports a function creating MerkleProof, which is used in order to verify the requested account or storage-data.

Offering these already existing function through the RPC-Interface as well would enable Applications to store and send these proofs to devices which are not directly connected to the p2p-network and still are able to verify the data. This could be used to verify data in mobile applications or IOT-devices, which are currently only using a remote client.

Specification

As Part of the eth-Module, an additional Method called eth_getProof should be defined as follows:

eth_getProof

Returns the account- and storage-values of the specified account including the Merkle-proof.

Parameters
  1. DATA, 20 Bytes - address of the account.
  2. ARRAY, 32 Bytes - array of storage-keys which should be proofed and included. See eth_getStorageAt
  3. QUANTITY|TAG - integer block number, or the string "latest" or "earliest", see the default block parameter
Returns

Object - A account object:

  • balance: QUANTITY - the balance of the account. See eth_getBalance

  • codeHash: DATA, 32 Bytes - hash of the code of the account. For a simple Account without code it will return "0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470"

  • nonce: QUANTITY, - nonce of the account. See eth_getTransactionCount

  • storageHash: DATA, 32 Bytes - SHA3 of the StorageRoot. All storage will deliver a MerkleProof starting with this rootHash.

  • accountProof: ARRAY - Array of rlp-serialized MerkleTree-Nodes, starting with the stateRoot-Node, following the path of the SHA3 (address) as key.

  • storageProof: ARRAY - Array of storage-entries as requested. Each entry is a object with these properties:

    • key: QUANTITY - the requested storage key
    • value: QUANTITY - the storage value
    • proof: ARRAY - Array of rlp-serialized MerkleTree-Nodes, starting with the storageHash-Node, following the path of the SHA3 (key) as path.
Example
{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "eth_getProof",
  "params": [
    "0x7F0d15C7FAae65896648C8273B6d7E43f58Fa842",
    [  "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421" ],
    "latest"
  ]
}

The result will look like this:

{
  "id": 1,
  "jsonrpc": "2.0",
  "result": {
    "accountProof": [
      "0xf90211a...0701bc80",
      "0xf90211a...0d832380",
      "0xf90211a...5fb20c80",
      "0xf90211a...0675b80",
      "0xf90151a0...ca08080"
    ],
    "balance": "0x0",
    "codeHash": "0xc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470",
    "nonce": "0x0",
    "storageHash": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
    "storageProof": [
      {
        "key": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
        "proof": [
          "0xf90211a...0701bc80",
          "0xf90211a...0d832380"
        ],
        "value": "0x1"
      }
    ]
  }
}

Rationale

This one Method actually returns 3 different important data points:

  1. The 4 fields of an account-object as specified in the yellow paper [nonce, balance, storageHash, codeHash ], which allows storing a hash of the account-object in order to keep track of changes.
  2. The MerkleProof for the account starting with a stateRoot from the specified block.
  3. The MerkleProof for each requested storage entry starting with a storageHash from the account.

Combining these in one Method allows the client to work very efficient since the required data are already fetched from the db.

Proofs for non existant values

In case an address or storage-value does not exist, the proof needs to provide enough data to verify this fact. This means the client needs to follow the path from the root node and deliver until the last matching node. If the last matching node is a branch, the proof value in the node must be an empty one. In case of leaf-type, it must be pointing to a different relative-path in order to proof that the requested path does not exist.

possible Changes to be discussed:

  • instead of providing the blocknumber maybe the blockhash would be better since it would allow proofs of uncles-states.
  • in order to reduce data, the account-object may only provide the accountProof and storageProof. The Fields balance, nonce, storageHash and codeHash could be taken from the last Node in the proof by deserializing it.

Backwards Compatibility

Since this only adds a new Method there are no issues with Backwards Compatibility.

Test Cases

Tests still need to be implemented, but the core function creating the proof already exists inside the clients and are well tested.

Implementation

We implemented this function for:

Copyright

Copyright and related rights waived via CC0.

@MicahZoltu

This comment has been minimized.

Contributor

MicahZoltu commented Jun 28, 2018

It feels like this should be paired with an eth_verifyProof method. Getting a proof with no easy way to verify it feels significantly less useful.

@simon-jentzsch

This comment has been minimized.

Contributor

simon-jentzsch commented Jun 28, 2018

a eth_verifyProof might be helpful, but not required since you can easily verify this inside you dapp. (or even outside). , you just call eth_getBlockBy... and take the stateRoot. and then verify the proof.

import * as Trie from 'merkle-patricia-tree'
import * as util from 'ethereumjs-util'

const [block, account ] = await Promise.all([
   // we need the blockheader to get the stateRoot 
  web3.eth.getBlockByNumber('latest',false),

  // and we need the proof
  web3.eth.getProof(address,[],'latest')
])

// this function simply verifies the proof by starting with the stateRoot from the header and hopefully end with the leaf-node containing the rlp-serialized value of the account
Trie.verifyProof( block.stateRoot, util.sha3 ( address ), account.accountProof, (err, value) => {
   if (err || !value.equals(util.rlp.encode(account.nonce,account.balance,account.storageHash, account.codeHash)))
       console.log('proof failed :',err)
   else 
       console.log('verified!')
})

But I think it would be a good idea to offer a function in the web3-library like:

web3.eth.verify.account( account,  blockHash )
web3.eth.verify.storage( account.storageProof, account.stateRoot )
@5chdn

This comment has been minimized.

Contributor

5chdn commented Jul 2, 2018

Please create PR, not an issue.

@simon-jentzsch

This comment has been minimized.

Contributor

simon-jentzsch commented Sep 21, 2018

thanks, just created the PR

@simon-jentzsch

This comment has been minimized.

Contributor

simon-jentzsch commented Sep 22, 2018

I also created an Reference-Implementation for geth now:

PR:ethereum/go-ethereum#17737 (Status: pending )
Docker: https://hub.docker.com/r/slockit/geth-in3/tags/

@zmitton

This comment has been minimized.

zmitton commented Nov 20, 2018

@simon-jentzsch is it (inconveniently) true that the intermediary nodes to the other trees (i.e. transactions & receipts) are not stored in levelDB (geth/parity)? In my library I currently do multiple RPC calls for all the transactions of the particular block and re-create the tree locally.

@zmitton

This comment has been minimized.

zmitton commented Nov 28, 2018

Seems like this is the case, so my library is still useful in building those proofs (and of course checking them).

@simon-jentzsch

This comment has been minimized.

Contributor

simon-jentzsch commented Nov 30, 2018

@zmitton yes, the other tries like transactions and receipts are only created temporarly since all data are available in the block, but I agree this means getting a merkle proof for a transactrionReceipt means running at least a bulk-request to get each receipt and construct the tree. (like here https://github.com/slockit/in3-server/blob/master/src/chains/proof.ts#L205 )
But at least these information are available. (I'm also thinking about caching these trees to optimize performance)

@juan794

This comment has been minimized.

juan794 commented Dec 4, 2018

a eth_verifyProof might be helpful, but not required since you can easily verify this inside you dapp. (or even outside). , you just call eth_getBlockBy... and take the stateRoot. and then verify the proof.

import * as Trie from 'merkle-patricia-tree'
import * as util from 'ethereumjs-util'

const [block, account ] = await Promise.all([
   // we need the blockheader to get the stateRoot 
  web3.eth.getBlockByNumber('latest',false),

  // and we need the proof
  web3.eth.getProof(address,[],'latest')
])

// this function simply verifies the proof by starting with the stateRoot from the header and hopefully end with the leaf-node containing the rlp-serialized value of the account
Trie.verifyProof( block.stateRoot, util.sha3 ( address ), account.accountProof, (err, value) => {
   if (err || !value.equals(util.rlp.encode(account.nonce,account.balance,account.storageHash, account.codeHash)))
       console.log('proof failed :',err)
   else 
       console.log('verified!')
})

But I think it would be a good idea to offer a function in the web3-library like:

web3.eth.verify.account( account,  blockHash )
web3.eth.verify.storage( account.storageProof, account.stateRoot )

I am not sure if it is an error or I am the only one experience it, but I think it is better to comment on it.

I am testing an implementation that requires offline existence verification of accounts. RLP decoding of the value in Trie.verifyProof, following the example above, takes account's balance (smart contract's in this case) as a string data type when the value is 0x0 which makes the verification fail. When I deposit some Ethers, the smart contract's balance is taken as an integer data type and the verification works ok. I am using Geth 1.18, NodeJS 8.10, and Rinkeby testnet.

@zmitton

This comment has been minimized.

zmitton commented Dec 4, 2018

@juan794 this doesn't sound like an issue with the EIP. From the above code I dont see RLP needing to be decoded but you might want to bring this up with the rlp repo or 'merkle-patricia-tree' (depending on your code which i havent seen)

@juan794

This comment has been minimized.

juan794 commented Dec 4, 2018

Thanks @zmitton. I used RLP to understand why the verification was not working. I thought it is liked to this EIP because it is the contract's balance which makes the verification works straightforwardly as the example above, but you are right, it is closer to an RLP problem itself.

@zmitton

This comment has been minimized.

zmitton commented Dec 5, 2018

@juan794 I've seen this issue before. the problem is that the number 0 is represented in ethereum as bytes<> not bytes<00> so the RLP of it becomes bytes<80> and not bytes<00> (which would be the rlp of bytes<00> because rlp of anything single byte under 80 is itself).

So find out find out where the software could use updating but I bet you its not here. the return value of 0 or empty from RPC is generally been string "0x0" I believe and so it should probably keep this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment