RPC missing mixHash and nonce fields in Header responses

[stevenroose]

Before filing a new issue, please provide the following information.

I'm running:

• Which Parity version?:

$ parity --version
Parity
  version Parity/v1.10.6-stable-bc0d134-20180605/x86_64-linux-gnu/rustc1.26.1
Copyright 2015, 2016, 2017, 2018 Parity Technologies (UK) Ltd
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

By Wood/Paronyan/Kotewicz/Drwięga/Volf
   Habermeier/Czaban/Greeff/Gotchac/Redmann

• Which operating system?: Linux
• How installed?: via installer
• Are you fully synchronized?: yes
• Which network are you connected to?: private Aura

I use the Go ethclient package to interface with the Parity RPC. When calling eth_getBlockByNumber, it complaints about missing the mixHash and nonce fields.

The same issue was reported before (~2 years ago), and then supposedly fixed: #3148

[Tbaut]

edit: you're talking about the header:

• transaction's nonce was present in my test (kovan) but not the block's nonce (PoW).
• the Ethereum documentation doesn't state that it returns any mixHash.

Can you point me to the source document where it states that this RPC should return it?

For the record:
curl --data '{"method":"eth_getBlockByNumber","params":["latest",true],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST localhost:8545

{
    "jsonrpc": "2.0",
    "result": {
        "author": "0x007733a1fe69cf3f2cf989f81c7b4cac1693387a",
        "difficulty": "0xfffffffffffffffffffffffffffffffe",
        "extraData": "0xd583010a068650617269747986312e32362e31826c69",
        "gasLimit": "0x79f37d",
        "gasUsed": "0x41cfa",
        "hash": "0xba66a7a3371062f5265039dbf8cd0d6cae00836ad87bff4549311ca96b6705a3",
        "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080001000000000020000000000000000000000000000000000000000000",
        "miner": "0x007733a1fe69cf3f2cf989f81c7b4cac1693387a",
        "number": "0x742153",
        "parentHash": "0x1bb362a835a24627246b10e5146900a801ea0ca685ce664ced7c0257904e66bc",
        "receiptsRoot": "0xcec7ab613d5c2d38802e9c38e7a6c6af0f9987500c6a67a25e38204818b40ad4",
        "sealFields": ["0x8416c698d4", "0xb841aeee675bcd99558e108e2abefcec192463a46deb06725d5039f20f4b106a0e266d4ccb8ee134bfac9972cd10653887214713267afb0f470cbf384a1158e6d8b800"],
        "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
        "signature": "aeee675bcd99558e108e2abefcec192463a46deb06725d5039f20f4b106a0e266d4ccb8ee134bfac9972cd10653887214713267afb0f470cbf384a1158e6d8b800",
        "size": "0x6f8",
        "stateRoot": "0x80b5ca154f42e24844996c5407c23f894e094be0f59c2bcd47d95dc52a2e8108",
        "step": "382114004",
        "timestamp": "0x5b1a6350",
        "totalDifficulty": "0x71de44000000000000000000000004851c3702",
        "transactions": [{
            "blockHash": "0xba66a7a3371062f5265039dbf8cd0d6cae00836ad87bff4549311ca96b6705a3",
            "blockNumber": "0x742153",
            "chainId": null,
            "condition": null,
            "creates": null,
            "from": "0x0011a6fc5bd53fd8e2f5b6c7c123629ef9185362",
            "gas": "0x4ef93",
            "gasPrice": "0xdf8475800",
            "hash": "0x12f6069821549820965b66a277507247783edb11c744baba61b1918ada8eda08",
            "input": "0x2c10e90c00000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000240000000000000000000000000000000000000000000000000000000000000000f000000000000000000000000a27af8713623fcc239d49108b1a7b187c133e88b000000000000000000000000dc5fc5dab642f688bc5bb58bef6e0d452d7ae12300000000000000000000000071be1386ec4d6d780b208529abc349ee69c241cd00000000000000000000000082789d9a0951974e58ebb91df13b84424e51acc4000000000000000000000000560848c7a470b9aa87f7283fdb31bd67737716600000000000000000000000008925daf23ad19c68ff317038c6c6018cfbb7070400000000000000000000000047eb525440e976387b80b41cb22cf4dc0a7e9db0000000000000000000000000e67e7ce765b67aa1dcd731458ed50c04bfc78d7400000000000000000000000089c6dd6a72e2e5838b1cfdc1059dad22ce4b6a4b000000000000000000000000fdbe0d69a110d1f0dc6aec8588804da88173b8f0000000000000000000000000b04985bfab8550c43c00528d9d06492916cb9df600000000000000000000000019044ff4c8f00b16ab660407abfffac8fd1ce83300000000000000000000000030c64c8d3e985d91f9c1efaf4ccde66b2925ea7c00000000000000000000000046c4b65f40f686c3f3738395c36acea70c06f3c2000000000000000000000000e6b8dc9357b42fa67a0ac9bdee59d4b6ebe11406000000000000000000000000000000000000000000000000000000000000000f0000000000000000000000000000000000000000000000000de0b6b3a764000000000000000000000000000000000000000000000000000000ed27d2729e070000000000000000000000000000000000000000000000000012208a9134e8b2000000000000000000000000000000000000000000000000000005ec7ca0b6fd00000000000000000000000000000000000000000000000000001094250242bf0000000000000000000000000000000000000000000000000000076909d5d36e00000000000000000000000000000000000000000000000000000197f665a87b000000000000000000000000000000000000000000000000000000000010713f4f0000000000000000000000000000000000000000000000000197018e7029ac0000000000000000000000000000000000000000000000000000414c2bf7269c000000000000000000000000000000000000000000000000000001e6159843ea0000000000000000000000000000000000000000000000000000e235aa5f86f2000000000000000000000000000000000000000000000000000000c37ea4e88f000000000000000000000000000000000000000000000000000007f06bfafc9a0000000000000000000000000000000000000000000000001fd25ea81f7f011c00",
            "nonce": "0xee4",
            "publicKey": "0x6a2fb6b38904021713b70aed45886792608014b8f498557a5a8596e91227bc931b4a8390a0037d1d74b6a17b4b045ab4601aad74eadee81710a2e681177c0849",
            "r": "0x7b43bb33b83e627bc67bad9c55eeb13f3ddada35d6c7d48403788a943fc9bac2",
            "raw": "0xf904ad820ee4850df84758008304ef93947d8a7b8c59af9a86bbab3e0daa171930fa857e8980b904442c10e90c00000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000240000000000000000000000000000000000000000000000000000000000000000f000000000000000000000000a27af8713623fcc239d49108b1a7b187c133e88b000000000000000000000000dc5fc5dab642f688bc5bb58bef6e0d452d7ae12300000000000000000000000071be1386ec4d6d780b208529abc349ee69c241cd00000000000000000000000082789d9a0951974e58ebb91df13b84424e51acc4000000000000000000000000560848c7a470b9aa87f7283fdb31bd67737716600000000000000000000000008925daf23ad19c68ff317038c6c6018cfbb7070400000000000000000000000047eb525440e976387b80b41cb22cf4dc0a7e9db0000000000000000000000000e67e7ce765b67aa1dcd731458ed50c04bfc78d7400000000000000000000000089c6dd6a72e2e5838b1cfdc1059dad22ce4b6a4b000000000000000000000000fdbe0d69a110d1f0dc6aec8588804da88173b8f0000000000000000000000000b04985bfab8550c43c00528d9d06492916cb9df600000000000000000000000019044ff4c8f00b16ab660407abfffac8fd1ce83300000000000000000000000030c64c8d3e985d91f9c1efaf4ccde66b2925ea7c00000000000000000000000046c4b65f40f686c3f3738395c36acea70c06f3c2000000000000000000000000e6b8dc9357b42fa67a0ac9bdee59d4b6ebe11406000000000000000000000000000000000000000000000000000000000000000f0000000000000000000000000000000000000000000000000de0b6b3a764000000000000000000000000000000000000000000000000000000ed27d2729e070000000000000000000000000000000000000000000000000012208a9134e8b2000000000000000000000000000000000000000000000000000005ec7ca0b6fd00000000000000000000000000000000000000000000000000001094250242bf0000000000000000000000000000000000000000000000000000076909d5d36e00000000000000000000000000000000000000000000000000000197f665a87b000000000000000000000000000000000000000000000000000000000010713f4f0000000000000000000000000000000000000000000000000197018e7029ac0000000000000000000000000000000000000000000000000000414c2bf7269c000000000000000000000000000000000000000000000000000001e6159843ea0000000000000000000000000000000000000000000000000000e235aa5f86f2000000000000000000000000000000000000000000000000000000c37ea4e88f000000000000000000000000000000000000000000000000000007f06bfafc9a0000000000000000000000000000000000000000000000001fd25ea81f7f011c001ca07b43bb33b83e627bc67bad9c55eeb13f3ddada35d6c7d48403788a943fc9bac2a03f9d5e4c193bd5523e2071d0a997f8c221d2cb529cdca620b1d707cfad1303a5",
            "s": "0x3f9d5e4c193bd5523e2071d0a997f8c221d2cb529cdca620b1d707cfad1303a5",
            "standardV": "0x1",
            "to": "0x7d8a7b8c59af9a86bbab3e0daa171930fa857e89",
            "transactionIndex": "0x0",
            "v": "0x1c",
            "value": "0x0"
        }],
        "transactionsRoot": "0x2c76a4849e37e25c0efb23fd410202f04316f6ae1c233f48879295f55eac7e76",
        "uncles": []
    },
    "id": 1
}

[stevenroose]

Here the ethclient parses the response:
https://github.com/ethereum/go-ethereum/blob/master/ethclient/ethclient.go#L96

And here is the parsing of types.Header done:
https://github.com/ethereum/go-ethereum/blob/master/core/types/gen_header_json.go#L130

I don't say either side is wrong about what should be returned. I'm just observing that ethclient is not working with Parity. When I reported this first to the go-ethereum project in ethereum/go-ethereum#3230, I got told it was Parity that was not implementing it correctly. Which lead to #3148.

[stevenroose]

(Unrelated, I patched another incompatibility in ethclient recently: ethereum/go-ethereum#16924)

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 0.45 ETH (240.73 USD @ $534.95/ETH) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $26,757.23 more funded OSS Work available on the Gitcoin Issue Explorer

[mttmartin]

Has this been fixed already, or am I missing something? I seem to see both mixHash and nonce fields getting returned. On the latest master branch I am getting this:

curl --data '{"method":"eth_getBlockByNumber","params":["latest",true],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST localhost:8545

{
    "jsonrpc":"2.0",
    "result :{
"author":"0x28921e4e2c9d84f4c0f0c0ceb991f45751a0fe93",
"difficulty":"0x5e7033926","extraData":"0x476574682f76312e302e302f6c696e75782f676f312e342e32",
"gasLimit":"0x1388",
"gasUsed":"0x0",
"hash":"0xa384921d436c0b4498faf12142255c992586c573bd85531121f9694771e14418",
"logsBloom":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
"miner":"0x28921e4e2c9d84f4c0f0c0ceb991f45751a0fe93",
"mixHash":"0x51ea5bded00c56dbf2e785a2366339ff9a892fcec9e2982f61688aa74b3958d2",
"nonce":"0xd2eea6de816a6fcf",
"number":"0x32b",
"parentHash":"0x563aa0aa60d3251519cec5a01839c30c6b0757f3826fa4ca48af7d333ced1963",
"receiptsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
"sealFields":["0xa051ea5bded00c56dbf2e785a2366339ff9a892fcec9e2982f61688aa74b3958d2","0x88d2eea6de816a6fcf"],
"sha3Uncles":"0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
"size":"0x21b",
"stateRoot":"0xd5cf7fce49a5e49d1aaee1ea181364caa81413dc8b0d8ebd90c8f2d7bbc1951a",
"timestamp":"0x55ba4930",
"totalDifficulty":"0xf75f6d83a15",
"transactions":[],
"transactionsRoot":"0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
"uncles":[]},
"id":1
}

[Tbaut]

Your totally right @mttmartin these fields make no sense on PoA as there is no mining involved. I missed it.

[gitcoinbot]

Issue Status: 1. Open 2. Cancelled

---

The funding of 0.45 ETH (205.84 USD @ $457.43/ETH) attached to this issue has been cancelled by the bounty submitter

• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $15,480.33 more funded OSS Work available on the Gitcoin Issue Explorer

[pstehlik]

Heya @Tbaut - I would love to create a more graceful handling for this condition on the go-ethereum client side and wanted to ask about implementation suggestions.
Essentially some POA chains + client combinations (i.e. kovan + parity) omit returning this field, which causes the go-ethereum client to error out.
I was thinking this could be solved by handling the response a bit more gracefully on the client side by potentially finding another marker that this is indeed a POA chain and with that ignore the header field or set to 0x0 (as geth does when running against rinkeby, for example). The other option would be to introduce a new client option/parameter/config that would allow to simply skip this check and fill 0x0 into the field if set by the person using the lib.
Is another header we could use as a proxy for detecting if the chain is a POA chain?

Rinkeby example

curl --data '{"method":"eth_getBlockByNumber","params":["latest",true],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST https://rinkeby.infura.io
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "difficulty": "0x2",
    "extraData": "0xd883010810846765746888676f312e31302e34856c696e757800000000000000dfe8a052a7dcaaa6879b851e8dfe70da18eef47a025f168423deb814c48eaed51ba117814f352a118baa99d044a948dea2998bfe078a2786e1270bfda45f224500",
    "gasLimit": "0x7738fd",
    "gasUsed": "0x4667d5",
    "hash": "0x12bbf7cbd3b6cc37620da0d34b20719281ac3ff383da2c61e25909e9f5c78717",
    "logsBloom": "...",
    "miner": "0x0000000000000000000000000000000000000000",
    "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "nonce": "0x0000000000000000",
    "number": "0x2f1522",
    "parentHash": "0xec200aba54597e27dde2e4d71815805488d3d2eab072c034b76ef03d7053e68a",
    "receiptsRoot": "0x0be223efde71634a9d0a0bc3b7db82db11d25cca45fbf2061aab85aa0b5e5786",
    "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
    "size": "0x4111",
    "stateRoot": "0xae1d250dc4637929acaa80bdadefcb58aa2a75a574d2f2e6344bcef737663a48",
    "timestamp": "0x5bb24a55",
    "totalDifficulty": "0x576c6c",
    "transactions": [...
    ],
    "transactionsRoot": "0xd56bcf2e5b44622b545e6e9513fc091c20bfae7bc9213a5fa5f73251c80ae35f",
    "uncles": []
  }
}

Kovan example (infura)

curl --data '{"method":"eth_getBlockByNumber","params":["latest",true],"id":1,"jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST https://kovan.infura.io
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "author": "0x0010f94b296a852aaac52ea6c5ac72e03afd032d",
    "difficulty": "0xfffffffffffffffffffffffffffffffe",
    "extraData": "0xd583010a098650617269747986312e32372e30826c69",
    "gasLimit": "0x7a1200",
    "gasUsed": "0x46540",
    "hash": "0x447c14bac202a4ad48db178a9438b1114f170a9f03c68c9644af2b058088635c",
    "logsBloom": "...",
    "miner": "0x0010f94b296a852aaac52ea6c5ac72e03afd032d",
    "number": "0x882e39",
    "parentHash": "0xd27c8e423c5368ec0e13ee745baf426cb23c4c8f32a7b81af1a701e136129834",
    "receiptsRoot": "0xe79da2d22fb7e7dfc84d16522d004807c86bee5006cbc591efea4e750117ae5f",
    "sealFields": [
      "0x8416ec929e",
 "0xb841c68825e3241bd0d40288f685538cd81797aa2a553f38986b262efb2d998487bc7893d0eaa798aca6a8937595871eab649f4591b04cb0be3717c1e2f26ee6cab601"
    ],
    "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
    "signature": "c68825e3241bd0d40288f685538cd81797aa2a553f38986b262efb2d998487bc7893d0eaa798aca6a8937595871eab649f4591b04cb0be3717c1e2f26ee6cab601",
    "size": "0x739",
    "stateRoot": "0xddc78e90381f0a3fe818dc0884beb841495ec49df56e6dd2207d8e5a5f8eaf4e",
    "step": "384602782",
    "timestamp": "0x5bb24a78",
    "totalDifficulty": "0x85eb2a00000000000000000000000484e23052",
    "transactions": [...
    ],
    "transactionsRoot": "0x316dddead77f65964957932692a3ba15498e792015b2ff99dab37400f5d7fd96",
    "uncles": []
  }
}

Kovan example (local node)

{
  "jsonrpc": "2.0",
  "result": {
    "author": "0x00e6d2b931f55a3f1701c7389d592a7778897879",
    "difficulty": "0xfffffffffffffffffffffffffffffffd",
    "extraData": "0xde830200048f5061726974792d457468657265756d86312e32382e30826c69",
    "gasLimit": "0x7a1200",
    "gasUsed": "0x0",
    "hash": "0xba9c3d6082f0ef5f98295440a5cec15f879729edf082ec2ce8a53401415ab09f",
    "logsBloom": ".",
    "miner": "0x00e6d2b931f55a3f1701c7389d592a7778897879",
    "number": "0x87b2f4",
    "parentHash": "0x511a9d8fafd86f8ed7e7d38ba8f5002cf35c4174bbdb84a0211347bc60d20366",
    "receiptsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
    "sealFields": [
      "0x8416ebb014", "0xb841edabb43a93c8aa79bd1afdb4465be71d995a3206672f64733fe9ab78e373c7424ba84f37d726da1c1d56ffd5e84bf4c6c465b1bb4bd705ec3af88c801d16552200"
    ],
    "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
    "signature": "edabb43a93c8aa79bd1afdb4465be71d995a3206672f64733fe9ab78e373c7424ba84f37d726da1c1d56ffd5e84bf4c6c465b1bb4bd705ec3af88c801d16552200",
    "size": "0x24c",
    "stateRoot": "0x1a79132c5ccc505df62d7b7aa8a4864bd0c0dbfb6710db4554b8e25b32a907bb",
    "step": "384544788",
    "timestamp": "0x5baec050",
    "totalDifficulty": "0x856fe500000000000000000000000484e38e21",
    "transactions": [],
    "transactionsRoot": "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
    "uncles": []
  },
  "id": 1
}

[Tbaut]

Let me ping @debris who can answer that better than me :)