Unable to re-create receipts root only for block 22982400

[sirpy]

Expected Behavior

merkle-patricia-tree root based on receipts should match the block reported receipts root hash

Actual Behavior

no match

Steps to reproduce the behavior

I'm using merkle-patricia-tree package to generate the tries and proofs.
This has been working just fine except for this block 22982400 for later blocks it also works, when I build the tree from the receipts the final root doesn't match the one reported in the block header

Chain/Network: Mainnet

code:

import { BaseTrie as Tree } from 'merkle-patricia-tree';
import { Receipt } from 'eth-object';
import { encode } from 'eth-util-lite';

const targetReceipt = await provider.send('eth_getTransactionReceipt', [txHash]);
const rpcBlock = await provider.send('eth_getBlockByHash', [targetReceipt.blockHash, false]);
const receipts = (
    await Promise.all(
      rpcBlock.transactions
        .map(async (siblingTxHash) => {
          return provider.send('eth_getTransactionReceipt', [siblingTxHash]);
        })
    )
  ).filter((_) => _);

  const tree = new Tree();

  await Promise.all(
    receipts.map((siblingReceipt, index) => {
      const siblingPath = encode(index);
      let serializedReceipt = Receipt.fromRpc(siblingReceipt).serialize();
      if (siblingReceipt.type && siblingReceipt.type != '0x0') {
        serializedReceipt = Buffer.concat([Buffer.from([siblingReceipt.type]), serializedReceipt]);
      }
      return tree.put(siblingPath, serializedReceipt);
    }),
  );

  const receiptsRoot = '0x' + tree.root.toString('hex');
  if (receiptsRoot !== rpcBlock.receiptsRoot) {
    console.error(receipts, {
      receiptsRoot,
      blockReceiptsRoot: blockHeader.block.receiptsRoot,
    });
    throw new Error('receiptsRoot mismatch');
  }

[karlb]

Thanks for the bug report! I can reproduce your observation that the receipts root for block 22982400 does not match the merkle root calculated from the receipts that are returned via RPC.

Block 22982400 is an epoch block and it looks like this mismatch happens for all epoch blocks and all other blocks are fine. Epoch blocks have special epoch transactions to distribute the epoch rewards rewards (e.g. rewards for block 22982400). These won't show up as normal transactions in the RPC. I assume that the epoch transactions are the reason for the mismatch, although I have not yet looked up how receipts for them are handled.

[karlb]

When events are emitted by the block processing outside of user transaction (e.g. epoch rewards), then one "block receipt" is added to the block. This receipt can't be fetched with eth_getTransactionReceipt but with the Celo-specific eth_getBlockReceipt.

Combining these receipts makes the receiptsRoot match again:

import { ethers } from "ethers";
import { BaseTrie as Tree } from 'merkle-patricia-tree';
import { Receipt } from 'eth-object';
import { encode } from 'eth-util-lite';

const provider = new ethers.JsonRpcProvider('https://forno.celo.org');

const block = '0x' + (22982400).toString(16);
const rpcBlock = await provider.send('eth_getBlockByNumber', [block, false]);
const transactionReceipts = (
  await Promise.all(
    rpcBlock.transactions
    .map(async (siblingTxHash) => {
      return provider.send('eth_getTransactionReceipt', [siblingTxHash]);
    })
  )
)
const blockReceipt = (
  await Promise.all(
    [rpcBlock.hash]
    .map(async (siblingTxHash) => {
      return provider.send('eth_getBlockReceipt', [siblingTxHash]);
    })
  )
)
const receipts = transactionReceipts.concat(blockReceipt);

const tree = new Tree();

await Promise.all(
  receipts.map((siblingReceipt, index) => {
    const siblingPath = encode(index);
    let serializedReceipt = Receipt.fromRpc(siblingReceipt).serialize();
    if (siblingReceipt.type && siblingReceipt.type != '0x0') {
      serializedReceipt = Buffer.concat([Buffer.from([siblingReceipt.type]), serializedReceipt]);
    }
    return tree.put(siblingPath, serializedReceipt);
  }),
);

const receiptsRoot = '0x' + tree.root.toString('hex');
if (receiptsRoot !== rpcBlock.receiptsRoot) {
  console.error(receipts, {
    receiptsRoot,
    blockReceiptsRoot: rpcBlock.receiptsRoot,
  });
  throw new Error('receiptsRoot mismatch');
}

I admit that this situation is currently under-documented.

[arthurgousset]

(notes-to-self for future reference)

It looks like the Celo-specific eth_getBlockReceipt was added in:

• Add a new RPC method eth_getBlockReceipt to get a block's "system calls" receipt #1628

From the PR:

An earlier version of this PR added support for block receipts to eth_getTransactionReceipt, but after more consideration it was found that adding a separate method results in both a cleaner API and a cleaner implementation.

The method can be added to a web3 instance as follows (where web3 is a Web3 instance, as for example the one available in contractkit as kit.web3):

web3.eth.extend({methods: [{name: "getBlockReceipt", call: "eth_getBlockReceipt", params: 1, outputFormatter: web3.extend.formatters.outputTransactionReceiptFormatter}]})

[sirpy]

@arthurgousset
Thanks. so yeah we already had the getblockreceipt but we removed it post 1.8.
i'm guessing there's another bug here.
For blocks before 1.8 adding the blockreceipt was required, but for blocks post 1.8 they fail the receiptsroot check if we add the block receipt.
So it seems post 1.8 the blockreceipt should only be added to epoch blocks.
Is this the expected behaviour?
Is there a way to identify an epoch block from a regular block?

[karlb]

It looks like the block receipt only goes into the block's receiptRoot if it contains logs. I works for me for both epoch blocks and non-epoch blocks when I do

if (blockReceipt[0].logs.length === 0) {
  blockReceipt = [];
}

I tested it with 23708159, 23708160, 22982399 and 22982400. Before/after version 1.8 did not make a difference. Here's the the code I used:

import { ethers } from "ethers";
import { BaseTrie as Tree } from 'merkle-patricia-tree';
import { Receipt } from 'eth-object';
import { encode } from 'eth-util-lite';

const provider = new ethers.JsonRpcProvider('https://forno.celo.org');

[
  23708159, 23708160, 22982399, 22982400
].forEach(async blockNumber => {
  const block = '0x' + (blockNumber).toString(16);
  const rpcBlock = await provider.send('eth_getBlockByNumber', [block, false]);
  const transactionReceipts = (
    await Promise.all(
      rpcBlock.transactions
      .map(async (siblingTxHash) => {
        return provider.send('eth_getTransactionReceipt', [siblingTxHash]);
      })
    )
  )
  let blockReceipt = (
    await Promise.all(
      [rpcBlock.hash]
      .map(async (siblingTxHash) => {
        return provider.send('eth_getBlockReceipt', [siblingTxHash]);
      })
    )
  )
  if (blockReceipt[0].logs.length === 0) {
    blockReceipt = [];
  }
  const receipts = transactionReceipts.concat(blockReceipt);

  const tree = new Tree();

  await Promise.all(
    receipts.map((siblingReceipt, index) => {
      const siblingPath = encode(index);
      let serializedReceipt = Receipt.fromRpc(siblingReceipt).serialize();
      if (siblingReceipt.type && siblingReceipt.type != '0x0') {
        serializedReceipt = Buffer.concat([Buffer.from([siblingReceipt.type]), serializedReceipt]);
      }
      return tree.put(siblingPath, serializedReceipt);
    }),
  );

  const receiptsRoot = '0x' + tree.root.toString('hex');
  if (receiptsRoot !== rpcBlock.receiptsRoot) {
    console.error('Failed for block', blockNumber);
    throw new Error('receiptsRoot mismatch');
  }
});

If this is how it works, the eth_getBlockReceipt return value is misleading for non-epoch blocks and it should return an empty result if no logs are present rather than a receipt with no logs.

[karlb]

Here is the code where an empty receipt is created for those cases where no block receipt exists because no events have been emitted by the system. Since the receipt is created on the fly in the RPC, it is not included in the receiptsRoot.

That feels like an odd choice. I assume the original author wanted to keep the shape of the RPC response consistent across blocks and therefore chose to create a dummy receipt if none exists.

[karlb]

So it seems post 1.8 the blockreceipt should only be added to epoch blocks. Is this the expected behaviour?

That sounds wrong to me. I'll investigate.

Is there a way to identify an epoch block from a regular block?

Yes, it is every 17280th block. See https://github.com/celo-org/epochs?tab=readme-ov-file#epoch-blocks.