`header not found` error when starting ganache forking with infura RPC endpoint #615

mikeseese · 2020-08-21T00:28:24Z

I'm not sure if this is specific to forking or infura, but periodically on v6.10.1 I receive the following error immediately on startup:

Ganache CLI v6.10.1 (ganache-core: 2.11.2)
Error: Returned error: header not found
    at Object.ErrorResponse (/home/mike/.nvm/versions/node/v12.18.3/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:39:2207407)
    at /home/mike/.nvm/versions/node/v12.18.3/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:39:2206279
    at /home/mike/.nvm/versions/node/v12.18.3/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:39:2193803
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Restarting will fix the issue fairly quickly. I'm wondering if this is related to #579 🤔 (doubt it though)

Your Environment

Version used: Ganache CLI v6.10.1 (ganache-core: 2.11.2)

The text was updated successfully, but these errors were encountered:

mds1 · 2020-08-22T16:03:45Z

My stack trace when seeing this error is different, as seen below:

Error: Returned error: Returned error: header not found
at PromiEvent (node_modules/@truffle/contract/lib/promievent.js:9:30)
 at /Users/mds/Documents/projects/<projectName>/contracts/node_modules/@truffle/contract/lib/execute.js:223:26
 at Function.new (node_modules/@truffle/contract/lib/contract/constructorMethods.js:57:53)
 at Context.<anonymous> (test/contract-test.js:50:34)
 at processTicksAndRejections (internal/process/task_queues.js:97:5)

I'm using ganache-core 2.11.2 through OpenZeppelin Test Environment, and am forking the mainnet with an Infura endpoint. Like you said @seesemichaelj, this is resolved by restarting ganache

rishanb · 2020-10-06T05:08:59Z

Similarish error - I've been trying to locate some kind of config file to see where "account 0xd0a8356c4889d2a9876c882a8bd9cc0ee7e4430b" comes from but have been unsuccessful so far.

ganache-cli -f https://cloudflare-eth.com
Ganache CLI v6.11.0 (ganache-core: 2.12.1)
Error: The fork provider errored when checking the nonce for account 0xd0a8356c4889d2a9876c882a8bd9cc0ee7e4430b: Returned error: header not found
    at /usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1994078
    at Object._fireError (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:75947)
    at s (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:119479)
    at n (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:2007203)
    at /usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1993001
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  data: null
}

mikeseese · 2020-10-06T05:17:43Z

@rishanb ganache generates 10 accounts from a random mnemonic by default, even when you're forking. The screen just had not yet gotten to the point where it displays the accounts

I'm curious, is the error reproducible for you @rishanb? Does restarting no longer show the error?

rishanb · 2020-10-06T05:45:10Z

Apologies for the delay, ran into some weirdness and wanted to collect it all rather than creating confusion.

I received the original error above (header not found) when trying to run ganache-cli while forking from Cloudflare:
ganache-cli -f https://cloudflare-eth.com

Gave:

Ganache CLI v6.11.0 (ganache-core: 2.12.1)
Error: The fork provider errored when checking the nonce for account 0xcfd0f9c9dbb310707ab1ea60151e96b03daf1d32: Returned error: header not found
    at /usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1994078
    at Object._fireError (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:75947)
    at s (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:119479)
    at n (/usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:2007203)
    at /usr/local/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1993001
    at processTicksAndRejections (internal/process/task_queues.js:93:5) {
  data: null
}

Retrying the same command provided the same error. I then uninstalled and reinstalled ganache-cli and got the same error with a different account address at the top.

If I leave out the "-f ..." part I get the callback error which made me aware that I need to revert my node version. I'm currently in the process of figuring out nvm and will report back if this error persists from node 12.

rishanb · 2020-10-06T06:03:24Z

Unfortunately, I'm getting the same issue with nvm 12 when making an initial attempt and retrying:

(base) ~ $ nvm use 12 && ganache-cli -f https://cloudflare-eth.com
Now using node v12.18.4 (npm v6.14.6)
Ganache CLI v6.11.0 (ganache-core: 2.12.1)
Error: The fork provider errored when checking the nonce for account 0x0f711c427f7d3497c1b457297820fe39dc911d37: Returned error: header not found
    at /Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1994078
    at Object._fireError (/Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:75947)
    at s (/Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:119479)
    at n (/Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:2007203)
    at /Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1993001
    at processTicksAndRejections (internal/process/task_queues.js:97:5) {
  data: null
}
^C%                                                                                                                                 
(base) ~  $ nvm use 12 && ganache-cli -f https://cloudflare-eth.com
Now using node v12.18.4 (npm v6.14.6)
Ganache CLI v6.11.0 (ganache-core: 2.12.1)
Error: The fork provider errored when checking the nonce for account 0xae3be88f190e877f05cc50952e6f201355b5a8ef: Invalid JSON RPC response: "502 Bad Gateway"
    at /Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1994078
    at Object._fireError (/Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:75947)
    at s (/Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:4:119479)
    at n (/Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:2007187)
    at /Users/rishanwins/.nvm/versions/node/v12.18.4/lib/node_modules/ganache-cli/build/ganache-core.node.cli.js:48:1993001
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
^C%                                                                                                                                 (base) ~/  $

And in all this I think I've combined two issues, the original "header" issue applicable to this thread and my new "nonce" issue above. So the "nonce" issue occurs when trying to fork mainnet.

I'm glad to report the "header" issue is gone once running ganache from node 12:

nvm use 12 && ganache-cli
Now using node v12.18.4 (npm v6.14.6)
Ganache CLI v6.11.0 (ganache-core: 2.12.1)

Available Accounts
==================
(0) 0xC9eB8621Ba2F95B390F7B3023A737E6d94Eb8963 (100 ETH)
(1) 0xbe92c4Deb7d6911A30F4D0c701724d5E33DcCD0B (100 ETH)
(2) 0xB716b7287A4e97F2F93983F3abBFE50837604105 (100 ETH)
(3) 0xEd1b0d778133d3f03cD28ab9Dd87225e2dbfDD37 (100 ETH)
(4) 0x4A237BaEc634b14A4b15683B8Fc48742bE3Aef99 (100 ETH)
(5) 0xb1Fc1B264df2C4d99db7032ADb3aF36B6707d9f4 (100 ETH)
(6) 0xbFceF391cE1e1Aa4DB83E154e525315AEE114a7A (100 ETH)
(7) 0x09FFDa666C8B7454dcbe8fF440a22b9c8991Eeb0 (100 ETH)
(8) 0x6bBbE4d4E6eC798E430dFDEA13F585fB4E7f72C0 (100 ETH)
(9) 0xA6F92b338235Ed4DfC3EC155Ce28cb8dd44AcabA (100 ETH)

@seesemichaelj

mikeseese · 2020-10-06T06:13:45Z

I'm glad to report the "header" issue is gone once running ganache from node 12:

Well both errors only occur when you are trying to fork a network, so glad that you're not hitting that code when you're not configured for forking haha

The second try you post when forking provides the error:

Invalid JSON RPC response: "502 Bad Gateway"

Which sounds like an issue with the Cloudflare endpoint to some degree. I believe that error should be a brand new issue referring to cloudflare endpoint support since I can reproduce the 502 Bad Gateway error consistently when using the cloudflare endpoint. I've created #644 to document the cloudflare-specific issue

I'd suggest using Infura

Now the header not found error just coincidentally also showed up, which I do believe is separate from the cloudflare specific issue

rishanb · 2020-10-06T06:18:27Z

Thank you sir! Infura worked like a charm.

And as to my silliness above, in my last post I kinda of didn't scroll right and so missed the "header" error part of the first attempt...and incorrectly concluded it was some new nonce error ;p

So in summary: reverting to node 12 resolved all issues when not forking, and furthermore, forking from Infura resolved all forking issues. As you've pointed out, there appears to be an issue with Forking the Cloudflare ETH Endpoint.

mikeseese · 2020-10-06T06:34:05Z

Do note that Infura has a 128 block limit before needing to purchase the data archive add-on; you can read more about the limit here: https://infura.io/docs/ethereum/add-ons/archiveData. Depending on what you're using forking for, you may run into issues with the data archival.

Truffle Teams provides Sandboxes which are just cloud-hosted Ganache instances. Forking is available in Truffle Teams Sandboxes without the 128 block limit. Currently our free tier of Truffle Teams allows you to have 1 Sandbox, and you can currently choose forking in the free tier. Not sure if it'll always be available in the free tier, but it's free now! You can read more about Truffle Teams Sandboxes in our docs. You can sign up for a free Truffle Teams account at https://my.truffleteams.com

(Truffle Teams does a lot more cool stuff too! Sandboxes is just one of them)

dexcell · 2020-10-31T10:48:22Z

Still exist when forking cloudflare, but not on infura.
v6.12.1

davidmurdoch · 2020-10-31T12:34:06Z

@dexcell This might be because cloudflare doesn't support archive state.

dexcell · 2020-10-31T20:28:20Z

@davidmurdoch I see thanks

@seesemichaelj
So truffle teams offer fork with archival data too? Right now i'm paying infura to test mainnetfork with archival data, i wish i saw this issue earlier.

mikeseese · 2020-10-31T20:39:08Z

@dexcell That's correct! At the current time, you can use your free sandbox to fork with archival data. Check out our docs to see how to make a sandbox

gnidan · 2021-03-24T18:14:31Z

It's not clear what Ganache should do in these situations, but maybe this issue is one of better error reporting? I'll label this issue as such, since that's one path forward (i.e., augment the geth-provided error with additional detail on what's going wrong and possible ways to fix).

In the meantime, if anyone has ideas for handling this problem more robustly, we're all ears!

Thanks all.

mikeseese added the forking label Aug 21, 2020

mikeseese self-assigned this Aug 21, 2020

mds1 mentioned this issue Aug 22, 2020

Cannot get state root with uncommitted checkpoints error when starting ganache forking with infura RPC endpoint #618

Closed

mikeseese mentioned this issue Oct 6, 2020

Issues with Forking Cloudflare ETH Endpoint #644

Closed

gnidan added needs requirements error better issue indicates improvement to error messaging priority4 📋 labels Mar 24, 2021

gnidan assigned davidmurdoch and unassigned mikeseese Mar 24, 2021

gnidan added this to the 7.0.0 milestone Mar 24, 2021

davidmurdoch modified the milestones: 7.0.0, 7.1.0 Aug 6, 2021

seaona mentioned this issue May 10, 2022

Sentry Error: [ethjs-query] while formatting outputs from RPC - header not found_Last 30 Days: 66k MetaMask/metamask-extension#14676

Closed

davidmurdoch removed their assignment Jul 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`header not found` error when starting ganache forking with infura RPC endpoint #615

`header not found` error when starting ganache forking with infura RPC endpoint #615

mikeseese commented Aug 21, 2020

mds1 commented Aug 22, 2020 •

edited

rishanb commented Oct 6, 2020 •

edited

mikeseese commented Oct 6, 2020

rishanb commented Oct 6, 2020 •

edited

rishanb commented Oct 6, 2020 •

edited

mikeseese commented Oct 6, 2020 •

edited

rishanb commented Oct 6, 2020

mikeseese commented Oct 6, 2020 •

edited

dexcell commented Oct 31, 2020

davidmurdoch commented Oct 31, 2020

dexcell commented Oct 31, 2020

mikeseese commented Oct 31, 2020

gnidan commented Mar 24, 2021

header not found error when starting ganache forking with infura RPC endpoint #615

header not found error when starting ganache forking with infura RPC endpoint #615

Comments

mikeseese commented Aug 21, 2020

Your Environment

mds1 commented Aug 22, 2020 • edited

rishanb commented Oct 6, 2020 • edited

mikeseese commented Oct 6, 2020

rishanb commented Oct 6, 2020 • edited

rishanb commented Oct 6, 2020 • edited

mikeseese commented Oct 6, 2020 • edited

rishanb commented Oct 6, 2020

mikeseese commented Oct 6, 2020 • edited

dexcell commented Oct 31, 2020

davidmurdoch commented Oct 31, 2020

dexcell commented Oct 31, 2020

mikeseese commented Oct 31, 2020

gnidan commented Mar 24, 2021

`header not found` error when starting ganache forking with infura RPC endpoint #615

`header not found` error when starting ganache forking with infura RPC endpoint #615

mds1 commented Aug 22, 2020 •

edited

rishanb commented Oct 6, 2020 •

edited

rishanb commented Oct 6, 2020 •

edited

rishanb commented Oct 6, 2020 •

edited

mikeseese commented Oct 6, 2020 •

edited

mikeseese commented Oct 6, 2020 •

edited