Allow to sync forked chain to the latest block

[hihidev]

It would be nice ganache-cli has a command to make the forked chain sync to the latest block, as now if I want it restores to the latest block, I need to restart the program which takes time.

Here's my use case:
I'm trying to write a monitor to keeping simulating the result of some pending tx(s) after some blocks / other tx(s) confirmed.
As there are always some new pending tx and blocks coming, I need to restore the state to the latest block in order to test the new pending tx(s).

Running eth_call() on real eth client doesn't work as I need to simulate the state after a few blocks, and also some combination of pending tx merged.

evm_snapshot / evm_revert doesn't help also, as it cannot bring the forked chain to the latest block when there's a new block confirmed.

[rymnc]

@hihidev , if you want to bring the latest state of the chain that you are forking, it would involve you losing state of whichever transactions you have executed
If that's okay, then I think it would be fairly easy for a flag/option which could restart the ganache instance after a fixed number of blocks
What do you think @davidmurdoch?

[hihidev]

Thanks @aaryamannchallani

Yes, I think it makes sense losing the state as I would love to reset the state to the latest block / env to test to run the tests again.
However, I prefer it's a command like "evm_sync_latest", but not auto restart ganache instance after a fixed number of blocks, as it's possible user is running some tests and not ready to be reset (sync the chain) automatically even after a fixed number of blocks.
Also (not sure how ganache works :D) it would be nice if we could just restore the memory / storage state, rather than restarting the instance, so it can reduce the time needed for this action?

[hihidev]

Oh, I just found out if I create Ganache.provider(fork:) instance directly, it's pretty fast if my client is already running in localhost!

[hihidev]

Apparently, running Ganache.provider() can have a faster boot time, but at the end it will be much slower to process tx due to
#481

Hence I guess @aaryamannchallani is correct, having a flag to reset ganache instance in ganache-cli should fix most of the issues :D

[davidmurdoch]

Cli is only faster than ganache.server when the process that started ganache.server is busy, like when running tests. If you start it in a separate process that isn't constantly busy doing other IO and CPU laden tasks it should be just as fast at running txs, etc.

The reason ganache cli start up time is somewhat slow is because of node's initial start up time.

CLI startup time has never been a priority, as no one has commented on it before (that I know of). If this is something important to you and others I encourage you to open an issue so we can gauge interest/need and discuss further!

[taarushv]

+1. I'm working on a tool to monitor the mempool + events emitted by txs and this would be an A+ addition to the cli tool. I'm going to explore spawning a ganache server (at latest block height) infrequently (using a JS wrapper) for now but a flag sounds great. Something that would let me run the cli-tool in background while it automatically flushes the state and forks off the latest block after every ~n blocks. Thanks!

[gnidan]

We can totally add an RPC method to reset Ganache's internal state to achieve this.

This would be the same as restarting Ganache with the fork option pointing to the new latest. I suppose that's a workaround in the meantime?

Thanks for the discussion on this, all!

[taarushv]

Yup, that’s pretty much what I meant. I’ve been using this script as a workaround but I suspect there’s a more clever/efficient way of doing things https://github.com/taarushv/helios/blob/main/panther/src/index.js

Tx tracing just requires too much custom tooling (which is worth it for complex use cases) but I can see simple use cases (like parsing events of an oracle update tx before it’s mined) adopt ganache if this was a native feature. Thanks!

[soulmachine]

This feature will be useful to build a simulator like https://www.blocknative.com/simulation-platform and https://tenderly.co/transaction-simulator

[Toeplitz]

We can totally add an RPC method to reset Ganache's internal state to achieve this.

This would be the same as restarting Ganache with the fork option pointing to the new latest. I suppose that's a workaround in the meantime?

Thanks for the discussion on this, all!

Has there been any progress on this since this reply in 2021?

[wrong7]

Hey, any news about this? I would like to resync my fork on every test I run without having to restart the file

[MicaiahReid]

Hi all, we have begun work on this. I don't have an estimated release date yet, but it has moved up on our priority list because of all of the community interest. Thanks for your participation to show us what's important to you!

[acaladolopes]

Hi, I'm also very interested in this feature. I need to perform simulations against the very latest state of the blockchain. I'm currently using hardhat for forking/simulation, but it's slow compared to Ganache.

[nikolinsko]

same here.

i'm currently setting up a new instance on every request via:

const options = {
        fork: {
            url: "ws://xxxxx",
            preLatestConfirmations: 1
        },
        miner: {
            defaultGasPrice: "0x12A05F200",
            blockTime: 0,
            instamine: 'eager'
        },
        server: {
            ws: true,
        },
        chain: {
            networkId: 56,
            chainId: 56
        }
    };

const web3 = new Web3(ganache.provider(options));

i'm using this to verify the behaviour of contracts and need the data as fast as possible. Currently a call which consumes somewhere between 100k and 900k gas in total takes around 500ms-1 second. The node runs locally and is well within specs/only serves one client.

Any tips on how to speed this up currently?

[davidmurdoch]

@nikolinsko you can try passing in memdown() (https://github.com/Level/memdown) to options.database.db:

const web3 = new Web3(ganache.provider({database:{db:memdown()}}));

[nikolinsko]

@davidmurdoch thanks for the tip, i did not test it isolated but forking using a scheduler together with memdown seemed to speed it up a bit.

one quick question,

I mainly use two transactions, one is an interaction with a smart contract not owned by myself the other one is a contract deployed by myself.

Would deploying the last one using ganache speed up the process?

[davidmurdoch]

I don't think so. The reason it is slow is because Ganache's forking cache can't be used; transactions in the newer block may have changed the state we cached before. If you do find a way to speed things up further please do let us know.

[wrong7]

Hey how is this feature going? Any ETA?

[Jovonni]

any update on this? +1

[ahmadina]

any update on this? +1

[davidmurdoch]

An update: progress is being made!