Skip to content
This repository has been archived by the owner on Feb 26, 2024. It is now read-only.

perf: cache requests to the underlying fork provider #581

Closed
wants to merge 1 commit into from
Closed

perf: cache requests to the underlying fork provider #581

wants to merge 1 commit into from

Conversation

fubhy
Copy link
Contributor

@fubhy fubhy commented Jun 10, 2020

We managed to improve the performance of the ganache fork we use for testing by a factor of 10 to 20 by caching identical requests to the underlying fork (our slowest test used to take ~250 seconds and frequently crash because of exceeding rate limits on our archive node / timeouts - It's now at 13 seconds consistently).

To achieve this, I simply patched the web3 provider used by the fork to intercept all calls to send and run them through a LRU cache.

@fubhy
Copy link
Contributor Author

fubhy commented Jun 10, 2020

In addition to that we also put together a reverse proxy for caching and persisting these requests in Redis between CI runs if anyone is interested: https://github.com/avantgardefinance/ganache-cache-proxy

@davidmurdoch
Copy link
Member

This is amazing! We'll review this week and try to ship to production soon after! Thanks!

@coveralls
Copy link

coveralls commented Jun 11, 2020

Coverage Status

Coverage increased (+0.02%) to 82.216% when pulling 179a140 on fubhy:develop into 94fdc56 on trufflesuite:develop.

@fubhy
Copy link
Contributor Author

fubhy commented Jun 11, 2020

I fixed that one failing test (serialization / deserialization issue in lru-cache apparently).

I wanted to add a test to verify that the caching actually works but noticed that a) you are using mocha (which doesn't have native method spies) for your test suite and b) there's currently no method spy dependency installed (spy.js / sinon). I didn't want to add yet another dependency as part of this PR so how'd you want to implement such a test without spying on the .send method?

@davidmurdoch
Copy link
Member

This looks really great! I'm working on some subtle fixes to the code, as well as writing some tests.

@davidmurdoch davidmurdoch changed the title Cache requests to the underlying fork provider perf: cache requests to the underlying fork provider Jun 18, 2020
@davidmurdoch
Copy link
Member

Closing. Will be merging via #585.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants