Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Event filtering is slow #15091

Closed
jannikluhn opened this issue Sep 5, 2017 · 4 comments
Closed

Event filtering is slow #15091

jannikluhn opened this issue Sep 5, 2017 · 4 comments

Comments

@jannikluhn
Copy link

@jannikluhn jannikluhn commented Sep 5, 2017

System information

Geth
Version: 1.6.7-stable
Git Commit: ab5646c
Architecture: amd64
Protocol Versions: [63 62]
Network Id: 1
Go Version: go1.8.1
Operating System: linux

Issue

Getting logs from a large block range is slow, even if the filter doesn't match anything:

time curl -X POST --data '{"id":8,"jsonrpc":"2.0","method":"eth_getLogs","params":[{"fromBlock":"0x0","toBlock":"0xf0be2", "address": "0x33990122638b9132ca29c723bdf037f1a891a70d"}]}' -H "Content-Type: application/json" http://127.0.0.1:8545 >> /dev/null

takes 27s instead of 0.02s promised in this stackexchange answer. Tested on Ropsten.

@karalabe
Copy link
Member

@karalabe karalabe commented Sep 5, 2017

There's a PR pending review/merge that replaces the entire event filtering to a completely different mechanism #14631. It should be orders of magnitude faster. Feel free to take it to a spin an report on actual numbers. I'd be curious for real world tests.

@karalabe
Copy link
Member

@karalabe karalabe commented Sep 5, 2017

Just did a fresh sync on Ropsten with the bloombits PR and tried your command:

$ time curl -s -X POST --data '{"id":8,"jsonrpc":"2.0","method":"eth_getLogs","params":[{"fromBlock":"0x0","toBlock":"0xf0be2", "address": "0x33990122638b9132ca29c723bdf037f1a891a70d"}]}' -H "Content-Type: application/json" http://127.0.0.1:8545 >> /dev/null

real	0m0.168s
user	0m0.004s
sys	0m0.000s

$ time curl -s -X POST --data '{"id":8,"jsonrpc":"2.0","method":"eth_getLogs","params":[{"fromBlock":"0x0","toBlock":"0xf0be2", "address": "0x33990122638b9132ca29c723bdf037f1a891a70d"}]}' -H "Content-Type: application/json" http://127.0.0.1:8545 >> /dev/null

real	0m0.012s
user	0m0.004s
sys	0m0.000s

Running the a full chain filtering for all ENS .eth events on Ropsten:

$ time curl -s -X POST --data '{"id":1,"jsonrpc":"2.0","method":"eth_getLogs","params":[{"fromBlock":"0x0","toBlock":"latest", "address": "0xC68De5B43C3d980B0C110A77a5F78d3c4c4d63B4"}]}' http://127.0.0.1:8545 >> /dev/null

real	0m2.203s
user	0m0.008s
sys	0m0.024s

$ time curl -s -X POST --data '{"id":1,"jsonrpc":"2.0","method":"eth_getLogs","params":[{"fromBlock":"0x0","toBlock":"latest", "address": "0xC68De5B43C3d980B0C110A77a5F78d3c4c4d63B4"}]}' http://127.0.0.1:8545 > /dev/null

real	0m0.658s
user	0m0.000s
sys	0m0.008s
@karalabe
Copy link
Member

@karalabe karalabe commented Sep 6, 2017

The PR was merged into master. Feel free to grab a fresh build and try it out yourself. Note, master does a few database upgrades, so you'll probably need to wait for that to finish (or resync from zero).

@karalabe karalabe closed this Sep 6, 2017
@jannikluhn
Copy link
Author

@jannikluhn jannikluhn commented Sep 6, 2017

Awesome, now it's literally a thousand times better (down to 0.03s). Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants