Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fork-observer: reduce query interval #175

Merged
merged 1 commit into from
Nov 2, 2023

Conversation

willcl-ark
Copy link
Contributor

For S-C-A-L-I-N-G

Reduce query interval to every 60 seconds by default. Helps on larger networks.

@vercel
Copy link

vercel bot commented Oct 31, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
warnet ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 2, 2023 9:15pm

@0xB10C
Copy link
Contributor

0xB10C commented Oct 31, 2023

lgtm

I might want to add some randomness to the query interval for each node in fork-observer so you don't end up firing 200 RPC calls exactly ever 60s.

@0xB10C
Copy link
Contributor

0xB10C commented Oct 31, 2023

Also, keep in mind that fork-observer currently has a limit of 256 nodes per network. It's trivial to increase it but I haven't gotten to it yet. 0xB10C/fork-observer#22

@willcl-ark
Copy link
Contributor Author

Oh right, I think I will also add a commit here to disable if the graph is >256 nodes then

@0xB10C
Copy link
Contributor

0xB10C commented Oct 31, 2023

I've fixed it in fork-observer: 0xB10C/fork-observer#24

The new docker image should allow Warnet to theoretically attach more than 256 nodes. Not sure if that's practical though.

@pinheadmz
Copy link
Contributor

Do we still need this now that fork observer was updated? There's also #177

@willcl-ark
Copy link
Contributor Author

Yeah we want this. Updated FO lets us attach more nodes (256 limit), this queries the nodes less frequently. You know how you saw FO was using the most CPU/bandwidth on the graphs earlier... ;)

#177 is different

@willcl-ark
Copy link
Contributor Author

Would be even nicer (for us) if FO would query on a chill scheduler, e.g. If you configure 100 nodes and query every 60 seconds it could query one node every 0.6 seconds to keep resource usage as flat as possible. Vs currently where every n seconds it will query all nodes at the same time (or as fast as it can).

But I haven't thought about side effects of that for wider FO usage and that's a PR for another repo in any case!

@pinheadmz
Copy link
Contributor

can remove this commit though? bf2e058

@willcl-ark
Copy link
Contributor Author

ohhhhh, sure

@willcl-ark
Copy link
Contributor Author

rebased and dropped commit

@pinheadmz pinheadmz merged commit f64435b into bitcoin-dev-project:main Nov 2, 2023
6 checks passed
@0xB10C
Copy link
Contributor

0xB10C commented Nov 3, 2023

Would be even nicer (for us) if FO would query on a chill scheduler, e.g. If you configure 100 nodes and query every 60 seconds it could query one node every 0.6 seconds to keep resource usage as flat as possible. Vs currently where every n seconds it will query all nodes at the same time (or as fast as it can).

Agree, I made an issue for this 0xB10C/fork-observer#23 the other day.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants