Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

getHeader and submitBlindBlock latencies #203

Closed
terencechain opened this issue Jul 11, 2022 · 8 comments
Closed

getHeader and submitBlindBlock latencies #203

terencechain opened this issue Jul 11, 2022 · 8 comments
Labels

Comments

@terencechain
Copy link
Collaborator

Not sure if this is the best place to track this. Feel free to close this and let me know a more appropriate place, we can move the issue somewhere else.

I've been observing high latencies with both getStatus, getHeader and submitBlindBlock calls using the Kiln relay.

getStatus, between 100-500ms
Screen Shot 2022-07-11 at 2 34 00 PM

getHeader, this gotten worse in the last 24 hours, many calls exceeded 1-2s
Screen Shot 2022-07-11 at 2 33 26 PM

submitBlindBlock, mostly around 200ms
Screen Shot 2022-07-11 at 2 32 49 PM

This is a prysm beacon node getting a beacon block for validator to sign. We are seeing 10-25x latency increases
Screen Shot 2022-07-11 at 2 37 25 PM

I understand that many moving parts are currently under optimized, but I still want to open an issue for tracking

@metachris
Copy link
Collaborator

That's great, thank you for reporting. Well investigate today and update here.

@metachris
Copy link
Collaborator

We've updated our Relay and builder infrastructure across all the supported testnets (Kiln, Ropsten and Sepolia), and we're seeing very quick response timings now (in the single-digit milliseconds mostly). Also it's not necessary anymore to batch validator registrations in small chunks, you can many registrations with a single call without problems.

Let us know if you can confirm the numbers from your end, or if we need to further investigate network delays.

@terencechain
Copy link
Collaborator Author

getHeader has gotten much better 🎉
Screen Shot 2022-07-13 at 8 09 37 AM

submitBlindBlock and getStatus are pretty much the same as before. 95% percentile still around ~200-500ms.
We won't be calling getStatus anymore. Only once during startup so that's not going to be an issue for us. Let me know if you want me to close this and just track submitBlindBlock separately

@tbenr
Copy link

tbenr commented Jul 13, 2022

Last 24h for us on Ropsten.
Focus on the pies, we had 6 slow responses (between 1s and 2s) for GetHeader and 3 slow (between 300 and 500 ms) for GetPayload

image

@tbenr
Copy link

tbenr commented Jul 13, 2022

Last 24h for us on Sepolia.

image

@tbenr
Copy link

tbenr commented Jul 13, 2022

registration response time improved significantly

image

@metachris
Copy link
Collaborator

@tbenr @terencechain do you have any current charts for the last days/week?

@tbenr
Copy link

tbenr commented Jul 22, 2022

Last 2 days in sepolia:
image
getPayload:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants