Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bench signature verification #16

Merged
merged 1 commit into from Jul 11, 2022
Merged

Conversation

metachris
Copy link
Collaborator

@metachris metachris commented Jul 7, 2022

📝 Summary

Add a benchmark for signature verification

make bench outputs:

aws i3en.large - cpu: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz

BenchmarkSignatureVerification-2   	     705	   1,735,334 ns/op

aws m1.metal

BenchmarkSignatureVerification-16    	     223	   5,377,079 ns/op

m1 max mbp - cpu: VirtualApple @ 2.50GHz

BenchmarkSignatureVerification-10            784           1,502,121 ns/op

Hetzner: cpu: AMD Ryzen 7 3700X 8-Core Processor

BenchmarkSignatureVerification-16    	     872	   1,313,259 ns/op

✅ I have run these commands

  • make lint
  • make test
  • go mod tidy

@metachris metachris requested a review from Ruteri July 7, 2022 18:53
@metachris metachris force-pushed the bench-signature-verification branch 2 times, most recently from 9fffe6d to 297ca97 Compare July 7, 2022 19:48
@metachris metachris requested a review from jtraglia July 7, 2022 19:59
@metachris metachris force-pushed the bench-signature-verification branch from 297ca97 to 563afbe Compare July 7, 2022 20:21
Copy link
Collaborator

@jtraglia jtraglia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I'm not a fan of how go reports the results, but I don't think there's anything you can do about that. For example, I wish there were a table header. Without looking at the docs, I'm unsure what the first column is.

I also tested it on Linux/ARM64:

goos: linux
goarch: arm64
pkg: github.com/flashbots/go-boost-utils/types
BenchmarkSignatureVerification-8   	    1365	    872678 ns/op	    4929 B/op	      19 allocs/op
PASS
ok  	github.com/flashbots/go-boost-utils/types	1.290s

@metachris
Copy link
Collaborator Author

Thanks for the test! Yes, the output format is not great :(

The first column is the number of times the code has run. It seems on your ARM machine it was the fastest of all the test environments. What CPU is that?

@metachris metachris force-pushed the bench-signature-verification branch from 563afbe to 4dc46d8 Compare July 8, 2022 08:49
@metachris metachris force-pushed the bench-signature-verification branch from 4dc46d8 to f551695 Compare July 8, 2022 08:49
@jtraglia
Copy link
Collaborator

jtraglia commented Jul 8, 2022

Those results are from my Mac Studio (M1 Ultra) running a Linux VM with 8 CPU cores.

Here are some other results:

Mac Studio (M1 Ultra); macOS; go1.18.0:

BenchmarkSignatureVerification-20    	    1491	    804403 ns/op	    4933 B/op	      19 allocs/op

MacBook Air (M1); macOS; go1.18.1:

BenchmarkSignatureVerification-8   	    1471	    815220 ns/op	    4931 B/op	      19 allocs/op

Which makes sense. The M1 Ultra is essentially a scaled up base M1 processor. They use the same cores, but on M1 Pro/Max/Ultra performance cores are at a slightly higher clock (3.23 GHz vs 3.20 GHz) than the base M1 performance core. The results for MacBook Pro with M1 Max in your comment seem wrong. Which version of golang is being used there?

Also, it seems that this benchmark is only being run on a single core. Is that intended?

@metachris metachris merged commit de5c762 into main Jul 11, 2022
@metachris metachris deleted the bench-signature-verification branch July 11, 2022 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants