Skip to content

Conversation

@jonathanpwang
Copy link
Contributor

@jonathanpwang jonathanpwang commented May 29, 2025

also fixes openvm-circuit integration tests

  • fix test_vm_hint

@codspeed-hq
Copy link

codspeed-hq bot commented May 29, 2025

CodSpeed Instrumentation Performance Report

Merging #1696 will not alter performance

Comparing feat/new-execution-benchmarks-fix (7d74e03) with feat/new-execution (3f3147b)

Summary

✅ 10 untouched benchmarks

@github-actions

This comment has been minimized.

@shuklaayush shuklaayush force-pushed the feat/new-execution-benchmarks-fix branch from 757ddbf to 1a97e6b Compare May 30, 2025 19:41
@jonathanpwang jonathanpwang marked this pull request as ready for review May 30, 2025 19:44
@openvm-org openvm-org deleted a comment from codspeed-hq bot May 30, 2025
@codspeed-hq
Copy link

codspeed-hq bot commented May 30, 2025

CodSpeed Walltime Performance Report

Merging #1696 will degrade performances by 16.31%

Comparing feat/new-execution-benchmarks-fix (7d74e03) with feat/new-execution (3f3147b)

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

❌ 10 regressions

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
benchmark_execute[bubblesort] 491.3 ms 586.5 ms -16.24%
benchmark_execute[factorial_iterative_u256] 497.4 ms 585.5 ms -15.04%
benchmark_execute[fibonacci_iterative] 490.1 ms 585.6 ms -16.31%
benchmark_execute[fibonacci_recursive] 468.9 ms 552.7 ms -15.17%
benchmark_execute[keccak256] 487.9 ms 574.4 ms -15.05%
benchmark_execute[keccak256_iter] 479.8 ms 568.8 ms -15.64%
benchmark_execute[quicksort] 494.4 ms 578.8 ms -14.59%
benchmark_execute[revm_snailtracer] 544.8 ms 648.5 ms -15.99%
benchmark_execute[sha256] 482 ms 570.7 ms -15.55%
benchmark_execute[sha256_iter] 481.5 ms 563.9 ms -14.6%

@github-actions

This comment has been minimized.

@jonathanpwang jonathanpwang merged commit 9b217c5 into feat/new-execution May 30, 2025
20 of 21 checks passed
@jonathanpwang jonathanpwang deleted the feat/new-execution-benchmarks-fix branch May 30, 2025 20:21
@github-actions
Copy link

group app.proof_time_ms app.cycles app.cells_used leaf.proof_time_ms leaf.cycles leaf.cells_used
verify_fibair (-108 [-9.1%]) 1,079 334,008 17,675,804 - - -
fibonacci (-265 [-10.6%]) 2,245 1,500,277 50,589,503 - - -
regex (-297 [-4.0%]) 7,068 4,165,432 166,519,456 - - -
ecrecover (+29 [+2.1%]) 1,418 289,547 (+43030 [+0.3%]) 14,513,216 - - -
pairing (-69 [-1.5%]) 4,428 1,820,436 (+346586 [+0.4%]) 96,178,993 - - -
fib_e2e 15,105 12,000,277 404,296,645 19,088 7,703,683 432,129,560
kitchen_sink 20,313 90,774 896,440,796 30,333 10,414,169 923,151,098

Commit: 7d74e03

Benchmark Workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants