Skip to content

Conversation

@penelopeysm
Copy link
Member

@penelopeysm penelopeysm commented Nov 18, 2025

  • Prints more info during the run itself
  • Sorts the table more carefully so that options that are similar to each other are grouped together
  • Adds extra lines to help guide the eye
  • Adds tgrad / tref column so that we can tell how much something improves gradients without having to do mental multiplication

@github-actions
Copy link
Contributor

github-actions bot commented Nov 18, 2025

Benchmark Report

  • this PR's head: 4e57712b1813861d11e157b2fc093a353c5a0414
  • base branch: 8553e401182b894a653b638e5978f00ae42ae031

Computer Information

Julia Version 1.11.7
Commit f2b3dbda30a (2025-09-08 12:10 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × AMD EPYC 7763 64-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)
Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)

Benchmark Results

┌───────────────────────┬───────┬─────────────┬───────────────────┬────────┬─────────────────────────────────┬───────────────────────────┬─────────────────────────────────┐
│                       │       │             │                   │        │        t(eval) / t(ref)         │     t(grad) / t(eval)     │        t(grad) / t(ref)         │
│                       │       │             │                   │        │ ──────────┬───────────┬──────── │ ──────┬─────────┬──────── │ ──────────┬───────────┬──────── │
│                 Model │   Dim │  AD Backend │           VarInfo │ Linked │      base │   this PR │ speedup │  base │ this PR │ speedup │      base │   this PR │ speedup │
├───────────────────────┼───────┼─────────────┼───────────────────┼────────┼───────────┼───────────┼─────────┼───────┼─────────┼─────────┼───────────┼───────────┼─────────┤
│               Dynamic │    10 │    mooncake │             typed │   true │    519.33 │    380.85 │    1.36 │  8.96 │   11.58 │    0.77 │   4651.05 │   4410.28 │    1.05 │
│                   LDA │    12 │ reversediff │             typed │   true │   2945.40 │   2704.08 │    1.09 │  1.98 │    1.97 │    1.00 │   5822.08 │   5321.36 │    1.09 │
│   Loop univariate 10k │ 10000 │    mooncake │             typed │   true │ 175512.86 │ 142763.28 │    1.23 │  5.00 │    5.78 │    0.87 │ 878051.65 │ 824952.49 │    1.06 │
├───────────────────────┼───────┼─────────────┼───────────────────┼────────┼───────────┼───────────┼─────────┼───────┼─────────┼─────────┼───────────┼───────────┼─────────┤
│    Loop univariate 1k │  1000 │    mooncake │             typed │   true │  15848.71 │  14577.39 │    1.09 │  5.19 │    5.45 │    0.95 │  82213.87 │  79396.63 │    1.04 │
│      Multivariate 10k │ 10000 │    mooncake │             typed │   true │  35443.18 │  28195.34 │    1.26 │  9.02 │   10.11 │    0.89 │ 319647.48 │ 284970.67 │    1.12 │
│       Multivariate 1k │  1000 │    mooncake │             typed │   true │   3884.77 │   3351.04 │    1.16 │  8.41 │    8.93 │    0.94 │  32685.99 │  29912.61 │    1.09 │
├───────────────────────┼───────┼─────────────┼───────────────────┼────────┼───────────┼───────────┼─────────┼───────┼─────────┼─────────┼───────────┼───────────┼─────────┤
│ Simple assume observe │     1 │ forwarddiff │             typed │  false │     18.08 │     16.52 │    1.09 │  1.89 │    1.83 │    1.03 │     34.26 │     30.27 │    1.13 │
│           Smorgasbord │   201 │ forwarddiff │             typed │  false │   2669.01 │   2411.90 │    1.11 │ 85.58 │   45.30 │    1.89 │ 228402.83 │ 109269.90 │    2.09 │
│           Smorgasbord │   201 │ forwarddiff │       simple_dict │   true │  24579.98 │  23757.61 │    1.03 │ 25.83 │   26.24 │    0.98 │ 634861.41 │ 623463.26 │    1.02 │
├───────────────────────┼───────┼─────────────┼───────────────────┼────────┼───────────┼───────────┼─────────┼───────┼─────────┼─────────┼───────────┼───────────┼─────────┤
│           Smorgasbord │   201 │ forwarddiff │ simple_namedtuple │   true │   1086.36 │    990.94 │    1.10 │ 86.99 │   78.38 │    1.11 │  94507.29 │  77673.10 │    1.22 │
│           Smorgasbord │   201 │      enzyme │             typed │   true │   2648.14 │   2420.15 │    1.09 │  5.43 │    5.08 │    1.07 │  14375.94 │  12306.01 │    1.17 │
│           Smorgasbord │   201 │    mooncake │             typed │   true │   2657.08 │   2414.65 │    1.10 │  5.65 │    5.83 │    0.97 │  15021.01 │  14074.25 │    1.07 │
├───────────────────────┼───────┼─────────────┼───────────────────┼────────┼───────────┼───────────┼─────────┼───────┼─────────┼─────────┼───────────┼───────────┼─────────┤
│           Smorgasbord │   201 │ reversediff │             typed │   true │   2639.04 │   2433.91 │    1.08 │ 56.44 │   56.41 │    1.00 │ 148942.12 │ 137293.07 │    1.08 │
│           Smorgasbord │   201 │ forwarddiff │      typed_vector │   true │   2700.46 │   2437.90 │    1.11 │ 48.02 │   45.26 │    1.06 │ 129688.24 │ 110349.57 │    1.18 │
│           Smorgasbord │   201 │ forwarddiff │           untyped │   true │   2370.40 │   2167.86 │    1.09 │ 48.52 │   44.90 │    1.08 │ 115021.81 │  97339.44 │    1.18 │
├───────────────────────┼───────┼─────────────┼───────────────────┼────────┼───────────┼───────────┼─────────┼───────┼─────────┼─────────┼───────────┼───────────┼─────────┤
│           Smorgasbord │   201 │ forwarddiff │    untyped_vector │   true │   2389.78 │   2195.51 │    1.09 │ 49.17 │   48.19 │    1.02 │ 117499.70 │ 105791.06 │    1.11 │
│              Submodel │     1 │    mooncake │             typed │   true │     26.44 │     25.05 │    1.06 │  5.25 │    5.27 │    0.99 │    138.66 │    132.09 │    1.05 │
└───────────────────────┴───────┴─────────────┴───────────────────┴────────┴───────────┴───────────┴─────────┴───────┴─────────┴─────────┴───────────┴───────────┴─────────┘

@codecov
Copy link

codecov bot commented Nov 18, 2025

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (breaking@8553e40). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/test_utils/ad.jl 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             breaking    #1146   +/-   ##
===========================================
  Coverage            ?   81.67%           
===========================================
  Files               ?       42           
  Lines               ?     3930           
  Branches            ?        0           
===========================================
  Hits                ?     3210           
  Misses              ?      720           
  Partials            ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@penelopeysm penelopeysm merged commit 3cd8d34 into breaking Nov 18, 2025
21 checks passed
@penelopeysm penelopeysm deleted the py/bench1 branch November 18, 2025 17:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants