docs(performance): republish the comparative table at 3.2.0 - #415
Merged
Conversation
Regenerated with just bench-report (5 paired runs, reproduced by a second independent 5-run pass). Every absolute is 23-29% above the 2026-07-28 publication for all five frameworks on identical library, rival, CPython and OS versions -- the machine was slower on the day -- so the page now states that outright and reads the ratios against it. Against that shift, C2 and C6 moved toward modern-di and C1/C3/C4 did not, which matches exactly what 3.2.0 touched. modern-di's warm singleton is now faster than dishka's on dishka's own by-type basis (0.98), the first time that cell has gone below 1.0. The event-loop floor inside every C4 cell was re-measured on the same machine (0.31 -> 0.35 us per request). The alias hop has no cell here; the suite has no alias scenario. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Benchmark
Details
| Benchmark suite | Current: a783d34 | Previous: bb8794c | Ratio |
|---|---|---|---|
benchmarks/test_guard_by_type.py::test_g16_resolve_by_type |
2164401.082438699 iter/sec (stddev: 2.3648389538978368e-7) |
2019181.2353840743 iter/sec (stddev: 2.666764939640821e-7) |
0.93 |
benchmarks/test_guard_by_type.py::test_g17_resolve_by_type_large_registry |
2671665.888731955 iter/sec (stddev: 5.442083947869368e-8) |
2573115.8072158024 iter/sec (stddev: 6.926923363934469e-8) |
0.96 |
benchmarks/test_guard_cold.py::test_g8_cold_first_resolve |
23877.899821643336 iter/sec (stddev: 0.000020834385935854022) |
24097.39715231142 iter/sec (stddev: 0.00001787777072004501) |
1.01 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[1] |
427.92736293575393 iter/sec (stddev: 0.00003489070252223252) |
431.67108978858033 iter/sec (stddev: 0.00008907895116799332) |
1.01 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[2] |
404.9740240554243 iter/sec (stddev: 0.00006366351467864004) |
396.46131091162596 iter/sec (stddev: 0.000047395030597096453) |
0.98 |
benchmarks/test_guard_concurrency.py::test_g14_concurrent_cached_hit[4] |
359.3159382815722 iter/sec (stddev: 0.00015467660006469294) |
351.15399216035996 iter/sec (stddev: 0.0001675935139304426) |
0.98 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[1] |
2294.948546679101 iter/sec (stddev: 0.000028650654652852542) |
1915.1996822300337 iter/sec (stddev: 0.00009439567009852235) |
0.83 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[2] |
1588.730204604128 iter/sec (stddev: 0.00034945455977955525) |
1475.0720566551136 iter/sec (stddev: 0.0003267425748683087) |
0.93 |
benchmarks/test_guard_concurrency.py::test_g15_concurrent_first_resolve[4] |
1193.6097594942505 iter/sec (stddev: 0.00003609625055284982) |
1086.933896973317 iter/sec (stddev: 0.00004751332275126381) |
0.91 |
benchmarks/test_guard_lifecycle.py::test_g6_build_child_container |
668360.0795311286 iter/sec (stddev: 4.808512275897948e-7) |
681375.2366989609 iter/sec (stddev: 0.0000011975191328828107) |
1.02 |
benchmarks/test_guard_lifecycle.py::test_g6b_build_child_container_auto_scope |
629105.7951969058 iter/sec (stddev: 4.93666191215643e-7) |
648770.3052589889 iter/sec (stddev: 4.6421942393821645e-7) |
1.03 |
benchmarks/test_guard_lifecycle.py::test_g7_request_lifecycle_batch |
2402.799082861268 iter/sec (stddev: 0.00001918674188455211) |
2322.6008299153827 iter/sec (stddev: 0.00003192617391221412) |
0.97 |
benchmarks/test_guard_lifecycle.py::test_g7c_event_loop_floor_control |
62181.66777729213 iter/sec (stddev: 0.0000020346677352899183) |
62612.44186400999 iter/sec (stddev: 0.0000019367844975850882) |
1.01 |
benchmarks/test_guard_lifecycle.py::test_g13_teardown_at_scale |
46597.43742679612 iter/sec (stddev: 0.0000020693079263258163) |
46373.322180892625 iter/sec (stddev: 0.0000021399737339881666) |
1.00 |
benchmarks/test_guard_resolve.py::test_g1_transient_resolve |
1346128.8054790264 iter/sec (stddev: 7.377308180034315e-7) |
1330558.9044018583 iter/sec (stddev: 3.5143313542242677e-7) |
0.99 |
benchmarks/test_guard_resolve.py::test_g2_cached_resolve |
3420318.795433564 iter/sec (stddev: 5.062500962017588e-8) |
3289436.386684446 iter/sec (stddev: 4.8116935656836214e-8) |
0.96 |
benchmarks/test_guard_resolve.py::test_g3_deep_chain |
522401.88670970744 iter/sec (stddev: 4.6938294849708725e-7) |
516234.312285702 iter/sec (stddev: 5.562755524293426e-7) |
0.99 |
benchmarks/test_guard_resolve.py::test_g4_wide_resolve |
324940.7494024595 iter/sec (stddev: 5.88039607071334e-7) |
324763.134943507 iter/sec (stddev: 6.578727884301413e-7) |
1.00 |
benchmarks/test_guard_resolve.py::test_g5_cross_scope |
1163091.192513428 iter/sec (stddev: 3.561650722500084e-7) |
1131250.1665050173 iter/sec (stddev: 3.659748695694099e-7) |
0.97 |
benchmarks/test_guard_resolve.py::test_g9_context_resolve |
645865.8156131203 iter/sec (stddev: 4.811536505466293e-7) |
650099.8557224047 iter/sec (stddev: 5.042087123500081e-7) |
1.01 |
benchmarks/test_guard_resolve.py::test_g12_override_active_resolve |
392731.3919028174 iter/sec (stddev: 4.681645734967068e-7) |
391370.2035443222 iter/sec (stddev: 4.3516683943055756e-7) |
1.00 |
benchmarks/test_guard_validate.py::test_g10_validate_deep_chain |
28370.930127348573 iter/sec (stddev: 0.0000035748254524779354) |
27353.337807909375 iter/sec (stddev: 0.0000045394108062455774) |
0.96 |
benchmarks/test_guard_validate.py::test_g11_validate_wide |
17018.9170255009 iter/sec (stddev: 0.000005312913804031902) |
16743.572312827022 iter/sec (stddev: 0.000004839805353893621) |
0.98 |
This comment was automatically generated by workflow using github-action-benchmark.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
docs/introduction/performance.mdreported modern-di 3.1.2 and predated all four of 3.2.0's resolve-path trims. It is the page carrying our comparative claims against dependency-injector, dishka, that-depends and wireup, and it was one published version behind whatpip install modern-dinow gives you.Design
Tables regenerated with
just bench-report(5 paired runs) and published verbatim -- never hand-assembled. I ran it a second time, independently, before writing any prose: the two passes agree closely (C1 347/353 ns, C2 157/157, C3 957/965; every ratio within 0.05), so the run published here is a reproducible measurement rather than one sample.The finding that shaped the rewrite: every absolute on the page is 23-29% higher than the 2026-07-28 publication -- for all five frameworks, on the same machine, same macOS 26.5, same CPython 3.14.6, same pinned rival versions. The implied rival absolutes this page already prints moved as a block (dependency-injector's C2 hit ~48 -> ~60 ns, that-depends ~67 -> ~83, dishka ~172 -> ~215, wireup ~76 -> ~95). The machine was slower on the day. Rather than publish a 25% worse absolute column and let readers conclude modern-di regressed, the page now states the shift outright and reads the ratios against it -- which is what the paired-per-run methodology exists for.
Against that baseline, the cells that moved are exactly the paths 3.2.0 touched:
C1/C3/C4 are the control and behave like one: twelve ratio cells, max move 0.04, four up / three down / five unchanged. C2's four cells moved 0.08-0.31 and C6's 0.03-0.11, every one toward modern-di.
The arithmetic is stated so it can be checked: modern-di's C2 rose 10.6% against a pack that rose 24-25%; unchanged it would sit near 176 ns, and 3.2.0's
MAKE_CELLremoval measures -11.3%, giving 156 against the 157 published.modern-di's warm singleton is now faster than dishka's on dishka's own by-type basis (0.98) -- the first publication where that cell is below 1.0. I have written it as "level, marginally ahead" rather than a decisive lead, because the spread comes close to covering 1.00.
Also re-measured on the same machine: the asyncio floor inside every C4 cell, 0.31 -> 0.35 us per request, since that figure is machine-dependent and would otherwise have gone stale silently.
Non-goals
Verification
just lint-ci: clean, including the Markdown link/anchor checker.just check-links: OK.just docs-buildcould not run locally -- it fetches mkdocs from PyPI and this environment has no network for it. CI runs it on this PR, which is the check that matters for the nav and link warnings.Every ratio, absolute and derived figure in the prose was recomputed from the published run; I grepped for surviving 3.1.2-era numbers and the only remaining ones are deliberate historical comparisons and the version-history section.
Before merging
just lint-cipasses;just test-ciunaffected (docs only).