Skip to content

Speed up multipart callback dispatch - #317

Closed
Kludex wants to merge 4 commits into
mainfrom
speed-up-multipart-callback-dispatch
Closed

Speed up multipart callback dispatch#317
Kludex wants to merge 4 commits into
mainfrom
speed-up-multipart-callback-dispatch

Conversation

@Kludex

@Kludex Kludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner

Summary

  • normalize omitted and explicit-None multipart callbacks to shared no-op functions once during parser initialization
  • cache callbacks in private dispatch slots and keep them synchronized through set_callback()
  • dispatch callbacks directly instead of constructing callback names and calling BaseParser.callback() for every event
  • pass resolved data callbacks into the existing mark/lookbehind helper

Performance

CodSpeed reports a 24.15% efficiency improvement for the corrected 100-field complete-parse benchmark, from 3.7 ms to 3.0 ms. The other four corrected benchmarks are unchanged. CodSpeed notes that the compared runs used different runtime environments, so the same-environment local results below provide additional context.

Same-environment local wall-time benchmarks on Python 3.13:

Benchmark Base Head Improvement
Simple multipart form 15.53 µs 13.63 µs 12.2%
100-field multipart form 403.28 µs 334.28 µs 17.1%
1,000-field multipart form 3,988.07 µs 3,341.27 µs 16.2%
8 MiB file upload 605.73 µs 599.75 µs effectively unchanged
1 MiB CR/LF-dense upload 57.96 µs 55.52 µs 4.2%

A Starlette-shaped 100-field multipart parse improved from approximately 661 µs to 616 µs (about 6.8%).

Validation

  • 162 tests pass
  • 100% statement coverage
  • Ruff passes
  • mypy passes
  • source distribution check passes

Review in cubic

@codspeed-hq

codspeed-hq Bot commented Aug 9, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 24.15%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
✅ 4 untouched benchmarks
⏩ 5 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation test_parse_multipart_large_form 3.7 ms 3 ms +24.15%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing speed-up-multipart-callback-dispatch (d74fe57) with main (d9cb4c6)

Open in CodSpeed

Footnotes

  1. 5 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c875e1e81

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread python_multipart/multipart.py Outdated
@Kludex

Kludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

After rebasing onto the corrected benchmark lifecycle from #318, CodSpeed now reports a 24.15% efficiency improvement for the 100-field complete-parse benchmark (3.7 ms to 3.0 ms), with the other four corrected benchmarks unchanged. The report warns that the runs used different environments, while same-environment local measurements show a 17.1% improvement for the equivalent workload. The Cubic finding is marked addressed in d74fe57 and its thread is resolved; no further code change is needed.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread tests/test_multipart.py Outdated
@Kludex

Kludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

The current CodSpeed comparison is not representative because the benchmark fixture reuses an already-finished parser across measured iterations. I opened #318 to create and finalize a fresh parser per benchmark invocation. PR #317 should be rebased and evaluated again after #318 merges; the corrected baseline will provide a meaningful performance comparison.

@Kludex
Kludex force-pushed the speed-up-multipart-callback-dispatch branch from c83a120 to d74fe57 Compare August 9, 2026 12:29
@Kludex

Kludex commented Aug 9, 2026

Copy link
Copy Markdown
Owner Author

Rebased onto main after #318 merged. CodSpeed will now compare PR #317 against fresh-parser, complete-request benchmark iterations. Also addressed the Cubic review in d74fe57 by testing set_callback("part_data", None) and resolving the thread.

@Kludex Kludex closed this Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant