Skip to content

urlapi: run the urlparser perf test faster - #22924

Closed
bagder wants to merge 4 commits into
masterfrom
bagder/urlparse-faster
Closed

bagder wants to merge 4 commits into
masterfrom
bagder/urlparse-faster

Conversation

@bagder

@bagder bagder commented Sep 12, 2026 •

Copy link
Copy Markdown
Member

This completes the perf test roughly 17% faster on my machine.

- Batch contiguous unencoded characters into single curlx_dyn_addn()
  calls instead of invoking dynbuf append byte-by-byte
- Inline lowercasing in absolute url parsing
- Make junkscan pass over data only once instead of twice
- Use lookup-table to validate hostname in hostname_check()
- Improve needs_dedotdot with leading memchr calls for dots and percent
- Replace a strcspn() call with a while loop
- Replace several strchr with memchr
- Detect IPv6 earlier and skip some logic
- Skip the full URL junkscan:
  - allows for better return codes for where the problem lies
  - allows for proper support for the URLENCODE flag
  - runs a little faster

This completes the perf test roughly 17% faster on my machine in debug
mode.
@bagder
bagder force-pushed the bagder/urlparse-faster branch from b5e992a to 764c0dc Compare September 12, 2026 16:12
@github-actions github-actions Bot added the tests label Sep 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes security-sensitive URL parsing and validation behavior across multiple URL components.

Pull request overview

Optimizes URL parsing while preserving component-specific validation and error reporting.

Changes:

  • Batches URL-encoding operations and adds parser fast paths.
  • Moves control-character validation to individual URL components.
  • Updates URL API and HTTPS RR test expectations.
File summaries
File Description
lib/urlapi.c Optimizes parsing and component validation.
lib/vdns/doh.c Adds local hostname junk validation.
tests/libtest/lib1560.c Updates URL parsing expectations.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@bagder

bagder commented Sep 12, 2026

Copy link
Copy Markdown
Member Author

augment review

@bagder

bagder commented Sep 12, 2026

Copy link
Copy Markdown
Member Author

@aisle-analyzer

@aisle-research-bot

aisle-research-bot Bot commented Sep 12, 2026 •

Copy link
Copy Markdown

🔒 Aisle Security Analysis

✅ Aisle scan complete — no security issues detected.

Aisle supplements but does not replace security review.

Last updated on: 2026-09-12T16:51:30Z

@augmentcode

augmentcode Bot commented Sep 12, 2026

Copy link
Copy Markdown
🤖 Augment PR Summary

Summary: This PR optimizes URL API parsing to reduce the runtime of the URL parser performance test.

Changes:

  • Batches ordinary characters during URL encoding instead of appending them one byte at a time.
  • Replaces several unbounded string scans with length-aware memchr searches.
  • Uses a byte lookup table for hostname validation and separates IPv6 host handling.
  • Moves global malformed-byte validation into URL component-specific parsing paths.
  • Allows CURLU_URLENCODE to encode control characters in paths, queries, and fragments.
  • Updates scheme lookup to use the known parsed scheme length.
  • Localizes DoH HTTPS-record hostname scanning after removing the shared URL scanner.
  • Updates lib1560 expectations for component-specific URL parsing errors and encoding behavior.
Technical notes: The changes preserve the maximum input-length guard while avoiding repeated full-string scans in hot parsing paths.

🤖 Was this summary useful? React with 👍 or 👎

@augmentcode augmentcode Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review completed. 1 suggestion posted.

Fix All in Augment

Comment augment review to trigger a new review at any time.

Comment thread lib/urlapi.c
They need to be literal
@bagder
bagder marked this pull request as ready for review September 12, 2026 17:02
@bagder bagder closed this in 0b04700 Sep 13, 2026
@bagder
bagder deleted the bagder/urlparse-faster branch September 13, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants