Skip to content

hash: vary bucket placement across process runs - #22916

Closed
MegaManSec wants to merge 1 commit into
curl:masterfrom
MegaManSec:worktree-agent-ad71ca556167a60fa
Closed

MegaManSec wants to merge 1 commit into
curl:masterfrom
MegaManSec:worktree-agent-ad71ca556167a60fa

Conversation

@MegaManSec

@MegaManSec MegaManSec commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

hash_str() always starts from the same value, so a given key set lands in the same buckets on every run. Mix in a random seed generated at global init.

This does not reduce the collision probability or change the worst case; it only stops a clustering from repeating between runs.

unit1603 covers the seeded calculation.

@MegaManSec
MegaManSec force-pushed the worktree-agent-ad71ca556167a60fa branch from 95b582e to 995e3e3 Compare September 11, 2026 14:41
@MegaManSec MegaManSec changed the title hash: seed Curl_hash_str to prevent offline collision precomputation hash: vary seed to avoid pathological bucket clustering Sep 11, 2026

@bagder bagder left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

@MegaManSec
MegaManSec force-pushed the worktree-agent-ad71ca556167a60fa branch from 995e3e3 to b25b9d2 Compare September 11, 2026 14:50
@MegaManSec
MegaManSec requested a review from bagder September 11, 2026 14:51
@bagder
bagder requested a balanced review from Copilot September 11, 2026 14:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Random-generation failure permanently disables the mitigation, and the seeded path lacks test coverage.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds process-lifetime randomization to byte-key hash bucket selection to reduce repeatable clustering.

Changes:

  • Initializes a random hash seed during global setup.
  • Mixes the seed into string hashing.
  • Adds required declarations.
File summaries
File Description
lib/rand.h Forward-declares Curl_easy.
lib/hash.h Declares hash global initialization.
lib/hash.c Generates and applies the hash seed.
lib/easy.c Initializes the seed during global setup.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread lib/hash.c Outdated
Comment thread lib/hash.c Outdated
@jay

jay commented Sep 11, 2026

Copy link
Copy Markdown
Member

Couldn't we have the same issue whether there's a random seed or not. Like it's already random in a sense.

@MegaManSec MegaManSec changed the title hash: vary seed to avoid pathological bucket clustering hash: vary bucket placement across process runs Sep 11, 2026
@github-actions github-actions Bot added the tests label Sep 11, 2026
@MegaManSec

MegaManSec commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor Author

@jay it doesn't reduce the normal collision probability or change the O(n) worst case, but it makes bucket placement vary between runs, so any that happens clustering isn't guaranteed to repeat. updated pr description to be a bit moreclear

@jay

jay commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

What I'm saying is it already isn't necessarily repeating because the input will vary but say you have the same command or input then yes with this pr the hash would vary the same strings between runs. I wonder why we should do that though if the input is already "random" even if it's the same over several runs.

@testclutch

Copy link
Copy Markdown

Analysis of PR #22916 at 2f40fa96:

Test 1603 failed, which has NOT been flaky recently, so there could be a real issue in this PR. Note that this test has failed in 46 different CI jobs (the link just goes to one of them).

Generated by Testclutch

@bagder

bagder commented Sep 13, 2026

Copy link
Copy Markdown
Member

The IntelC CI warning looks real.

Also, the mbedtls failures I believe is because the test case does not call curl_global_init() and for mbedTLS that means it didn't seed the random engine so things fail later.

hash_str() always starts from the same value, so a given key set lands
in the same buckets on every run. Mix in a random seed generated at
global init.
@MegaManSec
MegaManSec force-pushed the worktree-agent-ad71ca556167a60fa branch 2 times, most recently from f01b86d to c00cd23 Compare September 14, 2026 09:22
@dfandrich

dfandrich commented Sep 14, 2026 via email

Copy link
Copy Markdown
Contributor

@bagder bagder closed this in 32c6bd5 Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

6 participants