Skip to content

Fix effective worker count collapsing to 1 when ops_rate >> totalDocs…#47

Merged
ashwin2002 merged 1 commit into
mainfrom
doc_loading_enhancement
May 6, 2026
Merged

Fix effective worker count collapsing to 1 when ops_rate >> totalDocs…#47
ashwin2002 merged 1 commit into
mainfrom
doc_loading_enhancement

Conversation

@ashwin2002
Copy link
Copy Markdown
Contributor

…ToProcess

  • High ops values inflate docsPerWorker via batchSize=(ops/workers)*percent/100
  • ceil(totalDocs/docsPerWorker) then produces 1 even when multiple workers requested
  • Add floor of min(ws.workers, totalDocsToProcess) to preserve requested concurrency
  • Avoids silently serializing tests that depend on multiple threads (e.g. dedupe-disable)
  • Still caps workers at totalDocsToProcess to prevent truly idle threads (b1f5d24 preserved)

…ToProcess

- High ops values inflate docsPerWorker via batchSize=(ops/workers)*percent/100
- ceil(totalDocs/docsPerWorker) then produces 1 even when multiple workers requested
- Add floor of min(ws.workers, totalDocsToProcess) to preserve requested concurrency
- Avoids silently serializing tests that depend on multiple threads (e.g. dedupe-disable)
- Still caps workers at totalDocsToProcess to prevent truly idle threads (b1f5d24 preserved)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ashwin2002 ashwin2002 merged commit 65970ee into main May 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant