Skip to content

feat(index): parse worker timeouts and recycle#130

Merged
SutuSebastian merged 2 commits into
mainfrom
feat/parse-worker-hardening
May 25, 2026
Merged

feat(index): parse worker timeouts and recycle#130
SutuSebastian merged 2 commits into
mainfrom
feat/parse-worker-hardening

Conversation

@SutuSebastian
Copy link
Copy Markdown
Contributor

Summary

Completes PR 3 stack after #129 (index lock + errors.log):

  • Per-file parse timeout (10s + size scaling, cap 30s; CODEMAP_PARSE_TIMEOUT_MS override)
  • Worker recycle every 250 files (CODEMAP_WORKER_RECYCLE_EVERY)
  • Incremental / targeted indexing uses the worker pool (parity with full rebuild)
  • Parse failures counted in index summary; timeouts logged via errors.log through existing parseError path

Test plan

  • bun test src/application/parse-timeout.test.ts
  • bun test src/worker-pool.test.ts
  • bun test src/application/run-index.test.ts
  • lint-staged on commit

Process one file per worker message with Promise.race timeouts (10–30s
scaled by size), recycle workers every 250 files, route incremental parsing
through the worker pool, and surface parse failure counts in the index summary.
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 25, 2026

🦋 Changeset detected

Latest commit: 467fe68

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@stainless-code/codemap Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 25, 2026

Warning

Review limit reached

@SutuSebastian, we couldn't start this review because you've used your available PR reviews for now.

Your plan includes 1 review of capacity. Refill in 16 minutes and 16 seconds.

Your organization has run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After more review capacity refills, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than trial, open-source, and free plans. In all cases, review capacity refills continuously over time.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f7ca07ad-1f5b-4b98-9046-6bacfbb3e824

📥 Commits

Reviewing files that changed from the base of the PR and between 6648add and 467fe68.

📒 Files selected for processing (7)
  • .changeset/parse-worker-hardening.md
  • docs/plans/agent-surface-delivery.md
  • src/application/index-engine.ts
  • src/application/parse-timeout.test.ts
  • src/application/parse-timeout.ts
  • src/worker-pool.test.ts
  • src/worker-pool.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/parse-worker-hardening

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Keep per-file timeout via chunk bisection on timeout, inline parse for
small batches (<=12 files), and worker recycle. Fixes CLI integration
test timeouts from per-message worker spawn overhead.
@SutuSebastian SutuSebastian merged commit f5374b8 into main May 25, 2026
12 checks passed
@SutuSebastian SutuSebastian deleted the feat/parse-worker-hardening branch May 25, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant