fix(analyzer): bound native-build registry concurrency - #3988
Merged
Conversation
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Description
MAX_CONCURRENT_REGISTRY_QUERIES = 4inreview-enrichment/src/analyzers/native-build.tsto limit simultaneous registry calls.mapWithConcurrencyand replace the previousPromise.all(changes.map(...))fanout withmapWithConcurrency(changes, MAX_CONCURRENT_REGISTRY_QUERIES, ...)insidescanNativeBuildso lookups are throttled.MAX_QUERIES,MAX_NPM_VERSION_JSON_BYTES,MAX_PYPI_VERSION_JSON_BYTES) unchanged and retain result ordering and behavior ofscanNativeBuildandnpmNativeBuild.review-enrichment/test/native-build.test.tsthat submits multiple PyPI additions and asserts that all are processed while the observedmaxActiveconcurrent fetches never exceeds the concurrency cap.T | undefinedbeing passed into the worker (items[index]!) to satisfy the compiler.Testing
npm --prefix review-enrichment run build(REES build) — succeeded.node --test --experimental-strip-types review-enrichment/test/native-build.test.ts— all tests in that file passed, including the new concurrency regression test.npm run rees:test— review-enrichment tests passed locally.npm run test:ci; the REES pieces passed but the fulltest:cirun failed in unrelated root-level coverage/unit shards (roottest:coverageencountered timeouts / failures intest/unit/queue.test.tsunrelated to this change).npm audit --audit-level=moderatecould not complete in this environment due to the registry returning403 Forbiddenfor the audit endpoint.Codex Task