Fix intermittent ImageNotFound errors on large image builds caused by EMFILE - #17650
Open
astrobot-houston wants to merge 1 commit into
Open
Fix intermittent ImageNotFound errors on large image builds caused by EMFILE#17650astrobot-houston wants to merge 1 commit into
astrobot-houston wants to merge 1 commit into
Conversation
…ency limiting and retry Add a concurrency-limited file reader (`readFileWithRetry`) to `emitImageMetadata` that caps concurrent `fs.readFile` calls to 200 and retries transient OS errors (EMFILE, ENFILE, EAGAIN, EBUSY) with exponential backoff. The bare `catch` block that previously swallowed all errors now only returns `undefined` for ENOENT (file not found), re-throwing other errors with the real OS message instead of the misleading "Could not find requested image... Does it exist?" Fixes #17649
1 task
🦋 Changeset detectedLatest commit: 1df8c97 The changes in this PR will be included in the next version bump. This PR includes changesets to release 417 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
emitImageMetadatanow uses a concurrency-limited file reader (max 200 simultaneousfs.readFilecalls) to prevent exhausting OS file descriptors on projects with tens of thousands of images. Previously, all image imports were read concurrently with no limit, causingEMFILE: too many open fileserrors — especially afterastro checkorastro devalready consumed file descriptors.EMFILE,ENFILE,EAGAIN,EBUSY) are retried with exponential backoff instead of failing immediately.catchthat silently swallowed all errors (includingEMFILE) is replaced: onlyENOENTreturnsundefined; other errors are re-thrown with the real OS error message. This makesImageNotFoundaccurate — it now only fires when the file genuinely doesn't exist.Closes #17649
Testing
packages/astro/test/units/assets/emit-image-metadata.test.tscovering:undefinedid returnsundefined, a missing file (ENOENT) returnsundefined, and a real JPEG file returns correct width/height/format metadata.Docs
No docs update needed — this is a build reliability fix with no user-facing API change.