Fix issue #190: Implement robust retry logic for downloads - #203
Merged
Conversation
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 0 |
| Duplication | 0 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
This commit addresses the issue where files could appear incomplete or corrupt when encountering HTTP 429 errors or network interruptions. Key changes: - Added a retry loop to Link_download for range requests. - Fixed a bug in Link_download_full where data would accumulate on retries. - Resolved a resource leak and potential crash in Link_download_cleanup. - Moved blocking sleep from network.c to individual request handlers.
fangfufu
force-pushed
the
fix-issue-190-retry-logic
branch
from
May 9, 2026 23:57
c98cbe5 to
393bef8
Compare
|
This was referenced May 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request addresses issue #190 where files could appear incomplete or corrupt when encountering HTTP 429 (Rate Limited) errors or network interruptions.
Technical Analysis
After analyzing the source code in
src/link.candsrc/network.c, I identified several root causes for the reported behavior:Link_download: The function responsible for reading file content (via range requests) lacked a retry mechanism. Upon encountering an HTTP 429 error or a truncated download, it logged an error but proceeded with whatever partial data it had received.Link_downloadwould still attempt to callmemmoveon the data buffer using the negative error code (e.g.,-ENOENT) as the size. This could lead to memory corruption or crashes, and certainly resulted in incorrect read returns.network.cused a blockingsleep()while holding the globaltransfer_lock. This effectively froze the entire filesystem's network activity for all threads without actually retrying the failed request.Link_download_fulldid not reset the transfer buffer between attempts. Since the write callback appends data, a retry after a 429 would prepended the error page's HTML to the actual file content.Fixes Applied
Link_downloadnow includes ado-whileloop that handles both HTTP temporary failures and partial downloads by retrying until the full range is successfully retrieved.Link_downloadreturns immediately on unrecoverable errors, avoiding invalid memory operations on the output buffer.Link_download_fullto clear its data buffer before each retry attempt to prevent data corruption.Link_download_cleanupby ensuring the curl handle is always cleaned up, even on error paths.These improvements make
httpdirfsmuch more resilient to aggressive server-side rate limiting and unstable network connections.