Skip to content

Cache downloaded files atomically - #9999

Merged
tastelikefeet merged 1 commit into
modelscope:mainfrom
ump45nose:fix/atomic-cached-download
Aug 28, 2026
Merged

Cache downloaded files atomically#9999
tastelikefeet merged 1 commit into
modelscope:mainfrom
ump45nose:fix/atomic-cached-download

Conversation

@ump45nose

Copy link
Copy Markdown
Contributor

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

download_file() now streams into a temporary file in the cache directory and atomically replaces the final cache path only after the response finishes. Failed downloads remove their temporary file, so a later call cannot mistake truncated bytes for a valid cached artifact.

A focused regression test simulates a connection reset and verifies that the cache directory remains empty.

Experiment results

  • RED on current main: an interrupted stream left a final cache file containing b"half"
  • GREEN after the fix: the same interrupted stream leaves no cache entries
  • pre-commit run --files swift/utils/hub_utils.py tests/utils/test_hub_utils.py
  • python3 -m py_compile swift/utils/hub_utils.py tests/utils/test_hub_utils.py
  • git diff --check

@tastelikefeet
tastelikefeet merged commit 3ccc9d5 into modelscope:main Aug 28, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants