Skip to content

fix(docker): add retry logic for ML model downloads#248

Merged
nicoloboschi merged 1 commit intovectorize-io:mainfrom
slayoffer:fix/docker-ml-download-retry
Jan 30, 2026
Merged

fix(docker): add retry logic for ML model downloads#248
nicoloboschi merged 1 commit intovectorize-io:mainfrom
slayoffer:fix/docker-ml-download-retry

Conversation

@slayoffer
Copy link
Contributor

Summary

  • Add 3 retries with exponential backoff (10s → 20s → 40s) for HuggingFace model downloads
  • Set HF_HUB_DOWNLOAD_TIMEOUT=600 for longer timeout
  • Applied to both api-only and standalone stages

Problem

Docker builds occasionally fail with transient network errors when downloading ML models from HuggingFace during the image build. This is especially common in CI environments or when HuggingFace CDN has temporary issues.

Solution

Wrap the model download in a retry loop with exponential backoff. After 3 failed attempts, the build fails with a clear error message.

Test plan

  • Built image locally with retry logic
  • Verified models download successfully on first attempt
  • Simulated failure by temporarily blocking HuggingFace (retry logic triggers)

🤖 Generated with Claude Code

- Add 3 retries with exponential backoff (10s -> 20s -> 40s)
- Set HF_HUB_DOWNLOAD_TIMEOUT=600 for longer timeout
- Fixes transient network failures during HuggingFace downloads
- Applied to both api-only and standalone stages

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Collaborator

@nicoloboschi nicoloboschi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I was going to add it soon :)

@nicoloboschi nicoloboschi merged commit ecc590c into vectorize-io:main Jan 30, 2026
17 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants