Skip to content

refactor: lake: fetch artifact URLs in a single Reservoir request#13164

Merged
tydeu merged 2 commits intoleanprover:masterfrom
tydeu:lake/get-artifact-urls
Mar 28, 2026
Merged

refactor: lake: fetch artifact URLs in a single Reservoir request#13164
tydeu merged 2 commits intoleanprover:masterfrom
tydeu:lake/get-artifact-urls

Conversation

@tydeu
Copy link
Copy Markdown
Member

@tydeu tydeu commented Mar 27, 2026

This PR changes lake cache get to fetch artifact cloud storage URLs from Reservoir in a single bulk POST request rather than relying on per-artifact HTTP redirects. When downloading many artifacts, the redirect-based approach sends one request per artifact to the Reservoir web host (Netlify), which can be slow and risks hitting rate limits. The bulk endpoint returns all URLs at once, so curl only talks to the CDN after that.

Non-Reservoir cache services are unaffected and continue using direct URLs as before.

🤖 Prepared with Claude Code

@tydeu tydeu added changelog-lake Lake lake-ci Run all Lake tests labels Mar 27, 2026
@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Mar 28, 2026
@leanprover-bot
Copy link
Copy Markdown
Collaborator

Reference manual CI status:

  • ❗ Reference manual CI can not be attempted yet, as the nightly-testing-2026-03-25 tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto nightly-with-manual, reference manual CI should run now. You can force reference manual CI using the force-manual-ci label. (2026-03-28 00:26:39)

mathlib-nightly-testing bot pushed a commit to leanprover-community/batteries that referenced this pull request Mar 28, 2026
@github-actions github-actions bot added the mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN label Mar 28, 2026
mathlib-nightly-testing bot pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Mar 28, 2026
@tydeu tydeu marked this pull request as ready for review March 28, 2026 00:38
mathlib-nightly-testing bot pushed a commit to leanprover-community/batteries that referenced this pull request Mar 28, 2026
@mathlib-lean-pr-testing mathlib-lean-pr-testing bot added the builds-mathlib CI has verified that Mathlib builds against this PR label Mar 28, 2026
@mathlib-lean-pr-testing
Copy link
Copy Markdown

Mathlib CI status (docs):

@tydeu tydeu added this pull request to the merge queue Mar 28, 2026
Merged via the queue into leanprover:master with commit d6e6964 Mar 28, 2026
27 checks passed
@tydeu tydeu deleted the lake/get-artifact-urls branch March 28, 2026 05:21
github-merge-queue bot pushed a commit that referenced this pull request Mar 28, 2026
This PR fixes a bug in #13164 where the bulk request would hang if the
response was large.
volodeyka pushed a commit that referenced this pull request Apr 16, 2026
…3164)

This PR changes `lake cache get` to fetch artifact cloud storage URLs
from Reservoir in a single bulk POST request rather than relying on
per-artifact HTTP redirects. When downloading many artifacts, the
redirect-based approach sends one request per artifact to the Reservoir
web host (Netlify), which can be slow and risks hitting rate limits. The
bulk endpoint returns all URLs at once, so curl only talks to the CDN
after that.

Non-Reservoir cache services are unaffected and continue using direct
URLs as before.

🤖 Prepared with Claude Code
volodeyka pushed a commit that referenced this pull request Apr 16, 2026
This PR fixes a bug in #13164 where the bulk request would hang if the
response was large.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

builds-mathlib CI has verified that Mathlib builds against this PR changelog-lake Lake lake-ci Run all Lake tests mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants