Skip to content

Pre-load offers catalog#3785

Merged
r4victor merged 4 commits intomasterfrom
issue_3479_optimize_cold_cache_offers
Apr 16, 2026
Merged

Pre-load offers catalog#3785
r4victor merged 4 commits intomasterfrom
issue_3479_optimize_cold_cache_offers

Conversation

@r4victor
Copy link
Copy Markdown
Collaborator

@r4victor r4victor commented Apr 16, 2026

#3479

  • Add a server background task that periodically pre-loads gpuhunt offers catalog, thus avoiding catalog download on query (when getting offers).
  • Speed up AWS quotas collection by switching from list_service_quotas to get_service_quota.
  • Updated gpuhunt to make catalog.load() thread-safe.

Improvements

Offline providers:

  • amddevcloud — 1.71s => 1.34s (-22% – network variability)
  • aws — 41.43s => 6.61s (-84%)
  • azure — 12.49s => 5.50s (-56%)
  • gcp — 13.51s => 5.20s (-62%)
  • lambda — 8.75s => 2.18s (-75%)
  • nebius — 10.74s => 3.80s (-65%)
  • oci — 24.48s => 18.27s (-25%)
  • runpod — 9.36s => 0.09s (-99%)
  • verda — 9.49s => 2.33s (-76%)

Online providers:

  • crusoe — 2.90s => 3.59s (+24% – network variability)
  • cudo — 11.16s => 2.14s (-81%)
  • digitalocean — 1.69s => 1.74s (+3% – network variability)
  • hotaisle — 2.13s => 1.69s (-21% – network variability)
  • vastai — 4.99s => 2.69s (-46%)
  • vultr — 12.69s => 3.48s (-73%)

Notice online providers also improved although they don't use catalog. This is because catalog.query() used to trigger catalog.load() on the first call (S3 download + CSV parsing), which blocked all providers including online ones. With the catalog pre-warmed at startup, online providers no longer wait for that download. Some online providers were fast just accidentally because query() happened with the catalog already loaded.

@r4victor r4victor merged commit af3667b into master Apr 16, 2026
28 checks passed
@r4victor r4victor deleted the issue_3479_optimize_cold_cache_offers branch April 16, 2026 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant