Skip to content

v0.20.0

Choose a tag to compare

@michaelaye michaelaye released this 18 Sep 20:51
· 53 commits to main since this release

Remote size queries were slow and could silently lose sizes.

Released first as 0.19.1, then re-released as 0.20.0: the change adds a runtime dependency, which is not a patch-level change. Same code.

Changed

  • Remote size queries reuse connections. query_remote_sizes (behind get and update) opened a fresh TCP+TLS connection for every kernel, so a 102-kernel metakernel paid 102 handshakes. It now keeps one requests session alive per worker thread. Measured on juice_crema_5_2.tm (102 kernels) against the ESA archive: 135 s before, 17 s after.
  • requests>=2.31 is now a runtime dependency, replacing urllib for the size queries.

Fixed

  • Missing sizes no longer cause needless re-downloads. A failed HEAD was swallowed and recorded as "size unknown", which showed the kernel as unknown in the table, left it out of the download total, and made get re-download a file that was already complete on disk. 11 of 102 kernels hit this on the metakernel above; all 102 sizes now come back.
  • Size queries can no longer hang indefinitely. They had no timeout, so one unresponsive connection blocked its worker for as long as the server held the socket open (70 s in one measured case). Connect and read timeouts are now 15 s and 10 s, chosen from measured ESA handshake latency.

Full changelog: https://github.com/michaelaye/spice-kernel-db/blob/main/CHANGELOG.md