Skip to content

Releases: ianhi/snailmail

v0.4.1

Choose a tag to compare

@ianhi ianhi released this 24 Jun 17:53
v0.4.1
9155b95

Changed

  • Breaking: removed ObjectStore.icechunk_storage(). Use the standalone
    snailmail.convenience.icechunk_storage(store, prefix=...) instead. This keeps
    ObjectStore a general, domain-agnostic S3 server — Icechunk-specific wiring is a free
    convenience under snailmail.convenience, not a method on the store. (No deprecation
    shim; pre-1.0.)
  • The per-request log line (and the CLI --log output) now reads
    +Nms latency Nms total N in flight instead of the jargony +Nms rtt … inflight=N.

v0.4.0

Choose a tag to compare

@ianhi ianhi released this 24 Jun 17:12
v0.4.0
d56bd3c

Added

  • Per-request observability on both servers, alongside the existing stats():
    • report() — a high-level, JSON-serializable summary: totals plus by_label /
      by_status breakdowns (and, on ObjectStore, the metadata_requests /
      data_requests / other_requests split). Built from exact counters, so it stays
      complete even after the drill-down buffer rolls.
    • requests — the recent RequestRecord objects for drilling into individual
      requests: status, wire nbytes, byte range, injected latency_ms, total
      dur_ms, in_flight, label, and — on S3 — op and whether the write was
      conditional.
    • classify= to group keys for the by_label breakdown, and max_records= to cap the
      drill-down ring buffer (None = unbounded, 0 = counts only).
  • Per-request logging via the stdlib loggers snailmail.http / snailmail.s3 (off until
    you add a handler / raise the level). The CLI gains --log to stream one line per
    request, and its stop summary now shows the status breakdown.
  • ClientLink: a shared client uplink/downlink metered across multiple ObjectStores on
    top of each store's per-source bandwidth_mbs — for modelling an Icechunk store and the
    remote bucket it virtualizes contending for one connection. Asymmetric (down_mbs /
    up_mbs); ObjectStore-only for now.

Fixed

  • HTTPRangeServer recorded 206 for well-formed but unsatisfiable range requests (a
    range past EOF, or any range against an empty file) where aiohttp actually sends 416;
    the recorded status now matches the wire.

v0.3.0

Choose a tag to compare

@ianhi ianhi released this 23 Jun 00:59
v0.3.0
be47357

Added

  • ObjectStore — a local, in-process S3-compatible object store (backed by moto) with the same injectable latency/bandwidth model as the range server, for benchmarking the metadata round-trips of a consumer like Icechunk (config/refs/snapshots/manifests) that are free and invisible on a local filesystem. Optional [s3] extra. Latency is optional, so it doubles as a plain local S3 store. icechunk_storage() returns a ready-wired icechunk.Storage; stats() splits cost into metadata_requests vs data_requests.
  • StoreBehavior for emulating store quirks. conditional_writes="enforce"|"reject"|"ignore""reject" returns NotImplemented like JASMIN, making icechunk#2228 reproducible locally with no cloud credentials (repros/icechunk_2228.py).
  • LatencyMiddleware (generic WSGI middleware) and SharedPipe (sync twin of AsyncSharedPipe).

Changed

  • Renamed LatencyRangeServerHTTPRangeServer (no back-compat alias; pre-1.0). Both servers now share one shape: optional latency=/bandwidth_mbs= wire shaping plus per-server behavior=.

v0.2.0

Choose a tag to compare

@ianhi ianhi released this 22 Jun 16:14
f8d350f

Added

  • LatencyRangeServer.from_file(path, ...) — serve a single file directly, reachable at its basename, with no containing directory, no temp dir, no symlink, and no copy. It's streamed from disk via aiohttp's FileResponse (the same machinery directory mode uses), and because only the one pinned path is ever served there's no path-traversal surface (every other key 404s). The result is observationally a one-key directory server: describe(), files(), url(), and stats() behave identically to directory mode.
with LatencyRangeServer.from_file("CMU-1.tiff", latency=LogNormal(mode_ms=40)) as server:
    open_and_read(server.url("CMU-1.tiff"))

Full changelog: https://github.com/ianhi/snailmail/blob/v0.2.0/CHANGELOG.md

v0.1.1

Choose a tag to compare

@ianhi ianhi released this 19 Jun 01:55
6ca67e7

Fixed

  • files() / describe()["n_files"] over-counted symlinks whose target escapes the served root — they were listed but 404 on GET, since aiohttp (and _target_size) serve a key only when its resolved real path is a file inside the root. files() now applies that same resolve-then-in-root rule, so the index matches what is actually served. The root-containment rule is now documented in the README.

Full changelog: https://github.com/ianhi/snailmail/blob/v0.1.1/CHANGELOG.md

snailmail 0.1.0

Choose a tag to compare

@ianhi ianhi released this 18 Jun 18:15
c0d1bdf

Initial public release.

Added

  • LatencyRangeServer: a loopback HTTP server that serves a directory tree over HTTP Range with injectable latency and bandwidth limits, for benchmarking range / object-store / virtual-chunk reads. One object per file (the shape of an Icechunk virtual dataset), range- and traversal-safe; base is the root, url(key) builds a key URL, and files() lists the served keys.
  • Pluggable per-request latency distributions: LogNormal (the recommended default), Normal, Exponential, and Fixed, each with explicit, distribution-specific parameters. Draws are pre-generated and served round-robin (O(1), reproducible per seed).
  • Shared FIFO bandwidth pipe (AsyncSharedPipe) so response bytes serialize through a capped egress while round-trips stay parallel.
  • Request accounting via stats(): GET/request counts, 404 misses (n_misses), total bytes, peak concurrency (max_in_flight), and per-method / per-path breakdowns; persists until reset_counts().
  • snailmail CLI with a --dist selector and explicit per-distribution flags, a --json machine-readable address line (flushed before serving), and --version.