Releases: ianhi/snailmail
Releases · ianhi/snailmail
Release list
v0.4.1
Changed
- Breaking: removed
ObjectStore.icechunk_storage(). Use the standalone
snailmail.convenience.icechunk_storage(store, prefix=...)instead. This keeps
ObjectStorea general, domain-agnostic S3 server — Icechunk-specific wiring is a free
convenience undersnailmail.convenience, not a method on the store. (No deprecation
shim; pre-1.0.) - The per-request log line (and the CLI
--logoutput) now reads
+Nms latency Nms total N in flightinstead of the jargony+Nms rtt … inflight=N.
v0.4.0
Added
- Per-request observability on both servers, alongside the existing
stats():report()— a high-level, JSON-serializable summary: totals plusby_label/
by_statusbreakdowns (and, onObjectStore, themetadata_requests/
data_requests/other_requestssplit). Built from exact counters, so it stays
complete even after the drill-down buffer rolls.requests— the recentRequestRecordobjects for drilling into individual
requests:status, wirenbytes, byterange, injectedlatency_ms, total
dur_ms,in_flight,label, and — on S3 —opand whether the write was
conditional.classify=to group keys for theby_labelbreakdown, andmax_records=to cap the
drill-down ring buffer (None= unbounded,0= counts only).
- Per-request logging via the stdlib loggers
snailmail.http/snailmail.s3(off until
you add a handler / raise the level). The CLI gains--logto stream one line per
request, and its stop summary now shows the status breakdown. ClientLink: a shared client uplink/downlink metered across multipleObjectStores on
top of each store's per-sourcebandwidth_mbs— for modelling an Icechunk store and the
remote bucket it virtualizes contending for one connection. Asymmetric (down_mbs/
up_mbs);ObjectStore-only for now.
Fixed
HTTPRangeServerrecorded206for well-formed but unsatisfiable range requests (a
range past EOF, or any range against an empty file) where aiohttp actually sends416;
the recorded status now matches the wire.
v0.3.0
Added
ObjectStore— a local, in-process S3-compatible object store (backed by moto) with the same injectable latency/bandwidth model as the range server, for benchmarking the metadata round-trips of a consumer like Icechunk (config/refs/snapshots/manifests) that are free and invisible on a local filesystem. Optional[s3]extra. Latency is optional, so it doubles as a plain local S3 store.icechunk_storage()returns a ready-wiredicechunk.Storage;stats()splits cost intometadata_requestsvsdata_requests.StoreBehaviorfor emulating store quirks.conditional_writes="enforce"|"reject"|"ignore"—"reject"returnsNotImplementedlike JASMIN, making icechunk#2228 reproducible locally with no cloud credentials (repros/icechunk_2228.py).LatencyMiddleware(generic WSGI middleware) andSharedPipe(sync twin ofAsyncSharedPipe).
Changed
- Renamed
LatencyRangeServer→HTTPRangeServer(no back-compat alias; pre-1.0). Both servers now share one shape: optionallatency=/bandwidth_mbs=wire shaping plus per-serverbehavior=.
v0.2.0
Added
LatencyRangeServer.from_file(path, ...)— serve a single file directly, reachable at its basename, with no containing directory, no temp dir, no symlink, and no copy. It's streamed from disk via aiohttp'sFileResponse(the same machinery directory mode uses), and because only the one pinned path is ever served there's no path-traversal surface (every other key 404s). The result is observationally a one-key directory server:describe(),files(),url(), andstats()behave identically to directory mode.
with LatencyRangeServer.from_file("CMU-1.tiff", latency=LogNormal(mode_ms=40)) as server:
open_and_read(server.url("CMU-1.tiff"))Full changelog: https://github.com/ianhi/snailmail/blob/v0.2.0/CHANGELOG.md
v0.1.1
Fixed
files()/describe()["n_files"]over-counted symlinks whose target escapes the served root — they were listed but 404 on GET, since aiohttp (and_target_size) serve a key only when its resolved real path is a file inside the root.files()now applies that same resolve-then-in-root rule, so the index matches what is actually served. The root-containment rule is now documented in the README.
Full changelog: https://github.com/ianhi/snailmail/blob/v0.1.1/CHANGELOG.md
snailmail 0.1.0
Initial public release.
Added
LatencyRangeServer: a loopback HTTP server that serves a directory tree over HTTP Range with injectable latency and bandwidth limits, for benchmarking range / object-store / virtual-chunk reads. One object per file (the shape of an Icechunk virtual dataset), range- and traversal-safe;baseis the root,url(key)builds a key URL, andfiles()lists the served keys.- Pluggable per-request latency distributions:
LogNormal(the recommended default),Normal,Exponential, andFixed, each with explicit, distribution-specific parameters. Draws are pre-generated and served round-robin (O(1), reproducible per seed). - Shared FIFO bandwidth pipe (
AsyncSharedPipe) so response bytes serialize through a capped egress while round-trips stay parallel. - Request accounting via
stats(): GET/request counts, 404 misses (n_misses), total bytes, peak concurrency (max_in_flight), and per-method / per-path breakdowns; persists untilreset_counts(). snailmailCLI with a--distselector and explicit per-distribution flags, a--jsonmachine-readable address line (flushed before serving), and--version.