Skip to content

v0.1.12

Choose a tag to compare

@malisetti malisetti released this 22 Nov 00:55
· 12 commits to main since this release
  • One direct transfer runs over several connections and is more than
    twice as fast.
    Between two 8-vCPU machines in one cloud zone (0.2 ms,
    15.6 Gbit/s on one TCP stream), a 10 GiB direct transfer ran at
    5.88 Gbit/s with the default --connections 4 and 6.22 with
    --connections 8 (medians of 3; v0.1.11 on the same machines, run in
    turn with it: 2.59), 100 GiB at 6.82 with 8; relayed with 4, 4.79
    (v0.1.11: 3.26). BLAKE3 equal in every run (docs/scale.md
    test 7). The receiver's CPU is now the limit.
  • Protocol version 5, a receive striped over sessions: the
    receiver opens up to --connections sessions at once (default 4), each
    on a connection of its own, so each one's packets have a core at both
    ends. Each session's want names a set of the chunks still to fetch
    (interleaved 64 MiB runs); the sender sends that set in order, the
    receiver verifies and writes each chunk in place. A session that ends
    early (its connection cut, a chunk that does not verify) puts what it
    did not write back for another session; the slowest session is closed
    and its rest split once the others are done; one commit after the last
    session and the final check. Every session but the last is answered
    part (a --once sender serves on). A transfer of fewer 64 MiB runs
    than --connections takes one session per run (spec/Stripe.tla,
    checked before the code; its traces by spec/TraceStripe.tla).
  • The receiver reads nothing back before the commit of a file it
    verified chunk by chunk and wrote whole in this receive (the full re-read
    and re-hash cost about 1 CPU s per GiB and 1.5 s at the end of a 10 GiB
    receive); a file part of which a cut transfer left is verified on disk as
    before, and another program's write to a staged file once it is complete
    still refuses the commit (its mtime). spec/Stripe.tla Trusted.
  • A striped receive's writers take turns on one file instead of
    spinning on the kernel's inode lock, into blocks preallocated on Linux:
    4 writers on ext4 1.47 -> 0.58 CPU s per GiB (BenchmarkStripedWrite).
  • The direct path asks for fewer acknowledgements (QUIC's ACK
    Frequency extension, quiclink v0.8.0): the receiver acknowledges every
    10 packets and only a packet more than 10 places late at once. On a
    simulated path that reorders 30 % of its packets, 0.39 -> 0.10 ACK
    packets per data packet at the same rate; the CPU this saves on a real
    reordering path is not measured yet.
  • Hard cut: both ends need v0.1.12. v0.1.11 and earlier and this
    release refuse each other, neither retries, nothing is written:
    • a v0.1.12 receiver of an older sender exits 4 (refused):
      refused: the sender speaks protocol version 4, this receiver 5 to 5: update the older end;
    • an older receiver of a v0.1.12 sender exits 6, a protocol error (a
      frame it does not expect: the sender's answer naming both versions);
    • with a sender that has no account (direct only), the direct path does
      not form between the two (the QUIC peer without ACK Frequency is
      refused), so the receiver exits 12 (direct_required) once its
      direct attempts fail (2 s to about a minute on one machine); the
      other two refuse at the first frame.
  • A sender whose move to another relay replica failed moves when the
    relay asks again
    ; before, it ignored the second ask and stayed where
    the relay was rebalancing away from (relay client v0.24.5).
  • --connections <n> (receive, 1 to 16, default 4). 8 was faster than 4
    on the machines above; the default stays 4 until 8 is measured on
    slower receivers and long paths.
  • Built on link v0.13.3, pathswitch v0.1.4, quiclink v0.8.0 (QUIC fork
    v0.63.1-nfltr.7), relay client v0.24.5, relay proto v0.22.1 and
    relay-core v0.19.5.