Skip to content

Releases: doronz88/pmd-pytcp

v0.3.7

Choose a tag to compare

@doronz88 doronz88 released this 13 Aug 05:59

Final round of the 2026-08 stability audit — every remaining stack-side finding is fixed. Verified per-commit on Python 3.9 with the full test suite (4173 tests) and smoke-tested end-to-end against a physical iOS device over the pymobiledevice3 userspace tunnel (zero stranded tasks, latency unchanged).

Highlights

  • Fragment reassembly is bounded and self-cleaning: the flow table refuses new flows at 64 concurrent reassemblies (the 64 KB-datagram equivalent of Linux's 4 MB ipfrag_high_thresh) while in-progress flows still complete, and the periodic housekeeping task now reaps timed-out flows — previously the last burst of incomplete fragments was retained until the next fragment happened to arrive.
  • SYN floods can no longer mint unbounded embryonic sessions: a second admission gate bounds concurrent SYN_RCVD children per listening port (128, Linux tcp_max_syn_backlog parity) — the listen(backlog) gate only counted established-unaccepted children.
  • stop() finishes what it starts: one worker raising during the final await no longer aborts teardown midway; every remaining subsystem is still awaited and the sysctl reset now runs in a finally, so a failed run cannot leak its overrides into the next init() cycle.
  • The IPv6 DAD claim is cancellation-safe: a claim cancelled mid-probe (stack stop during boot, a replaced DHCPv6 lease) previously leaked its registry slot and nonces, left a stale TENTATIVE state entry, and stayed joined to a solicited-node multicast group; the probe body now runs under a single-teardown finally.
  • accept() never returns a corpse: a child reset by its peer while queued (CLOSED, deregistered) is reaped from the accept queue instead of being handed to the application — Linux inet_csk_accept parity.

Deliberately left as documented behavior: the standalone daemon's fd/lifecycle hygiene (the daemon is not part of the embedded-stack surface), classical PMTU-cache aging (entries are per-genuine-peer and O(1) each), and the idle delayed-ACK re-arm (10 Hz per established session, bounded).

v0.3.6

Choose a tag to compare

@doronz88 doronz88 released this 13 Aug 04:55

Third round from the 2026-08 stability audit: TCP hot-path hygiene and long-run state bounds. Verified per-commit on Python 3.9 with the full test suite (4167 tests) and smoke-tested end-to-end against a physical iOS device over the pymobiledevice3 userspace tunnel (zero stranded tasks, latency unchanged at ~0.8 s).

Highlights

  • Out-of-order queue hygiene. Entries stranded below RCV.NXT by a re-segmented retransmission were unreachable by the exact-key consume path: each pinned its full packet buffer for the session's remaining life and kept being advertised as a SACK block below the peer's SND.UNA — which RFC 2883 peers parse as DSACK, wrongly undoing their cwnd reductions. The consume path now purges them, and the queue is bounded (64 entries, Linux tcp_prune_ofo_queue-style) so disjoint tiny out-of-order segments can no longer pin one buffer per in-window byte.
  • Tail Loss Probe actually fires under long SRTT. The RFC 8985 §7.2 "do not outlast RTO" clamp read a timer field that no longer exists, so it was dead code: whenever 2×SRTT reached past the remaining RTO, the RTO always preempted the probe and every tail loss degraded to a full RTO + slow-start. The clamp now reads the live per-session retransmit deadline.
  • Datagram rx queues are bounded (512 datagrams, POSIX full-buffer newest-drop). Worst prior case: the IPv4 ACD defense socket buffered every ARP frame on the segment between DHCP BOUND polls — up to T1, i.e. hours.
  • Slow-growth state tables fixed for the runs-for-days profile: the IPv6 DAD registry now drops entries when their address is removed (each rotated RFC 8981 temp address previously leaked one forever); the TCP Fast-Open negative cache is FIFO-bounded like the cookie cache; timed-out default routers are removed per RFC 4861 §6.3.5 instead of merely hidden (an on-link RA spoofing attacker could previously grow the list without bound).

v0.3.5

Choose a tag to compare

@doronz88 doronz88 released this 12 Aug 23:51

Second stability round from the 2026-08 audit: every remaining P1 finding is fixed. All changes verified per-commit on Python 3.9 with the full test suite (4157 tests) and smoke-tested end-to-end against a physical iOS device over the pymobiledevice3 userspace tunnel (zero stranded tasks, latency unchanged).

Highlights

  • connect() failures always surface. A soft ICMP unreachable during the handshake used to wake the blocked connect() with an error value it never mapped — the caller returned as if connected to a session still mid-handshake. Per the stack's RFC 5927 hint-not-proof stance, the soft error is now recorded without waking the caller (a spoofed ICMP can't kill a pending connect) and is reported — as OSError(EHOSTUNREACH/ENETUNREACH) — if the R2 budget gives up, exactly Linux's sk_err_soft behavior. The R2 give-up also now releases a connect() parked through a crossed-SYN simultaneous open (SYN_RCVD), which previously hung forever.
  • Listener teardown resets its backlog. Closing/aborting a listening socket now aborts every established-but-unaccepted child (RST to the peer, socket deregistered, port released — Linux inet_csk_listen_stop parity) and wakes a blocked accept() with EBADF/ECONNABORTED instead of stranding it forever.
  • close() works on a session-less TCP socket. A bound-but-never-connected socket previously could not be closed at all (assert raised) and leaked its port permanently.
  • Datapath loops survive their first bug. One exception in a subsystem iteration (e.g. NUD maintenance), a periodic-housekeeping pass, or a frame's TX assembly used to kill the loop silently — neighbor resolution frozen, temp-address sweeps stopped, or egress wedged behind a poison frame — while the stack looked alive. All three now log through the stdlib logger and keep running.
  • recvmsg(MSG_ERRQUEUE) can no longer crash after an ICMP error burst: the bounded error queue's semaphore now stays in sync with the deque instead of accumulating phantom permits that ended in IndexError.

v0.3.4

Choose a tag to compare

@doronz88 doronz88 released this 12 Aug 23:09

Stability release: a systematic leak/liveness audit of the stack (driven by live measurement through pymobiledevice3's userspace RSD tunnel), fixing every confirmed way a session, port, waiter task, or timer could be stranded.

Highlights

  • TCP: blocked recv() waiters are now woken on every session-death transition. RST in FIN_WAIT_1/FIN_WAIT_2/CLOSING/CLOSE_WAIT, the peer's FIN entering TIME_WAIT/CLOSING, the FIN_WAIT_2 orphan reap, and the 2MSL expiry all left a parked recv() sleeping forever while the socket was unregistered under it (measured live: 24/30 abandoned tunnel relays leaked their reader tasks permanently). _change_state now owns a catch-all wake, and a reset connection raises ConnectionResetError (POSIX parity) instead of reading as a clean EOF — data received ahead of the failure is still delivered first.
  • NUD: FAILED neighbours re-resolve on next TX (Linux parity). A neighbour that missed one solicit burst (~3 s of unresponsiveness) was black-holed until process restart on small caches — GC never runs below 128 entries, so a point-to-point tunnel never recovered. FAILED entries now restart resolution on demand, drop their undeliverable queued frames, and are always GC-evictable.
  • TCP: CLOSING (simultaneous close) can no longer dead-end. The state had no timer handler, no FIN retransmission, and no give-up path: one lost ACK pinned the session and its port forever, with a 1 kHz pump spin on top. It now retransmits the FIN on the RTO ladder and collapses through the R2 budget like its siblings.
  • Datagram sockets: close() and stack.stop() wake blocked receivers with EBADF. UDP/RAW/PACKET recv() waiters were woken by nothing (stop() even skipped AF_PACKET sockets entirely); the default blocking mode parked those tasks for the loop's lifetime.
  • Zero-idle-CPU restored. Gracefully closed sessions spun the 1 ms pump through their entire TIME_WAIT; persist-timer deactivation, RACK reorder ticks, and TLP declines all left expired deadlines that re-armed the coalesced service at its 1 ms floor forever. Also fixes the RFC 8985 §6.2 boundary (>=) the RACK spin was masking.

No API breaks; recv()-family calls can now raise ConnectionResetError/ConnectionAbortedError/TimeoutError/OSError(EBADF) where they previously blocked forever or returned misleading b"".

Verified per-commit on Python 3.9 with the CI suites plus the full test suite (4145 tests), and end-to-end against a physical iOS 27.0 device over the userspace tunnel: connect/close, relay-abandonment, connection-refused, and cancelled-connect churn now return to baseline with zero stranded tasks, at unchanged throughput and latency.

pmd-pytcp 0.3.3

Choose a tag to compare

@doronz88 doronz88 released this 12 Aug 14:58

Bugfix release

tcp: orphaned FIN_WAIT_2 sessions were never reaped — a peer that vanished after ACKing our FIN pinned the TCB and its local port forever (#8, thanks @vToMy)

A connection whose socket the application has fully closed — an orphan, nobody can ever read the peer's remaining data — could sit in FIN_WAIT_2 indefinitely when the peer ACKed our FIN but never sent its own (crashed peer, silent network partition): FIN_WAIT_2 had no timer handler, and RFC 9293's unbounded FIN_WAIT_2 hold has no resource backstop. The stranded session kept its socket registered, permanently consuming its local port; under connection churn against unreliable peers this drained the ephemeral pool the same way #7's bug did, just more slowly.

Orphaned FIN_WAIT_2 sessions are now reaped after a timeout, matching Linux's net.ipv4.tcp_fin_timeout semantics — exposed as a new sysctl tcp.fin_wait_2.timeout_ms (default 60000, Linux parity). The reaper arms only when the connection becomes an orphan (socket fully closed); a shutdown(SHUT_WR) half-close with the socket still open is never reaped — the application may keep reading for as long as it likes, per RFC 9293 §3.6. The peer's FIN arriving in time cancels the reaper and hands off to TIME_WAIT's own 2MSL delay as before.

Only pmd-pytcp bumps (0.3.2 → 0.3.3); pmd-net-addr / pmd-net-proto stay at 0.0.3.

pmd-pytcp 0.3.2

Choose a tag to compare

@doronz88 doronz88 released this 12 Aug 05:39

Bugfix release

tcp: connect() could leave a half-open session registered when it raised; a concurrent close() was silently swallowed (#9, thanks @vToMy)

TcpSocket.connect() registers the socket (claiming its local port) before the handshake, then awaits the session's connect event. An exception thrown into that await — task cancellation being the everyday case — propagated with the session abandoned in SYN_SENT: it kept retransmitting SYNs until the R2 budget (~127 s) expired, its socket registered and its ephemeral port held the whole time. connect() now aborts the session on any non-TcpSessionError failure before re-raising — silent TCB deletion in unsynchronized states, RST if the handshake had just completed — so a connect() that raises never leaves anything registered.

Separately, a concurrent close()/abort() tearing down a pending connect made the session raise TcpSessionError("Connection canceled"), which matched neither the refused nor the timeout mapping and fell through — connect() returned normally on a dead, unregistered session, and the caller found out on their first send(). It now raises ConnectionAbortedError.

Only pmd-pytcp bumps (0.3.1 → 0.3.2); pmd-net-addr / pmd-net-proto stay at 0.0.3.

pmd-pytcp 0.3.1

Choose a tag to compare

@doronz88 doronz88 released this 10 Aug 18:34

Bugfix release

tcp: SysCall.ABORT was silently dropped by every FSM state — aborted sessions leaked their local ports (#7, thanks @vToMy)

No FSM state handler implemented SysCall.ABORT, and five states (FIN_WAIT_1, FIN_WAIT_2, CLOSING, LAST_ACK, TIME_WAIT) installed no syscall handler at all — so dispatch_syscall() silently no-op'd an ABORT in every state. The address-removal teardown cascade (stack.address.remove, the ip addr del path) relied on that dispatch: sessions bound to a removed address were never torn down (no RST, no CLOSED transition), stayed registered in stack.sockets forever, and each permanently consumed one ephemeral port. Long-running processes that add and remove interfaces drained the pool until connect() raised EADDRINUSE.

All eleven state handlers now implement ABORT per RFC 9293 §3.9.1, delegating to TcpSession.abort() — RST at SND.NXT for synchronized states, silent TCB deletion otherwise, always reaching the terminal CLOSED transition that unregisters the socket and releases its port.

Only pmd-pytcp bumps (0.3.0 → 0.3.1); pmd-net-addr / pmd-net-proto stay at 0.0.3.

pmd-pytcp 0.3.0

Choose a tag to compare

@doronz88 doronz88 released this 22 Jul 07:31
d66d76f

Highlights

  • Software RX checksum offload — new net.rx_cksum_validate sysctl (interface-scope, default True). The RFC 1071 TCP/UDP checksum pass is the single most expensive step of per-packet parsing (~15–20% of stack CPU on a bulk transfer). Set the knob False on an interface whose link already guarantees payload integrity — e.g. packets arriving over an AEAD-authenticated tunnel through an in-memory fd — and the RX handlers skip only the checksum arithmetic. Everything else stays active: every structural integrity check, the RFC 8200 §8.1 UDP zero-checksum drop policy, and all TX checksum generation (the peer still verifies what you send). The software analogue of a NIC's RX checksum offload (ethtool -K <iface> rx off). Measured through pymobiledevice3's --userspace tunnel (the motivating consumer): download 73 → 99 MB/s (+35%), byte-identical payload.

  • 16 pre-existing macOS test failures fixed. CI ran a Linux-only subset, so these were invisible; the full suite (all packages, every test) is now 12508 passed, 0 failed on macOS/Linux alike.

    • Platform-correct errno — 10 socket tests asserted Linux errno literals ([Errno 99], [Errno 98], …) against messages the stack builds from the errno module, whose values differ on macOS (49/48/61/60/39). Now asserted via the errno constants. One raise site (pick_local_port) hardcoded a Linux errno into its message instead of OSError(errno.EADDRINUSE, …) — fixed to the canonical form (which also sets .errno).
    • Strict IPv4 parsing independent of libcIp4Address/Ip4Mask/Ip4Wildcard trusted socket.inet_pton(AF_INET, …) to be strict, but Darwin's accepts inet_aton leniencies (leading-zero octets like 1.2.3.010). Parse canonical dotted-decimal in-package instead — same grammar as glibc on every platform, leading-zeros rejected per the ipaddress/CVE-2021-29921 precedent.
    • PLPMTUD honours a sub-BASE_PLPMTU operator seedPmtuSearch floored plpmtu_seed at BASE_PLPMTU (1200/1280), starting a small operator tcp.base_mss above its declared-safe size. Floored at the family minimum link MTU instead (Linux tcp_base_mss semantics).

Defaults are unchanged: RX checksum validation stays on for every interface unless a consumer opts out per interface.

Coordinated version bump across the three packages the feature spans: pmd-pytcp 0.3.0, pmd-net-proto 0.0.3, pmd-net-addr 0.0.3.

pmd-pytcp 0.2.0

Choose a tag to compare

@doronz88 doronz88 released this 21 Jul 07:21
293fce4

Highlights

  • Dynamic send-MSS: PLPMTUD probing mode now works (tcp.mtu_probing=2). Five defects fixed — the BASE-state deadlock (RFC 4821 §7.1 implicit confirmation is now wired), the interface-MTU working-seed that disarmed grow-on-ack, false probe-ACKs from repaired ranges (loss is now detected precisely at retransmission-overlap), single-lucky-ACK commits (candidates now need 3 consecutive ACKs), and missing black-hole recovery (an RTO at a probe-raised PLPMTU revokes the raise back to the operator seed and narrows the search). The working MSS starts at the operator-declared safe seed and only rises through sizes the path actually ACKed; every failure mode degrades to the seed, never below.
  • Runtime hot-path log gate: all 546 __debug__ and log(...) sites became log.enabled and log(...) — message f-strings are no longer formatted per packet when logging is off (previously 30–55% of bulk-transfer CPU in normal, non--O runs). stack.init() arms the gate; call refresh_log_enabled() after changing the pmd_pytcp logger level at runtime.
  • New sysctls: tcp.plpmtud.probe_timer_ms (RFC 8899 PROBE_TIMER, default 30000 — previously never fired at all; now polled from the probe-emit path) and tcp.rto.min_ms (RFC 6298 §2.4 floor for the computed RTO, default 1000 — lower it Linux-style on known-low-RTT paths).

Defaults are unchanged: probing stays off, the RTO floor stays at 1 s, classical shrink-only PMTUD is untouched. Measured through pymobiledevice3's --userspace tunnel (the motivating consumer): upload 9.1 → ~25 MB/s, download 63 → 79 MB/s.

30 new unit tests cover the probing-mode engine, the TCP adapter glue, the RTO floor and the log gate.

pmd-pytcp 0.1.0

Choose a tag to compare

@doronz88 doronz88 released this 13 Jul 07:50
607a376

pmd-pytcp 0.1.0 — pure-asyncio runtime (breaking)

The stack now runs entirely on one asyncio event loop; all threads and threading primitives are gone.

Breaking: blocking socket calls (connect/accept/recv*/send*/sendmsg/probe_pmtu) are now coroutines — await them. bind/listen/close/shutdown/setsockopt stay sync. stack.start()/stack.stop() are coroutines. The eventfd-backed fileno()/select interop is removed (asyncio replaces it). Consumers must migrate to await; there is no compatibility shim.

Highlights

  • Task-based Subsystem; Timer over loop.call_at; RxRing via loop.add_reader + deliver callback; TxRing drains with add_writer backpressure (no worker/eventfd/marshaling).
  • Async socket API, TCP session+FSM, DHCP4/6, link-local, ACD, ARP/ND, IPC (asyncio.start_unix_server), and run_daemon.
  • Preserves the 0.0.5/0.0.6 robustness fixes (writev OSError fallthrough, abort-open-TCP-on-stop, ring fd-teardown survival).
  • Python 3.9–3.14, Linux/macOS/Windows — CI green across all 18 combinations.