Speak TLS inside the simulation

CHANGELOG.md

@@ -2,6 +2,28 @@
 
 ## 0.2.0 (unreleased)
 
+- TLS runs inside the simulation. `create_connection(ssl=...)`,
+  `create_server(ssl=...)` and `start_tls` on an already-established
+  connection all work, and so do `asyncio.open_connection` and `start_server`
+  on top of them. The handshake is the real thing — the standard library's
+  `SSLProtocol` driving OpenSSL over a pair of memory BIOs, with real
+  certificate verification, so a hostname the certificate does not cover
+  raises `ssl.SSLCertVerificationError` — and no file descriptor, no real
+  socket and no wall-clock second is involved anywhere. Each flight OpenSSL
+  produces leaves as one ordinary simulated packet and pays the link's seeded
+  latency, so a client connect costs two round trips; `ssl_handshake_timeout`
+  and `ssl_shutdown_timeout` are ordinary loop timers, so a handshake a
+  partition stalls costs sixty virtual seconds and milliseconds of real ones.
+  aiohttp's `https://`, httpx's `https://` and websockets' `wss://` now run
+  under simulation, with the evidence in
+  [docs/compatibility.md](docs/compatibility.md). The caveat, stated plainly:
+  for a workload that uses TLS the hash promise gains two clauses — same
+  OpenSSL build, same TLS configuration — because the number of packets a
+  handshake makes is a property of the engine. Certificates are not among
+  them, measured: an EC leaf and an RSA leaf record the same hash, since the
+  trace records how many packets crossed and in what order, never their
+  bytes. A run that never asks for TLS is unaffected, which pinned reference
+  digests keep true.
 - `drain()` can finally block. `loop.net.set_flow_control()` gives stream
   transports a write buffer that holds every byte written but not yet
   received by the peer's protocol — still in flight, held by a partition,

README.md

@@ -283,8 +283,8 @@ and the campaign results:
 
 Code that goes through the event-loop API is supported; code that
 bypasses it is fenced: threads, raw socket reads and writes,
-subprocesses, signals, and loop-level TLS upgrades raise
-`SimulationFenceError` rather than silently breaking determinism.
+subprocesses and signals raise `SimulationFenceError` rather than
+silently breaking determinism.
 Executor submissions stay inside the line: `run_in_executor` runs the
 function inline at a seeded scheduling step — no pool, no thread — so
 `asyncio.to_thread` works, and `call_soon_threadsafe` is `call_soon`
@@ -296,6 +296,11 @@ simulated, so a client that connects a socket and hands it to
 Name resolution stays inside the simulation: `getaddrinfo` resolves sim
 host names to stable synthetic addresses and raises `socket.gaierror` for
 anything else — no real DNS, ever.
+TLS runs inside the simulation: a real handshake through the standard
+library's `SSLProtocol` over a pair of memory BIOs, with real certificate
+verification, no descriptor and no wall clock — each flight is one
+simulated packet paying the link's latency, and a handshake deadline fires
+in virtual time.
 Write-side flow control is simulated on request: `net.set_flow_control()`
 makes `drain()` really wait while the peer has not read, so backpressure
 deadlocks and missing pause/resume handling become findable. It is off by

docs/compatibility.md

@@ -5,7 +5,8 @@ simulates. This page answers the next question — what happens when a real
 library runs on top of them — with evidence rather than intent: every row
 below is the output of a script anyone can re-run.
 
-Recorded **2026-08-01**, against simloop 0.2.0 (unreleased) on Python 3.12.
+Recorded **2026-08-04**, against simloop 0.2.0 (unreleased) on Python 3.12
+with OpenSSL 3.5.7.
 
 ## What a probe is
 
@@ -23,8 +24,9 @@ network — and reports a single verdict:
 A verdict is a statement about that one run and nothing more. `works` means
 those calls, on that version, produced that result; it is not a support
 claim, and the same library may well fence one call later. The probes drive
-happy paths only: no TLS, no retries, no reconnection, no concurrency beyond
-what the probe itself starts.
+happy paths only: no retries, no reconnection, no concurrency beyond what the
+probe itself starts. TLS is a happy path they now drive, with certificates
+minted in memory for the sim hostnames the probes use.
 
 ## Regenerating the table
 
@@ -46,8 +48,11 @@ carries a date instead.
 | anyio | 4.14.2 | works: task group, memory object stream (one, two, three), anyio.sleep and move_on_after; virtual clock reached 1.75s | Asyncio backend only; nothing here touches a socket. |
 | redis (RESP wire protocol) | n/a | works: PING, SET and GET round trips over one connection: ['+PONG', '+OK', '0'] | Hand-rolled RESP over sim streams; no client library, no real server. |
 | websockets | 17.0.1 | works: handshake, one echoed frame ('HELLO') and close over ws:// | asyncio server and client on two sim hosts, ws:// only. |
+| websockets (wss) | 17.0.1 | works: handshake, one echoed frame ('HELLO') and close over wss:// | asyncio server and client on two sim hosts, over wss://. |
 | aiohttp (client) | 3.14.3 | works: ClientSession GET returned 'hello from the simulation' | ClientSession GET at a sim host answered by a raw stream server. |
+| aiohttp (client, https) | 3.14.3 | works: ClientSession GET over https returned 'hello from the simulation' | ClientSession GET over https at a sim host with a minted certificate. |
 | httpx | 0.28.1 | works: AsyncClient GET returned 'hello from the simulation' | AsyncClient GET at a sim host answered by a raw stream server. |
+| httpx (https) | 0.28.1 | works: AsyncClient GET over https returned 'hello from the simulation' | AsyncClient GET over https; the TLS engine is anyio's, not the loop's. |
 
 Rows are grouped: the libraries that need nothing but the loop and its
 streams first, then the client stacks that expect a socket object
@@ -104,8 +109,8 @@ on it. A transport with no operating-system socket now answers with a
 stand-in that reports the peer's synthetic address and port, so the
 introspection succeeds and the response body comes back.
 
-Both client probes make one request against a responder that sends
-`Connection: close`, so neither row says anything about connection reuse.
+Every client probe makes one request against a responder that sends
+`Connection: close`, so no row says anything about connection reuse.
 The piece a pool depends on is the descriptor `fileno()` returns: httpcore
 polls it to decide whether a pooled connection has died, and the
 simulation backs it with a parked descriptor the transport owns, which
@@ -114,32 +119,37 @@ peer's EOF arrives; a reset or a teardown closes it and `fileno()` returns
 `-1`, which the same poll reads as dead just as well. That contract is
 pinned by the test suite, not by these rows.
 
-Both client rows are `http://` only, and the two stacks stop differently
-on `https://`. aiohttp asks for TLS through `create_connection(ssl=...)`,
-which fences:
-
-```
-simloop does not simulate 'create_connection(ssl=...)'; see docs/supported-api.md for the supported asyncio subset
-```
-
-httpx reaches no fence. httpcore wraps the byte stream with anyio's
-`TLSStream`, which drives an `ssl` memory BIO inside the process and
-sends the handshake as ordinary bytes over the simulated connection, so
-`loop.start_tls` is never called and nothing stops the attempt. Where it
-ends is up to whatever is listening: aimed at the plaintext responder
-these probes use, the handshake goes unanswered and the request dies of
-httpx's own `ConnectTimeout`. That timeout is a one-off measurement
-rather than a row — no probe on this page requests `https://`.
+**aiohttp over `https://`** completes the same two-call connect its
+`http://` path uses, with `ssl` and `server_hostname` riding alongside
+`sock` in the `create_connection` call. What the connector does after the
+connect is answered by the two layers together: `sslcontext` and
+`ssl_object` come from the TLS layer, `peername` and the `setsockopt` on
+the stand-in socket from the simulated transport underneath it. The
+certificate is minted for the sim hostname `web` and the client context
+trusts that authority and nothing else, so the row says OpenSSL really
+verified rather than that verification was turned off.
+
+**httpx over `https://`** reaches no loop TLS API at all, which is why it
+is worth its own row. httpcore wraps the byte stream with anyio's
+`TLSStream`, which drives an `ssl` memory BIO inside the process and sends
+the handshake as ordinary bytes over the simulated connection. Nothing in
+simloop is involved in that handshake; what changed is that the simulation
+now has a peer on the other end that speaks TLS back, so the request
+completes instead of dying of httpx's own `ConnectTimeout`.
+
+**websockets over `wss://`** completes a handshake, echoes a frame and
+closes with both ends inside the simulation. It is the only row that
+drives `create_server(ssl=...)` and `create_connection(ssl=...)` in one
+run.
 
 ## Not tested
 
 - **asyncpg**: reaching its first fence needs a live PostgreSQL server to
   connect to, which no probe can provide; it is untested rather than
   fenced-or-not.
-- **TLS anywhere**: no probe on this page requests `https://` or `wss://`.
-  simloop fences `start_tls` and `create_connection(ssl=...)`, but a stack
-  that runs its handshake in memory reaches neither — it reaches a simulated
-  network with nothing on it that speaks TLS unless the test puts it there.
+- **The rest of TLS**: the three TLS rows drive a server certificate, one
+  cipher suite and TLS 1.3. Client certificates, a peer restricted to TLS
+  1.2, ALPN and h2 negotiation, and session resumption are not probed.
 - Anything that reaches outside the loop by design — threads, subprocesses,
   signals, real DNS. Those are fences, listed in
   [docs/supported-api.md](supported-api.md), not compatibility questions.

docs/design.md

@@ -136,9 +136,10 @@ production.
 
 Anything that would reach outside the simulation — real threads,
 signals, subprocesses, raw sockets, `add_reader`/`add_writer`,
-TLS, pipes, `sendfile` — raises `SimulationFenceError` naming the exact call,
-and optional stdlib kwargs that would smuggle those in (`ssl=`, `sock=`, …)
-are rejected the same way.
+pipes, `sendfile` — raises `SimulationFenceError` naming the exact call,
+and an optional stdlib argument that would smuggle one in is rejected the
+same way: whatever is left after the simulated arguments have been taken and
+is actually asked for fences rather than being quietly ignored.
 
 The tempting alternative was best-effort passthrough: hand `run_in_executor`
 a real thread pool, keep most libraries importable, appear more compatible.
@@ -256,10 +257,37 @@ Decisions inside that model, each doing real work:
   `datagram_received` belongs to the receiving machine, and `crash` knows
   exactly which tasks to kill.
 
-What was cut, deliberately: retransmission and congestion modeling, IP
-addresses, TLS. Each would deepen the simulation without widening the class
-of bugs it can catch; the supported-subset contract beats chasing 100% of
-the asyncio surface.
+What was cut, deliberately: retransmission and congestion modeling. It would
+deepen the simulation without widening the class of bugs it can catch; the
+supported-subset contract beats chasing 100% of the asyncio surface.
+
+## TLS: real bytes, and what that costs the hash
+
+The handshake is the standard library's `SSLProtocol` over a pair of memory
+BIOs — the same machinery uvloop reuses — because a hand-rolled pump would
+have to re-derive waiter semantics, handshake and shutdown deadlines,
+close-notify handling and flow-control passthrough, and would get one of them
+wrong. The transport underneath it gained exactly three things: a buffered
+delivery path, a `_force_close` that carries a failure into
+`connection_lost`, and the flag `start_tls` insists on before it will touch a
+transport at all.
+
+Its wire bytes are genuinely random and differ every run. That is affordable
+because the trace does not hash payloads — only the number and order of
+packets and callbacks — and because `SSLProtocol` drains its outgoing BIO
+once per flight, so one flight is one write is one simulated packet. A
+certificate's size and key type therefore leave the hash alone: an EC leaf
+and an RSA leaf record the same one. Anything that changes the flight
+structure does not: a different OpenSSL build, session tickets turned off, a
+peer that only speaks TLS 1.2. So the promise for a TLS workload is one
+clause longer than the promise for everything else, and only for a TLS
+workload — a run that never asks for TLS makes no new draw, arms no new timer
+and records no new event, which a pinned reference hash keeps true.
+
+The handshake pays the wire the way everything else does, which is free
+realism: a client connect costs two round trips of the configured latency,
+and `ssl_handshake_timeout` is an ordinary `call_later`, so a partition that
+outlasts it costs sixty virtual seconds and milliseconds of wall clock.
 
 ## Explorer and pytest plugin: a thin shell by design
 

docs/supported-api.md

@@ -38,6 +38,10 @@ a host belong to an implicit `driver` host.
 | API | Behavior under simulation |
 |---|---|
 | `loop.create_connection` / `create_server`, `asyncio.open_connection` / `start_server` | Real transports and protocols over reliable, ordered in-memory streams; connecting costs one round trip of virtual latency; connecting to a closed port raises `ConnectionRefusedError` |
+| `loop.create_connection(ssl=...)` / `create_server(ssl=...)`, `asyncio.open_connection(ssl=...)` / `start_server(ssl=...)` | A real TLS handshake, driven by the standard library's `SSLProtocol` over a pair of memory BIOs. No descriptor and no real socket anywhere: each flight OpenSSL produces leaves as one ordinary simulated packet and pays the link's seeded latency, so a client connect costs two round trips — one for the connection, one for the handshake — which is what a real TCP + TLS 1.3 connect costs. Verification is the real thing: a hostname the certificate does not cover raises `ssl.SSLCertVerificationError`. `ssl=True` means the standard library's default client context, which trusts the system store and therefore rejects a simulation's own certificate; on a server it is a `ValueError` when the listener is created, since there is no default certificate to present. A handshake a client fails is reset and does not fail the run |
+| `loop.start_tls` | Upgrades an established simulated connection in place and returns the new transport the protocol should write to — including the server-side case where the stream reader has already buffered the bytes the handshake needs |
+| `ssl_handshake_timeout` / `ssl_shutdown_timeout` | Ordinary loop timers, so they fire in virtual time at the standard library's defaults of 60 s and 30 s. A handshake a partition stalls costs sixty virtual seconds and milliseconds of wall clock, and raises `ConnectionAbortedError` |
+| `transport.get_extra_info` on a TLS transport | `ssl_object`, `peercert`, `cipher`, `compression` and `sslcontext` come from the TLS layer; `socket`, `peername` and `sockname` fall through to the simulated transport underneath, so the stand-in socket row below still applies |
 | `loop.create_datagram_endpoint` | Unreliable messaging: per-link drop, duplication, and latency apply per datagram |
 | `loop.net.set_defaults` / `set_link` | Per-direction latency ranges, drop and duplication probabilities, drawn from a seed-derived stream |
 | `loop.net.partition` / `heal` | Silent blackhole: datagrams are lost, stream traffic is held and resumes intact after healing; nothing errors — only your own timeouts fire |
@@ -73,25 +77,28 @@ so skewing a worker changes nothing the cluster decides. Clock faults
 reach only code that compares timestamps taken on different machines.
 
 Limitations, stated honestly: there is no retransmission or congestion model
-— streams are reliable by construction; and addressing is IPv4-only and
-entirely synthetic — there are no routes, no netmasks, and no service-name
-database.
+— streams are reliable by construction; addressing is IPv4-only and entirely
+synthetic — there are no routes, no netmasks, and no service-name database;
+TLS has no half-close, so `write_eof()` on a TLS transport raises
+`NotImplementedError` and `can_write_eof()` is `False`, exactly as on a real
+one; and DTLS is not simulated — datagram endpoints take no TLS arguments.
 
 ## Fenced
 
 Anything that reaches outside the simulation raises `SimulationFenceError`:
 real threads (`call_soon_threadsafe` from any thread but the loop's own),
 signal handlers, subprocesses, file-descriptor callbacks (`add_reader` /
-`add_writer`), loop-level TLS upgrades (`start_tls`,
-`create_connection(ssl=...)`), `sendfile`, pipes, and an eager task start
+`add_writer`), `sendfile`, pipes, and an eager task start
 (`create_task(eager_start=True)`), which would run a task's first step at
 creation time, before the seeded draw could order it against anything.
 Executor *submissions* are not in that list — `run_in_executor` runs the
 function inline, as the table above says — but the pool machinery around
 them still is: `set_default_executor` and `shutdown_default_executor`
 fence, because an executor that would never be used is refused rather than
-silently accepted. TLS a library performs in memory reaches no loop API
-and so reaches no fence; what that means in practice is in
+silently accepted. TLS is not in that list either, on either of its two
+routes: through the loop, as the table above describes, and inside a
+library's own memory BIO, which reaches no loop API at all and now finds a
+simulated peer that speaks TLS. What that means in practice is in
 [docs/compatibility.md](compatibility.md).
 
 The socket calls are fenced with one exception. `sock_connect` on an
@@ -147,6 +154,22 @@ differ from the ones 0.1.0 recorded — see the
 [changelog](../CHANGELOG.md). What a hash promises is unchanged: same seed,
 same code, same interpreter, same hash.
 
+TLS costs that promise one clause, and only for a workload that uses it. TLS
+records add no event kind — they are ordinary packets — but how many packets
+a handshake makes is a property of the TLS engine, so for such a workload the
+promise reads *same seed, same code, same interpreter, same OpenSSL build,
+same TLS configuration*. Certificates are not on that list, which is the
+reassuring half and is measured: an EC leaf and an RSA leaf record the
+same hash, because the trace hashes the number and order of packets and never
+their bytes, and the TLS engine emits exactly one write per flight. What does
+move it is anything that changes the flight structure — `SSLContext.num_tickets`
+(setting it to 0 drops one server packet from a connection that stays open
+long enough to be sent its session tickets; raising it above the default of 2
+changes nothing, because they all leave in one write), a client certificate
+request, a peer that only speaks TLS 1.2. A run that
+never asks for TLS makes no new draw, arms no new timer and records no new
+event, which a pinned reference hash in the test suite keeps true.
+
 `simloop.timeline_html(events, limit=5000)` renders a trace as a
 self-contained HTML page — one lane per machine plus one for the simulation,
 a dot per scheduling decision, an arrow for every `send` its `deliver`

probes/_http.py

@@ -26,6 +26,9 @@ async def respond(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) ->
     await reader.readuntil(b"\r\n\r\n")
     writer.write(RESPONSE)
     await writer.drain()
-    writer.write_eof()
+    # TLS has no half-close, so a TLS transport refuses one; the close below
+    # is what ends the response there.
+    if writer.can_write_eof():
+        writer.write_eof()
     writer.close()
     await writer.wait_closed()