First real-hardware RR-Net end-to-end for the ip65 product: 15 independent hardware checks on real CS8900a silicon

[JC-000]

Why

c64-https-ip65-onchip.prg is one of the three products make package ships, and no ip65 build had ever run on real CS8900a silicon. Every ip65 result in this repo — including the 36-minute stock-clock end-to-end in CLAUDE.md — came from tests/rig_vice_https_macos.py, which drives the same PRG inside a pcap-patched VICE against a feth pair. VICE's CS8900a is a model of the chip. This PR runs the chip.

[ RR-Net in the U64E's cartridge port ] <--cable, no switch--> [ Mac en4, 10.0.66.1 ]

The PRG is loaded over the U64's REST interface the usual way (DeviceLock, write + SYS), but nothing about the network path goes through the Ultimate: the C64 reaches the world through the cartridge, and the Mac is the only other station on the cable — DHCP server, DNS server and HTTPS origin at once.

The result

24/24 checks passed, 0 failed, 0 inconclusive, exit 0 — and the honest reading of that number is below, because it is not 24 facts about the cartridge. U64E (fw 3.15, commit 4011c97c, fpga 125), stock 1 MHz, no turbo, no REU. BACKEND=ip65 USE_NISTCURVES_ONCHIP=1 HTTPS_PORT=4433, PRG sha256 9d33683a…, 47,105 B.

What the 24 actually decompose into

• 8 involve no cartridge and no 6510: four off-device checker selftests against a synthetic pcap, a PRG file read, a build/labels.txt read, a U64 config write, and a Mac→Mac loopback probe of our own listener.
• 16 are genuine hardware results, of which 15 are independent: check_mac_on_wire cannot fail unless check_c64_originated also fails (the rig passes c64_mac as a constant from the rig script, so all six of its value-rejections are statically false and its only live branch duplicates the other's frame count over the same corpus). It is kept because those rejections go live the moment a caller passes a MAC read back from the device, but it should be counted once.

So the defensible headline is 15 independent hardware-attributable checks, not 24. Each line of the rig's own output is labelled honestly; it is the total that invites over-reading.

'G' to CONNECTION CLOSED: 1,979 s (33.0 min), against 2,159.7 s (36.0 min) for the same profile in VICE at honest 1 MHz — faster on real silicon by at least the ~8% the raw figures show. "At least", not "about": docs/engineering-notes.md records all four ip65 VICE rows at the libs/nistcurves v0.6.0 pin, and this ran at v0.11.2 — two binaries about five months apart, so the boundary alignment does not by itself isolate the cause to silicon-versus-emulator. The direction is in our favour, which is why the number survives as a bound: v0.7.0's public-key validation costs ~1.6% on a verify that is 1,417 s of the 2,159.7 s, so a current-pin emulator run would be slower and the true gap wider.

tls_state was sampled over DMA throughout, so the phases are the client's own state machine and not a screen scrape:

+s after 'G'	screen	tls_state
5	TCP CONNECTED	0
305	CH	2
640	SH	2
655	GOT	4
661	PROC	5
1959	FIN	6
1974	REQUEST SENT	7 (CONNECTED)
1979	CONNECTION CLOSED	0 (tls_close)

The ~5 min to CH is the X25519 keypair; the 1,298 s in PROC is the ECDSA P-256 verify. Both are in line with the VICE model.

What is actually proven

Every wire assertion discriminates by Ethernet source address, because there are two stations on that cable and the Mac is one of them:

• DHCP — the C64 holds the pinned 10.0.66.200, read from its own net_local_ip over DMA, not from dnsmasq's lease file (which would only prove our server answered). Rejected by value: 0.0.0.0, ip65's build-time 192.168.1.64, link-local, the host's own address, a pool address. Taken on the boot auto-attempt, zero retries.
• The cartridge is on the wire — 28 frames from 00:0e:3a:64:64:64, 210 from the Mac, 0 from any third MAC.
• DNS over the cartridge — the C64's own query for www.foo.invalid.
• ClientHello — 141 bytes, decoded as a record (not pattern-matched), SNI www.foo.invalid.
• Both directions encrypted — 2 application-data records from the C64 (157 B), 8 from the host (1,271 B).
• HTTP 200 and the exact 24-byte body, read out of http_resp_buf; net_last_error $00.
• No cleartext body on the wire, and conclusively: the positive control (the SNI hostname, which TLS 1.3 leaves in the clear) was found twice in the same corpus, so this is a claim about the wire and not about the searcher.

Capture accounting: /tmp/rrnet-https.pcap 39,734 → 114,149 B across the run (22:53:23 → 23:27:23), 352 frames total, 238 inside the run window, 114 correctly excluded as older.

What is here

file	role
tools/ip65_hw_checks.py	every verdict, as pure functions over bytes
tools/test_ip65_hw_checks_unit.py	red-green for each; no hardware, ms, in testpaths
tools/mutate_ip65_hw_checks.py	breaks each verdict and requires the suite to go red
tools/rig-up-rrnet-macos.sh	the segment (adapted from c64-wireguard's, attributed in its header)
tests/rig_ip65_rrnet_hw.py	the hardware rig — supplies the pcap and the DMA reads, decides nothing

The split is the point. A first-ever hardware result is the one nobody re-reads, and this repo has three recorded instances of a suite passing for the wrong reason (#158, #161, #176). c64-wireguard wrote their equivalent module after shipping a tool cited for two days as "verified byte-for-byte" whose verification function was defined and never called and whose main() returned 0 unconditionally.

The rig is not judgment-free, and an earlier draft of this PR and of the docs said it was. Counted: 15 RES.verdict() calls delegate to the library; 18 RES.check() calls are the rig's own opinion, with no red case, and invisible to the introspection backstop because that enumerates only check_* in the module. Several fire on the green path (the boot-menu and DHCP screen scrapes, the cartridge-preference write, the listener probe, the zero-fill tail, the four selftests). The wire and memory verdicts do delegate — that is the part worth claiming — and CLAUDE.md, tests/README.md and the printed CERTIFIES string now claim that and not more.

Red-green, and then a check on the red-green

The unit suite: 26 tests, 47 negative assertions, 14 verdicts each with at least one known-bad input, 12 paired with a NAIVE checker asserting both halves — the naive arm passes the bad input (the trap is live), the real one fails it. test_every_check_has_a_red_case enumerates the module by introspection, so a verdict cannot be added without a failing case.

That was not enough on its own. tools/mutate_ip65_hw_checks.py breaks one verdict at a time and requires the suite to go red, and on its first run four red cases survived — they passed whether or not the checker worked:

survivor	why it proved nothing
check_dhcp_lease accepts ip65's build-time default	every red case also passed subnet=, and 192.168.1.64 fails the subnet test on its own
check_shadow_ram_readable loses the ROM-prefix arm	a real $A000 read carries both ROM markers, so either arm alone still rejected it
check_http_response compares only a prefix	every red case had a wrong length and wrong content, so the length test covered for it
check_mac_on_wire never looks at the wire	(caught, listed for completeness of the sweep)

Each is now asserted in isolation. 13/13 detectable mutants caught, and one is reported as KNOWN-EQUIVALENT rather than suppressed: removing check_http_response's length test cannot be detected, because resp_buf[:resp_len] already has the wrong length whenever resp_len is wrong. That test is kept for its message, not its coverage, and the harness says so.

The harness also carries a trap it hit itself. Python caches bytecode on (mtime, size), and a mutation run rewrites one path repeatedly within the same second, so two mutants of equal length execute the first one's bytecode; its first run attributed three verdicts to a test covering none of them. It now runs the subprocess with PYTHONDONTWRITEBYTECODE=1 — a harness that certifies other checks must not itself be the unreliable part.

The most instructive defect: tcp_stream fusion

tcp_stream assembled every matching frame into one sequence space keyed on the absolute TCP sequence number. Two connections on the cable — which any retry produces, and ip65 retries whenever it does not yet have the next-hop MAC — have unrelated ISNs, so that zero-filled the gulf between them and destroyed the record framing of both. A completed handshake would have read as "no TLS records": a false negative in a checker whose entire job is to avoid false results, latent until the first retry, and this run's own capture would have triggered it had the first connect been refused.

tcp_streams now groups per (src, sport, dst, dport) and tcp_stream returns the longest. The red case fuses a short high-ISN stub with the real low-ISN connection and requires the ClientHello still to be found:

frames = ...   # attempt 1: stub at seq 0xF0000000; attempt 2: the real CH at seq 1000
streams = hw.tcp_streams(frames, eth_src=C64_MAC, dport=PORT)
assert len(streams) == 2
assert hw.check_client_hello_on_wire(frames, C64_MAC, port=PORT, expect_sni=SNI).ok

Two device findings, filed rather than absorbed

• c64-test-harness#231 — the first hardware attempt aborted at the image verification. Measured n=2: a single 47,103-byte write_memory lands in 0.22 s and reads back with exactly one wrong byte, at a different offset each time (2715/$12BC, then 3181/$146E, $C8 → $AB); the 84-byte chunked path takes ~33 s and is byte-exact. Not truncation, not a size cap — and the harness's own docstring says the POST form is "verified at 2048 bytes", with both bad offsets past that. Without the verify, a one-byte flip in 6502 code surfaces 40 minutes later and gets written up as "ip65 does not work on real silicon".
• c64-test-harness#216 — independently reproduced their $0801/$0802 post-reset zeroing from a different lane and code path: those bytes go to 00 00 between ~2 and ~5 s after READY., with no write involved.
• No hardware rig verifies the PRG image after loading it — propose a post-load verify (blast-radius assessment of harness#231) #199 (this repo) — blast-radius assessment. No existing rig is affected: they load via client.run_prg (a different endpoint) and every write_memory in tools/uci/ is under 64 bytes. But none of them verifies either, so the class of fault is invisible to our tooling. Proposed there, deliberately out of scope here. Issue REU preflight: read the harness item map, and fail closed when it cannot (#179) #181's validation never loaded a PRG and is untouched.

Notes for review

• The PRG's hash differs from master's default ip65-onchip build purely because of HTTPS_PORT=4433 — expected, not a mixed build.
• Segment is 10.0.66.0/24, deliberately not the feth rig's 10.0.65.0/24, so both rigs can be up at once and real silicon can be compared against the emulated pair. The rig reads the addressing out of the shell script (hw.rig_const) rather than copying it, so the two cannot drift; a red case proves it refuses to invent a value for a name the script does not define.
• Only $0801-$9FFF is verifiable from the host. The image's $A000-$BFFF tail is BSS zero fill and a host read of it returns the BASIC ROM until boot.s banks it out — check_shadow_ram_readable is the discriminator and ran green on the device, so every $A000+ read in this run is RAM.
• This is the only hardware rig here that does not call enable_uci: the UCI command interface is a second consumer of the same expansion bus and an RR-Net run does not need it. Its value is read and reported (Disabled), never written. Cartridge Preference was Auto, set to External for the run and restored; the rig refuses to set a value it could not read back.
• The last commit makes two post-run corrections (a firmware string I had cited from a note instead of the device, and picking the busiest rather than the first non-C64 MAC), so the rig file here is not byte-identical to the one the run's provenance stamped. Neither changes what the run did.
• Bare pytest at the root: 128 passed (the 3 skips are test_uci_data_acc.py on an ip65 build, via the documented C64_UCI_TESTS_OPTIONAL=1).

What this does NOT prove

The ip65/RR-Net product has now run once on real CS8900a silicon — a full TLS 1.3 handshake and GET at stock 1 MHz against a local self-signed listener, verified on the wire and in the client's own memory. That is one run, one clock, one cartridge, one device, and a local listener rather than a real server; the image differed from the shipped one only in its build-time target port. It says nothing about server-name validation, which the ip65 product still does not perform.

Spelled out:

• Not the shipped image. It differs from c64-https-ip65-onchip.prg in HTTPS_PORT alone. An earlier draft of this PR said "the configuration it ships in", which was wrong by exactly that flag.
• One clock, one device, one run. 1 MHz on one U64E. Nothing here says the U64 times CS8900a register cycles correctly at 48 MHz — that clock was deliberately not used, because the stock-C64 product never runs at it.
• One cartridge, one server. A local listener with a self-signed P-256 leaf on a two-station cable with no switch and no route off it. Not a real server, not the internet, not an RR-Net MK3 (whose EEPROM would override the MAC this rig pins the lease on).
• Name validation is not exercised — src/x509_name.s is UCI-only, so the ip65 product still does no server-name checking. This run says nothing about that gap; it is the gap. Now stated in CLAUDE.md, tests/README.md and the rig's printed CERTIFIES string, so it survives outside this description.
• n=1 for the e2e. One passing run. The 33.0 min figure is a single measurement, not a distribution.
• The cleartext-absence check does not fail closed on its own. check_body_not_on_wire takes no c64_mac; it relies on check_c64_originated running in the same stage to establish that C64 frames were in the corpus at all. Its control needle is genuinely a C64 artefact in this capture (www.foo.invalid appears as contiguous ASCII in exactly one frame, the ClientHello — DNS length-prefixes labels, so it never appears contiguously in the query or the response), but the coupling is structural and worth knowing.

🤖 Generated with Claude Code