write_memory errors when running test_https_local.py

[Armitage64]

Several errors show up in the console when running the test_https_local.py script. On the C64 screen, the following text is displayed but the test does not complete successfully:

SYS16896
CH
SH
HK1
KEYS
ENC1
RX

at this point, the machine appears to lock up and become unresponsive.

Console output is below:

Enabling UCI...
Setting turbo to 48 MHz (from Off/ 1)...
Resetting machine...
run_prg(PRG)...
net_initialized = $01
Starting U64 6510 debug stream -> 192.168.1.212:11002
WARNING: debug capture failed to start: PUT http://192.168.1.233/v1/streams/debug:start?ip=192.168.1.212%3A11002 returned HTTP 404: {
"errors" : [ "Network Host Resolve Error" ]
}
Sending 'Q' to exit PRG main_loop...
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:410: UserWarning: write_memory($0277, 2 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:414: UserWarning: write_memory($00C6, 1 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1487: UserWarning: write_memory($4200, 64 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1487: UserWarning: write_memory($4240, 53 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1491: UserWarning: write_memory($4300, 32 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(HOST_STR_ADDR, host_str.ljust(32, b"\x00"))
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1492: UserWarning: write_memory($4340, 8 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(PATH_STR_ADDR, path_str.ljust(8, b"\x00"))
/Users/armitage/git/c64-https/tools/uci/test_https_local.py:1495: UserWarning: write_memory($4380, 16 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(SENTINEL_ADDR, bytes(16))
Triggering: sys16896
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:410: UserWarning: write_memory($0277, 9 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
[ 0.5s] progress=0x03

[JC-000]

Thanks — and this one is a genuinely useful report, because the screen text you captured is the diagnostic.

The screen tells us where it stopped

SYS16896  CH  SH  HK1  KEYS  ENC1  RX

That is: ClientHello sent, ServerHello received, handshake keys derived, first encrypted record received... and then nothing. The next marker you should see is GOT2. Its absence, with everything before it healthy, matches a signature we have hit before and documented:

every TLS handshake stalling deterministically at the first encrypted record, screen ending KEYS ENC1 RX, never reaching GOT2, at any clock speed — while DHCP, TCP and plaintext HTTP all keep working.

On our bench that turned out to be transient device state, not client code. Soft resets did not clear it; a hard power cycle at the wall did. Firmware was unchanged throughout, and a known-good older commit failed identically while the device was in that state — which is why we wrote it down rather than bisecting again.

So the first thing worth trying is the cheapest: power the Ultimate fully off at the wall, not a menu reset, and re-run. If it then completes, that is the whole story.

If it survives a power cycle

Then it is worth diagnosing properly, and two values would settle it quickly. Both are printed in the failure dump, or readable from the artifact directory the script writes under /tmp/uci_https_debug/<timestamp>/:

• net_last_error — $86 is UCI_ERR_READ_FAIL (SOCKET_READ returned an error bit), which is what we saw in the wedge case
• tls_recv_sub_progress — $02 in that same case

Please paste those, plus tls_state / tls_last_state, and I will take it from there. A baseline run of python3 tools/uci/boot_check.py and python3 tools/uci/phase3_tcp_echo.py is also informative: if plain TCP echo works while TLS stalls at the same place every time, that points away from your setup and toward the device.

The console output is noise, not errors

The UserWarning: write_memory(...): no safe_regions declared lines are benign and expected on every run, including passing ones. They come from the harness's memory-policy guard, which is deliberately set to warn rather than deny for these scripts. They are not related to your failure. They should not be shouting at you on a normal run, though — that they read as errors is our problem, and I have noted it as something to quieten.

The debug capture failed to start ... Network Host Resolve Error 404 is also harmless: that is the optional 6510 bus-trace stream trying to reach 192.168.1.212:11002, which is a host/port your Ultimate could not resolve. The test proceeds without it. You can silence it with DEBUG_CAPTURE=0.

One thing that would help me confirm the match: does it fail at the same point every time, and does it also fail at TURBO_MHZ=1? Deterministic-at-any-clock was the distinguishing feature last time.

[Armitage64]

The test always fails with "RX" on the screen, even after a cold power cycle. There are no artifacts in the log directory at the time the test locks up with " [ 0.6s] progress=0x03" displayed on the console.

Here is the output of the boot_check.py script:

% python3 tools/uci/boot_check.py
Loaded 62977 bytes from /Users/armitage/git/c64-https/build/c64-https.prg
Expected backend: uci ('UCI NETWORKING')

--- PRG image checks ---
[PASS] image carries uci banner: expected bytes 'UCI NETWORKING' in the PRG
[PASS] image free of foreign banner: none present

Acquired DeviceLock(192.168.1.237)
Enabling UCI (Command Interface)...
Resetting machine...
run_prg(PRG)...
Waiting up to 60s for the main menu...
main menu reached

--- screen RAM decoded (non-empty lines) ---
00: c64-https client v0.1
02: tls 1.3 / chacha20-poly1305
03: uci networking
05: initializing network...
06: network ok
07: requesting dhcp...
08: dhcp ok - ip: 192.168.1.237
09: i=init h=http g=https q=quit
--- end screen ---

--- boot checks ---
[PASS] common banner: expected 'C64-HTTPS CLIENT V0.1'
[PASS] uci backend banner: expected 'UCI NETWORKING'
[PASS] no foreign backend banner: none present
[PASS] no FAILED on screen: no failure message
[PASS] main menu reached: expected 'Q=QUIT'

PASS: uci PRG booted cleanly to the menu
Disabling UCI...
Released DeviceLock(192.168.1.237)

Here is the output of phase3_tcp_echo.py:

% python3 tools/uci/phase3_tcp_echo.py
net_init = $2000
net_dhcp_acquire = $21CA
net_dns_resolve = $2497
net_tcp_connect = $2259
net_tcp_send = $232B
net_tcp_close = $245A
net_poll = $2044
net_recv_byte = $2536
net_send_len = $B9B1
net_tcp_state = $B9B0
net_last_error = $B9AF
uci_host_buf = $6760
MemoryPolicy reserved 7 region(s); arbiter allocations:
$4200-$42FF trampoline
$4300-$433F host_buf
$4340-$435F test_string
$4360-$43DF result_buf
$43E0-$43E0 sentinel
$43E1-$43E1 progress
$43E2-$43E2 connect_carry
$43E3-$43E3 send_carry
$43E4-$43E4 result_len
$43E5-$43E5 poll_count
Dev host LAN IP : 192.168.1.212
Echo port : 7777
Test string : b'HELLO UCI'
Echo server listening on 192.168.1.212:7777
Routine size : 175 bytes @ $4200
Acquired DeviceLock(192.168.1.237)
Enabling UCI (Command Interface)...
Resetting machine...
run_prg(PRG)...
net_initialized = $01
net_tcp_state = $00
Sending 'Q' to exit PRG main_loop...
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:410: UserWarning: write_memory($0277, 2 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py:414: UserWarning: write_memory($00C6, 1 B): no safe_regions declared and unknown_policy is not 'allow'
self.write_memory(
/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:428: UserWarning: write_memory($4200, 64 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(
/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:428: UserWarning: write_memory($4240, 64 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(
/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:428: UserWarning: write_memory($4280, 47 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(
/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:432: UserWarning: write_memory($4300, 32 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(HOST_BUF_ADDR, host_bytes)
/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py:433: UserWarning: write_memory($4340, 9 B): no safe_regions declared and unknown_policy is not 'allow'
transport.write_memory(TEST_STRING_ADDR, TEST_STRING)
Disabling UCI...
Released DeviceLock(192.168.1.237)
Traceback (most recent call last):
File "/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py", line 537, in
raise SystemExit(main())
File "/Users/armitage/git/c64-https/tools/uci/phase3_tcp_echo.py", line 438, in main
transport.write_memory(labels["uci_host_buf"], host_bytes)
File "/Users/armitage/git/c64-test-harness/src/c64_test_harness/backends/ultimate64.py", line 158, in write_memory
self._memory_policy.check_write(addr, len(data), override=override)
File "/Users/armitage/git/c64-test-harness/src/c64_test_harness/memory_policy.py", line 428, in check_write
raise MemoryPolicyError(
c64_test_harness.memory_policy.MemoryPolicyError: write_memory($6760, 32 B) → $677F blocked: overlaps reserved region $6000-$9FFF (segment:CRYPTO_HOT). Pass override="" to bypass for a single call, or update the harness MemoryPolicy.

[JC-000]

My last answer was wrong. Sorry — that power cycle was wasted effort.

You cold-cycled the machine and got the identical failure, so the wedge I pointed you at is ruled out: that one is transient device state and a hard power cycle clears it. Yours is deterministic, which is the opposite signature. Retracting it.

Two smaller corrections from that same comment, because both sent you looking in the wrong place:

It isn't locked up — it's spinning. tls_recv_encrypted (src/tls13.s) prints ENC1, then RX, then loops net_poll + tls_record_recv_and_decrypt up to 65,536 times before giving up. A UCI net_poll is a full firmware command round-trip at ~40 ms, so that loop runs for about 44 minutes. progress=0x03 only means "entered http_get, hasn't returned yet" — where it sits for the entire handshake. The console has nothing new to print.

That's also why there were no artifacts. The script writes its artifact dir when it finishes or when it hits SENTINEL_POLL_TIMEOUT, default 600 s. Interrupting before that leaves nothing behind. Run it with SENTINEL_POLL_TIMEOUT=180 and you'll get /tmp/uci_https_debug/<timestamp>/ including tls_state_dump.json.

We reproduced your failure, by turning the REU off

Not a hypothesis — a reproduction. On our U64E, same REU-profile PRG, two runs back to back at 48 MHz, the only variable being RAM Expansion Unit:

REU	result
Enabled	PASS in 83.4 s, http_status=200, body decrypted
Disabled	stalls forever at RX, progress=0x03

And the failing screen is character-for-character yours:

SYS16896
CH
SH
HK1
KEYS
ENC1
RX

Why

The default make BACKEND=uci build — the 62,977-byte PRG you have — is the REU profile. In it, X25519's field multiply (src/crypto/fe25519.s) fetches its multiplication rows from the REU by DMA on every call.

With no REU the DMA silently does nothing. It doesn't fault; the rows just stay at whatever reu_mul_init left behind, so every multiply returns a wrong-but-consistent result. The consequence lines up exactly with your screen:

• CH, SH — plaintext, so they work regardless. Your ServerHello arrives fine.
• HK1, KEYS — handshake keys get derived, but from a wrong ECDH shared secret.
• ENC1, RX — the first encrypted record arrives and fails its AEAD tag. Forever.

Corroborating detail from our failing run: tcp_recv_head and tcp_recv_tail both sat at $02B2, meaning the full ~690-byte server flight arrived and was consumed. The bytes are all there. The C64 just can't decrypt them.

What to try

Run the onchip image instead. It needs no REU at all — it generates those rows on the CPU:

c64-https-uci-onchip.prg (also as .d64)

Point the test at it with C64_PRG=/path/to/c64-https-uci-onchip.prg, or build it yourself with make clean && make BACKEND=uci USE_NISTCURVES_ONCHIP=1.

We have run that image on a U64E with the REU explicitly disabled and it completes the full handshake — 44.9 s at 48 MHz, 262 s at 8 MHz, 2131 s at stock 1 MHz. So if the REU is your problem, this image should just work, unchanged.

The other fix is to enable the REU: Settings → C64 and Cartridge Settings → RAM Expansion Unit. Worth knowing that the C64 Ultimate ships with it Disabled out of the box, and that setting is exactly what this build needs. Above roughly 18 MHz the onchip build is the faster of the two anyway, so it's a reasonable default even with an REU present.

Two things that would confirm it

1. net_last_error. This is the discriminator, and it's why I'm asking rather than deflecting — the two failure modes look identical on screen and differ only here:

   • 0x00 → network is healthy, the failure is in the crypto → REU. That's what our REU-less reproduction produced.
   • 0x86 → UCI_ERR_READ_FAIL, which is the device wedge I wrongly sent you after.

   Note tls_recv_sub_progress is 0x02 in both cases, so it can't tell them apart — which is part of why I misread yours.

2. Whether your Ultimate has the REU enabled (the menu path above).

Both are in tls_state_dump.json once you let the script reach its timeout; net_last_error is also printed directly in the failure dump.

Your phase3_tcp_echo.py crash is our bug, and it's fixed

MemoryPolicyError: write_memory($6760, 32 B) → $677F blocked: overlaps
reserved region $6000-$9FFF (segment:CRYPTO_HOT)

Nothing to do with your setup — that script cannot complete on any device. It DMA-writes uci_host_buf to pre-seed the hostname, and uci_host_buf sits at $6760, inside a region the harness's memory guard reserves wholesale. The write is legitimate; it was just missing the override= marker that says so. Fix in #105.

Thanks for pushing back on the first answer instead of accepting it. The cold power cycle is what made this diagnosable.

[Armitage64]

Confirmed that phase3_tcp_echo.py now runs successfully.

Confirmed that test_https_local.py now runs successfully with 16 MB REU enabled. Note that the readme.md does not indicate the REU configuration is required in the section labelled "Ultimate 64 Elite Hardware Tests (UCI backend)". Either add this to the Readme, or update the test to enable and configure the REU via the command interface. A test to verify the REU is enabled at the beginning of the script would be useful.

[JC-000]

Your diagnosis was right, and your close-out request is now the fix

Confirmed on both counts. And you were right that a check at the start of the script is the better answer than a README paragraph — a prerequisite a program can verify should not be left as prose. That is what shipped: #107, the check first and the docs second.

What changed

tools/uci/_reu_preflight.py, called under the DeviceLock the script already holds, right after enable_uci and before the reset that starts the long run:

• REU-profile build meets a device with the REU disabled → exits 4 in about 2 seconds with a message naming both remedies (enable it via Settings → C64 and Cartridge Settings → RAM Expansion Unit, or build USE_NISTCURVES_ONCHIP=1, which needs no REU). Not after 44 minutes.
• On-chip build → let through untouched, with no REST call at all. That case mattered most: a blanket check would have broken the exact configuration we recommend to people without an REU. The profile is detected from build/labels.txt using four independent markers checked as a union, so no single upstream rename can misclassify an on-chip build and block it.
• C64_SKIP_REU_PREFLIGHT=1 if you ever want it out of the way.

Guarded: test_https_local.py (and the test_https_print_body.py / test_https_local_p384.py wrappers that delegate to it), test_https_bad_finished.py, bench_ecdsa_u64e.py. Not guarded, on purpose: boot_check, phase2_check, phase3_tcp_echo, test_http_local, test_http_live — none of them touches the REU.

Measured on our U64E at 48 MHz, all three cases, REU setting restored afterwards:

build	device REU	result	wall
REU	Disabled	exit 4, actionable message	2.08 s
REU	Enabled	PASS, HTTP 200	128.42 s
on-chip	Disabled	PASS, HTTP 200	90.99 s

The check itself costs 0.073–0.081 s on the REU path and 0.001–0.002 s on the on-chip path.

One thing it deliberately does not do is enable the REU for you, which you offered as the other option. Our U64E is queue-shared across several projects and its config writes persist until the next power cycle, so a test that silently reconfigures the hardware trades a legible error for a mystery two runs later on somebody else's branch. The message tells you the menu path instead.

The README's "Ultimate 64 Elite Hardware Tests (UCI backend)" section now states the requirement and that the preflight enforces it.

A correction to something I told you earlier

In the diagnosis comment I suggested pointing the test at the on-chip image with C64_PRG=…. That only works for boot_check.py. The other scripts read build/c64-https.prg and build/labels.txt from the same link and DMA to addresses out of those labels, so a mismatched pair would misbehave in confusing ways. The right way is to build it: make clean && make BACKEND=uci USE_NISTCURVES_ONCHIP=1. Sorry for the loose advice.

The wedge note that sent you on the wasted power cycle has also been fixed at the source (#106). That screen — CH / SH / HK1 / KEYS / ENC1 / RX, never GOT2 — has two causes, and our notes only recorded one, which is why I confidently gave you the wrong one. It now carries the discriminator (net_last_error: $86 = device wedge, $00 = no REU behind a REU-profile build) and an explicit instruction not to act on the screen alone. Your report is what exposed that gap.

Thank you — properly

Across #90, #93 and #97 you have found three real gaps and caught one genuine misdiagnosis of ours:

• Module c64_test_harness missing when running UCI test scripts #90 — the UCI test path never told anyone to install c64-test-harness. It was documented under the VICE section only, so anyone who built with BACKEND=uci and went straight to tools/uci/ hit a bare ModuleNotFoundError.
• Certificate missing and is not created when running tools/uci/test_https_local.py #93 — tools/https_e2e/certs/ is gitignored throwaway material, so a fresh clone had no cert and the script died on it. Now generated on demand.
• write_memory errors when running test_https_local.py #97 — this one: phase3_tcp_echo.py could not complete on any device (a legitimate DMA write missing its override= marker, fixed in fix(tools/uci): phase3_tcp_echo blocked by MemoryPolicy on its own uci_host_buf write (#97) #105), the REU prerequisite was unstated, and our own wedge note was wrong.

Every one of those is a defect that only shows up for someone arriving from outside the bench it was developed on, and every report was precise enough to act on directly — the screen capture in particular is what made the REU diagnosis possible, and pushing back on my first answer instead of accepting it is what got us there. That is genuinely valuable, and we would not have found any of it on our own.

Closing this. Please keep filing.