Round 2 plan: honest FFI reference capabilities for stdlib + tools (#4925)

[SeanTAllen]

This is the Round 2 plan for the C-FFI safety effort (#4925; Round 1 was the Pointer/UnsafePointer split, #5427). The goal this round: make every FFI declaration in packages/ and tools/ capability-honest. The rule is simple. The capability on an FFI parameter must describe what C does to the memory. C writes through it, the declaration says a mutable cap. C only reads, weaker caps are fine.

Branch: ffi-cap-accuracy, stacked on Round 1's unsafe-pointer. Two standing constraints up front: the readonly optimization that #4927 disabled stays disabled (re-enabling it is a later round, after the ecosystem has had time to adjust), and the plan assumes no C changes (one open question below pokes at that boundary for a single function we own).

I audited every FFI declaration we have against the honesty rule. Most of the fixes are mechanical and the full plan for them is below. But one class of call site has no honest expression in today's language, and I'm leading with it because it blocks part of the plan and it deserves its own design conversation.

The problem that blocks part of the plan: you can't lend an iso to C

Today's declaration, from packages/net/udp_socket.pony:

use @pony_os_recvfrom[U8](event: AsioEventID, buffer: Pointer[U8] tag,
  size: USize, from: NetAddress tag, count_out: Pointer[USize])

C writes the datagram bytes into buffer and the sender's sockaddr into from (socket.c, pony_os_recvfrom). Both parameters say tag. Both are lies, and they're the exact class of lie that produced #4925: LLVM 21 started using our parameter attributes and deleted writes that C actually performs. The honest declaration is:

use @pony_os_recvfrom[U8](event: AsioEventID, buffer: Pointer[U8] ref,
  size: USize, from: NetAddress ref, count_out: Pointer[USize])

Two words change. Now look at the only call site, the POSIX read loop:

let data = _read_buf = recover Array[U8] .> undefined(size) end  // Array[U8] iso
let from = recover NetAddress end                                // NetAddress iso
var count: USize = 0
match \exhaustive\ _SocketResultDecoder(
  @pony_os_recvfrom(_event, data.cpointer(), data.space(),
    from, addressof count))
| _SocketResultOk =>
  data.truncate(count)
  _notify.received(this, consume data, consume from)

Both objects are iso, and they need to be: data is delivered to UDPNotify.received as Array[U8] iso, from as NetAddress (a val class; the consume gives iso^, which converts). Under the honest declaration, nothing here typechecks, and every route out is a dead end. I checked each of these against the compiler rather than reasoning from memory:

• data.cpointer() can't produce a ref pointer. Anything reachable through an alias of an iso is at most tag. That's not a flaw; that's the alias law that makes iso safe to send.
• Passing from directly aliases the iso, and an alias of iso is only tag-compatible. Same law.
• consume from doesn't help either. I'd assumed it would typecheck against the ref parameter and merely destroy the binding; the compiler corrected me. Arguments are aliased, so the consumed iso^ arrives at the parameter check as iso, and iso is not a subtype of ref. Transfer is expressible only by declaring the parameter iso and consuming into it, and then the object is gone unless C returns the pointer back. That works for runtime shims we own and fails for the actual point of FFI: read(2), getsockopt(2), OpenSSL. The world of C will never hand our pointers back to us.
• A recover block round-trips exactly one object. You consume the iso in, view it as ref for the call, take the result back out. That's the fix Joe and I sketched on LLVM 21 readonly optimization breaks FFI writes through tag pointers #4925 and it handles most of the audit. It cannot handle this site: one call fills two objects, they leave at different capabilities, and a recover lifts its result to one target capability. There is no (iso, val) exit. Recovering a tuple to iso is rejected outright.

Inside today's language, what's left is machinery: copy the address out field-by-field through a new package-private constructor, or invent a wrapper class so two objects ride through the one-object construct as one. I prototyped both. They compile. They're also patches around a missing language capability, and building them would entrench the gap instead of fixing it. We're not doing that.

The actual problem: iso, but lent

State what the call site wants and the shape of the gap is plain. C may mutate this memory for the duration of the call. C retains nothing. I keep my iso afterward. Pony has consume for permanent transfer and recover for one-object round trips, and nothing that says "yours for this call, mine again after."

Note what's true during the call itself: a synchronous FFI call suspends the calling actor, and iso already guarantees no other actor can reach the memory. So while C runs, the iso's exclusivity is intact. The consume the type system demands isn't a price the call needs; it's alias bookkeeping with no way to say "same iso, before and after."

The buffer half of the problem is harder still. What crosses the FFI for data isn't the array, it's data.cpointer(): a pointer derived from the iso mid-expression. Any language-level answer would have to cover not just object arguments but pointers derived from them, and that is real design work, not a one-line subtype check.

What today's language offers is the recover round-trip (one object per call) and, for C we own, shims that take ownership and give it back. Whether the language should grow something better than that is a genuine design question, and it's bigger than this plan. I want to hear shapes before anchoring on one. Two constraints any proposal has to respect: the buffer case crosses as a derived pointer, not as the object; and nothing in this space touches the retention problem below, which no call-boundary change of any shape can fix.

Retention is a different problem entirely

There's one place in the audit where even "honest during the call" is unreachable, and I want to name it because it bounds what any answer to the above could ever cover. On Windows, pony_os_recvfrom queues overlapped I/O: the kernel writes through the buffer and address pointers after the call returns, until the IOCP completion fires. That's not temporary access during a call. It isn't ref either. It's transfer-now, ownership-back-at-completion, a third semantics that no capability at a call boundary describes — and no language change to the synchronous case would change that. Today it's a documented exception, same as the asio event handles.

Until something changes here — a language answer for the synchronous case, or the owned-C shim in open question 4 — the plan leaves exactly one parameter knowingly dishonest: pony_os_recvfrom's from stays tag, with a comment pointing here. Everything else gets fixed now.

What the audit found

Every use @ declaration in packages/ and tools/ was classified: 476 top-level declarations (a further handful of use @... strings exist only inside test-fixture string literals and docstring examples — not compiled declarations). C mutation behavior was verified against src/libponyrt/, src/libponyc/, or documented libc/Win32 semantics, not guessed from parameter names.

Verdict	Count (decls)
genuinely unsafe (violation / too-permissive decl)	58 — 25 in stdlib + tool frontends, 33 in the pony_compiler library
accurate	~90
scalar-only (no pointers/objects involved)	~300, mostly LLVM intrinsics
runtime-intentional (the asio event family)	28 usages across 7 packages
provenance misclassifications (Round 1 follow-ups, not cap bugs)	~25

The genuinely-unsafe set, grouped by shape:

• Group A — C-written object out-params declared tag (the direct siblings of the original LLVM 21 readonly optimization breaks FFI writes through tag pointers #4925 getsockname bug): pony_os_sockname/peername/getaddr/recvfrom filling NetAddress tag, and the pony_os_stat/fstat/fstatat trio filling FileInfo tag, plus pony_os_nameinfo's out-pointers. Two of these are active val violations today, not just latent ones: UDPSocket._ip is an embedded val NetAddress that C fills in three constructors, and FileInfo.mode is a FileMode val whose bits C writes through p->mode->* (stat.c).
• Group B — buffer fills crossing the boundary at tag: every C-fills-a-Pony-buffer call goes through cpointer()/cstring(), which return tag today regardless of receiver. read/_read, pony_os_recv/recvfrom/stdin_read, getsockopt, snprintf/_snprintf (used as a write target through cstring()), ReadFile, and get_compiler_exe_directory in the three tool frontends.
• Group C — addressof out-params behind tag declarations: waitpid's stat_loc, ponyint_win_pipe_create's fd out-params, ponyint_win_process_create's error_msg, OpenProcessToken's token handle. Call sites already hold mutable locals; only the declarations lie.
• Group D — opaque foreign handles C mutates, passed at tag: the FILE* family (pony_os_std_flush/print/write; fprintf in debug, assert, json, iregex, and the tool _unreachable files), the directory handles (closedir, ponyint_unix_readdir, FindNextFileA, FindClose), and freeaddrinfo. A tag handle is sendable while C mutates behind it.
• Runtime-intentional, classified separately: the runtime mutates asio event internals through AsioEventID (UnsafePointer[AsioEvent] tag) by design, because events must stay sendable to be delivered through _event_notify behaviors and the runtime owns the synchronization. The plan documents this as the deliberate exception rather than pretending ref is achievable there.
• tools/lib/pony_compiler is its own problem: 33 findings, all stemming from one convention — the library claims Pointer[_AST] val and a sendable class val AST while libponyc mutates AST nodes in place (ast_passes_program rewrites the whole tree through a val parameter; even read paths mutate token caches; finalizers free through box). Fixing that honestly changes the library's public types and how pony-lsp/pony-doc/pony-lint hold ASTs. That's a design round of its own, deferred, with the findings tracked.

Two compiler facts shaped a lot of verdicts. Bare pointer types in FFI declarations default to ref, so many out-params (@pipe's fds, @clock_gettime's timespec, the count_outs) are already accurate. And a parameter typed Pointer[None]/UnsafePointer[None] goes through the void*-rule (void_star_param, src/libponyc/expr/ffi.c), which skips capability checking entirely — so several fixes tighten [None] to a concrete element type, or the new cap would enforce nothing.

The audit also confirmed the escape hatches: addressof mints Pointer[T] ref to any var regardless of receiver capability (including from fun box methods), .usize() launders pointers into integers that the void*-rule accepts back, and _unsafe() + from_cpointer is builtin's internal forgery tool. All current uses are authorized; closing the hatches is future language/lint work.

Decisions already made

• cpointer() itself changes, no second accessor. Viewpoint adaptation is the mechanism that exists for exactly this; details under Divergence 1.
• Recover round-trips are the idiom; invented machinery is not. I prototyped a parts-constructor/copy path on NetAddress and a wrapper-field consolidation for the Windows UDP reader. Both compiled, both were patches around the gap described at the top, both are out. The record of that road is below, so nobody re-walks it without new information.

Divergences

Each point where this plan changes scope or behavior relative to today, with an explicit justification. Why there are nine: #5 is the round's mission itself; #1, #4, and #9 are enablers without which the mission's declaration changes cannot compile; #2 and #3 are the two active val violations the audit found; #6 is a two-line bug fix on a declaration the mission already edits; #7 and #8 are policy decisions. Every entry traces to an audit finding or to a compiler constraint demonstrated with a scratch program — none is elective restructuring.

1. Array[A].cpointer() and String.cpointer() change return type from Pointer[...] tag to this->Pointer[A] / this->Pointer[U8]. This is a breaking change for code calling cpointer() on iso receivers.
   • Current: cpointer() always returns tag, regardless of receiver cap — so every C-fills-this-buffer FFI call crosses the boundary at tag, and the mission's Pointer[U8] ref declarations would be unsatisfiable.
   • Proposed: the returned pointer carries the receiver's view — ref holders get ref, val holders get val, box holders get box. All existing read-only uses on ref/val/box receivers keep compiling (every cap is a subtype of tag). iso receivers can no longer call it directly (auto-receiver-recovery rejects arrow-typed returns) and use the consume-into-recover patterns below.
   • Why: viewpoint adaptation is what the language has for "the answer depends on how you hold the object," and it's the original LLVM 21 readonly optimization breaks FFI writes through tag pointers #4925 sketch. Costs measured against the whole tree: chop/unchop in String and Array rewrite through the private _ptr field (~10 lines each pair; String rewrite verified compiling, Array is isomorphic); two read-only test sites (net/_test.pony:893, 900) bind the pointer before a recover; every other in-tree break is an FFI write site this plan restructures anyway. I tried the alternative (a second, fun ref accessor; cpointer() untouched): it avoids the breakage but leaves the main accessor permanently dishonest-by-weakness and adds API surface forever. cstring() does NOT change: its fresh-allocation fallback path returns a pointer that cannot honestly carry the receiver's cap (verified: the body fails to typecheck under an arrow return), and its legitimate uses are read-only — its docstring will say so.
2. FileInfo.mode field type changes from FileMode val to FileMode (ref).
   • Current: mode is FileMode val from birth, and the stat trio writes every mode bit through it — an active mutation of val-claimed memory, the exact LLVM 21 readonly optimization breaks FFI writes through tag pointers #4925 bug class.
   • Proposed: field declared FileMode (ref); C fills it during construction through an honest mutable cap.
   • Why: retyping the stat declarations to FileInfo ref is a half-truth while the object's own mode field still claims val during the write. There is no way to keep the field val and have C legally write it (embed would be allocation-free and honest, but it inlines the object and pony_stat_t declares mode as a pointer — the C mirror breaks, and C changes are off the table). Practical impact: none — every FileInfo is built new val, so viewpoint adaptation makes info.mode read as FileMode val exactly as today; only the declared field type (API-visible) changes.
3. UDPSocket._ip loses embed: becomes let _ip: NetAddress (val), assigned from a recover block in which C fills a ref NetAddress.
   • Current: embed _ip: NetAddress — an embedded val object that @pony_os_sockname fills in three constructors; the other active val violation.
   • Proposed: build the NetAddress as ref inside a recover, let C fill it, lift to val, assign to a let field.
   • Why: an embed field is constructed in place and can never pass through a recover, so there is no way to keep embed and have C write through an honest cap. Cost: one extra small allocation per UDPSocket constructed; behaviorally invisible.
4. _to_string/_format_float switch the snprintf output buffer from cstring() to cpointer().
   • Current: @snprintf(s.cstring(), ...) — a read accessor (null-terminated C string) used as a write target.
   • Proposed: s.cpointer() — s is a String ref at both sites, so the viewpoint-adapted return is Pointer[U8] ref.
   • Why: forced by the mission — once the snprintf str param is Pointer[U8] ref, the tag-returning cstring() no longer compiles there, and cstring() cannot be the thing that changes (see Should sequences be scopes #1).
5. The mission: FFI declarations across builtin, files, net, process, format, debug, assert, json, iregex, and the three tool frontends change caps (tag→ref, or tag→bare-which-defaults-to-ref) on params C writes; several [None] params/returns get concrete element types.
   • Current: 25 declarations let C mutate memory behind tag/val/box claims (Groups A-D above).
   • Proposed: every C-mutated param carries a mutable cap, except the recvfrom carve-out; reads are not churned.
   • Why: this is the round's purpose. The element-type tightening rides along because it is load-bearing, not cosmetic: a cap flip on a [None] param enforces nothing (void*-rule). Visible only inside the declaring packages — FFI declarations are package-scoped.
6. The Windows-error-path string in process is copied, not adopted: error_msg's inner type becomes UnsafePointer[U8] val and the call site switches String.from_cstring → String.copy_cstring.
   • Current: the char* that FormatMessageA LocalAlloc's is adopted as the backing store of a Pony String — foreign memory wearing a Pointer, which Pony's GC will eventually treat as pony heap (latent heap corruption, Windows error path only).
   • Proposed: copy the bytes into a fresh Pony String via the Round 1 bridge.
   • Why: the audit found it, and the mission already edits this exact declaration — fixing the adopt is two lines on a line we are touching. The remaining LocalFree leak is not fixed here (needs a new FFI binding — genuinely separate concern, filed as an issue in step 10).
7. AsioEventID keeps tag and gets an explicit documented exception (docstring at the type alias).
   • Current: the runtime mutates asio event internals through UnsafePointer[AsioEvent] tag handles; nothing documents that this contradicts the cap-honesty principle.
   • Proposed: no cap change; the exception and its rationale are documented where the type is declared.
   • Why no fix: sendability is load-bearing — events are delivered via be _event_notify, and behavior parameters must be sendable, so ref is impossible without redesigning runtime event delivery (far out of scope). The honest alternative to an impossible fix is a documented exception. Open question 2 if anyone sees a better shape.
8. pony_compiler (tools/lib) is NOT fixed in this round.
   • Current: the libponyc-wrapping library claims Pointer[_AST] val and a sendable class val AST while libponyc mutates ASTs in place — 33 findings.
   • Proposed: defer to a dedicated design round (tracking issue with the full finding list, step 10).
   • Why: honest caps there mean AST handles stop being val/sendable, which changes the library's public types and how the three tools hold and share them — a design problem. Cap-flipping 33 declarations onto a wrong architecture would entrench it. This narrows the round's literal "packages/ AND tools/" scope: tool frontends fixed, shared library deferred with a written plan-of-record. Open question 3.
9. The dns addrinfo handle is unified onto one typed handle (UnsafePointer[_AddrInfo]), retyping two returns and two read-only params the audit itself classified as accurate.
   • Current: the same foreign handle is UnsafePointer[U8] on returns and UnsafePointer[None] on params.
   • Proposed: one primitive _AddrInfo element type everywhere; caps change only on @freeaddrinfo (tag→ref — it frees the chain).
   • Why: forced, not elective. The freeaddrinfo cap fix enforces nothing while the param is [None] (void*-rule), and once that param has a concrete element type, every handle flowing into it must carry the same element type to typecheck. Read-only params keep tag — no cap churn.

Removed by decision (the record of the road not taken): a tenth and eleventh divergence existed in earlier drafts — NetAddress explicit constructors plus a package-private _from_parts/_copy pair, and a Windows UDP wrapper iso field consolidating _read_buf + _read_from. Both compiled. Both were patches around the gap at the top (one FFI call filling two objects that must exit at different caps, against a recover's single-target-cap limit). They are out; pony_os_recvfrom.from stays tag, tracked, instead.

Non-divergences worth stating: cstring()'s signature is untouched; no compiler or runtime C changes anywhere (pending open question 4); read-only tag params (paths, format strings, write/send buffers) are deliberately not churned.

Fix patterns

All verified against the compiler with scratch programs before they went in the plan. The recover mechanics the patterns rely on, each verified:

• A recover block lifts its result to one target cap. Recovering a tuple of ref-born objects to val works (everything becomes val), but you cannot get two different caps out of one recover — there is no (iso, val) exit, and recovering a multi-object tuple to iso is rejected outright. That single-target-cap constraint is why the recvfrom carve-out exists.
• Sendable values cross freely: scalar FFI results assign to outer vars from inside the recover; sendable fields (_fd: U32, _event: AsioEventID) are readable inside it; addressof on an outer var compiles; non-sendable values must be bound before it.
• Inside a recover, a method call on a field-access receiver is only allowed when args and return are all sendable — hoist the field to a recover-local binding first, then call on the local.
• new val constructors require sendable parameters, and let fields with initializers cannot be constructor-reassigned (both of these are why the rejected copy-machinery needed a scalar parts-constructor — recorded above).
• Auto-receiver-recovery rejects arrow-typed returns: an iso receiver cannot call the viewpoint-adapted cpointer() directly. The admission check (auto_recover_call, src/libponyc/expr/call.c) evaluates the method's formal result type without substituting the call-site receiver into the arrow, so this->Pointer[U8] fails a check that Pointer[U8] tag — the substituted result for an iso receiver — would pass. A compiler issue for this is filed in step 10; until then, iso write sites use the consume-into-recover patterns and iso read uses bind a pointer before a recover.

The patterns (the mutable pointer is always obtained by calling cpointer() on a ref binding):

• P-out (fresh object C fills, result needs val): recover val let x: T ref = T; @ffi(..., x); x end — the explicit ref annotation matters; an unannotated binding infers iso and the call fails.
• P-iso-local (iso buffer local to one method): buf = recover iso let b: Array[U8] ref = consume buf; n = @ffi(..., b.cpointer(), ...); b end — the FFI's scalar result threads out via an outer var when the post-fill logic (truncate length, errno checks) needs it.
• P-iso-field (long-lived iso field buffer C fills): destructive-read the field with a cheap placeholder (an empty array — no data allocation), run P-iso-local on the local, store the result back into the field. The same buffer object returns to the field. process_monitor.pony already uses this swap today.
• P-this (object under construction is the out-struct): new val constructors may pass this at ref to FFI — verified to typecheck. This is how the FileInfo stat calls stay structurally unchanged.
• P-decl (call site already mutable): decl-only flip tag→ref/bare.
• P-handle (opaque foreign handle C mutates): UnsafePointer[X] ref param; holders already have effective-ref handles from bare FFI returns.

Step ordering

The cpointer() signature change (step 1) immediately breaks every iso-receiver call site, so those restructures must land with it. The reverse dependency does not exist: a restructured call site produces a ref pointer, and ref satisfies the old tag declarations — so step 1 (signature + all call-site dances) leaves the whole tree green, and the per-package steps then flip declaration caps onto call sites that already supply ref. Every step compiles and passes its tests before the next; each step is one commit.

Step 1 — the viewpoint-adapted cpointer() + every call site it breaks

• packages/builtin/array.pony: fun cpointer(offset: USize = 0): this->Pointer[A].
• packages/builtin/string.pony: fun cpointer(offset: USize = 0): this->Pointer[U8].
• Docstrings: cpointer() documents that the returned pointer carries the receiver's capability and is the pointer to hand to C functions that write (mutable receiver required); cstring() documents its read-only contract and points writers at cpointer().
• Rewrite chop/unchop in String and Array through the private _ptr field (let start_ptr = _ptr is tag through the iso receiver — sendable, crosses into the existing recover; _unsafe()._offset(...) inside). String rewrite verified compiling; Array is isomorphic — verify here.
• Fix net/_test.pony:893, 900 (bind the pointer before the recover).
• Restructure every call site the signature change breaks (these are the write sites; the ref pointers they now produce still satisfy the old tag decls, so the tree stays green): builtin/stdin.pony (P-iso-local, count threaded out); files/file.pony read/read_string (P-iso-local, four paths); process/_pipe.pony _Pipe.read (its read_buf is an iso parameter — consume into the recover, thread len/errno out, rebuild the (buf, len, errno) returns outside; the several early-return branches all reshape the same way); net/tcp_connection.pony _queue_read/_pending_reads (P-iso-field on _read_buf); net/udp_socket.pony POSIX + Windows buffer sites (P-iso-local / P-iso-field; from continues to pass as today against its unchanged decl); net/ossocket.pony get_so (P-iso-local, getsockopt result threaded out; option_size stays an outer var — addressof on an outer var from inside a recover compiles, verified); the three tools' identical _find_exe_directory helpers in tools/pony-doc/main.pony, tools/pony-lint/main.pony, tools/pony-lsp/compiler_notify.pony (P-iso-local with the FFI's Bool threaded out; the String.from_array(consume buf) vs None branch stays outside the recover).
• packages/builtin_test/_test.pony: new test class \nodoc\ iso _TestCPointerViewpoint, registered in Main's TestList — declare a mutating FFI (e.g. @memset with a Pointer[U8] ref param), fill an Array and a String through cpointer() on ref receivers, assert contents; exercise P-iso-local for an iso buffer. Counterfactual-check each assertion (break it, confirm it fires) before calling the test done. No negative-compile tests: rejection of weak receivers is standard cap typing, not new compiler behavior.
• Verify: build/release/ponyc --pass=expr packages/stdlib, then make test-stdlib-release, then (tools were touched) make test-pony-compiler && make test-pony-lint && make test-pony-lsp && make test-pony-doc — deleting the tool test binaries first so the run is against fresh builds.

Step 2 — builtin decls

• stdin.pony: @pony_os_stdin_read(buffer: Pointer[U8] ref, ...) (site restructured in step 1).
• _to_string.pony: @snprintf/@_snprintf str param → Pointer[U8] ref; call sites switch s.cstring() → s.cpointer() (Divergence 4).
• std_stream.pony: pony_os_std_flush/print/write file param → UnsafePointer[U8] ref (concrete element type restores checking; matches the _stream field). Call sites unchanged — _stream is already effective ref.
• Verify: build/release/ponyc --pass=expr packages/stdlib, then make test-stdlib-release.

Step 3 — net

• Decl flips: pony_os_sockname/pony_os_peername/pony_os_getaddr ip/ipaddr → NetAddress ref; pony_os_recvfrom buffer → Pointer[U8] ref (from stays NetAddress tag — the carve-out, with a comment pointing at the problem section above); pony_os_recv buffer → Pointer[U8] ref; getsockopt option_value → Pointer[U8] ref; pony_os_nameinfo host/serv outer pointers → bare (effective ref); freeaddrinfo → UnsafePointer[_AddrInfo] ref.
• Introduce primitive _AddrInfo and unify the addrinfo handle (Divergence 9): @pony_os_addrinfo[UnsafePointer[_AddrInfo]], @pony_os_nextaddr[UnsafePointer[_AddrInfo]](addr: UnsafePointer[_AddrInfo] tag), @pony_os_getaddr(addr: UnsafePointer[_AddrInfo] tag, ...) (addr is read-only — tag stays), @freeaddrinfo(addr: UnsafePointer[_AddrInfo] ref).
• Call-site reworks: dns.pony _resolve (P-out for each NetAddress); tcp_listener/tcp_connection local_address/remote_address (P-out); udp_socket constructors embed _ip → let _ip: NetAddress via P-out (Divergence 3); udp POSIX read keeps the step-1 buffer shape, from keeps today's shape (the outer iso binding is sendable, so it is referenceable inside the buffer's recover, passes against the unchanged tag decl, and is consumed to received() as before); udp Windows path is P-iso-field on _read_buf only, _read_from untouched.
• Code comments at the two Windows IOCP sites noting pointer retention past the call.
• Verify: build/release/ponyc --pass=expr packages/stdlib, then make test-stdlib-release.

Step 4 — files

• file_info.pony: stat trio file param → FileInfo ref (P-this); mode field → let mode: FileMode = FileMode (Divergence 2; docstring updated).
• file.pony: @read/@_read buffer → Pointer[U8] ref (concrete type + honest cap; sites restructured in step 1).
• directory.pony: closedir/FindClose handle tag → ref; ponyint_unix_readdir dir → UnsafePointer[_DirectoryHandle] ref; FindNextFileA handle → UnsafePointer[_DirectoryHandle] ref.
• file_path.pony: @OpenProcessToken token_handle → Pointer[USize] (typed, effective ref — matches the addressof token site).
• Verify: build/release/ponyc --pass=expr packages/stdlib, then make test-stdlib-release.

Step 5 — process

• _process.pony: @waitpid stat_loc → bare Pointer[I32]; @ponyint_win_process_create error_msg → bare Pointer[UnsafePointer[U8] val] and the call site switches to String.copy_cstring (Divergence 6). The out-var types as UnsafePointer[U8] val (sendable) so the copy can happen inside a recover val, and the now-redundant recover message.clone() end goes away.
• _pipe.pony: @read buf → Pointer[U8] ref; @ReadFile buffer → Pointer[U8] ref (sites restructured in step 1; ReadFile here passes NULL overlapped — a synchronous read, no retained pointer); @ponyint_win_pipe_create near_fd/far_fd → bare Pointer[U32].
• Verify: build/release/ponyc --pass=expr packages/stdlib, then make test-stdlib-release.

Step 6 — format, debug, assert, json, iregex

• format/_format_float.pony: snprintf twins str → Pointer[U8] ref; sites switch cstring() → cpointer() (Divergence 4).
• The fprintf family one-touch fix in debug/debug.pony, assert/assert.pony, json/_mort.pony, iregex/_mort.pony: @pony_os_stdout/stderr returns → UnsafePointer[U8] (they're FILE*, foreign — a Round 1 leftover fixed where the cap fix already touches the line); @fprintf stream → UnsafePointer[U8] ref (fmt stays tag); debug.pony's _stream helper return type follows.
• Verify: build/release/ponyc --pass=expr packages/stdlib + make test-stdlib-release.

Step 7 — tool frontends

• tools/pony-doc/main.pony, tools/pony-lint/main.pony, tools/pony-lsp/compiler_notify.pony: @get_compiler_exe_directory output_path → Pointer[U8] ref (helpers restructured in step 1).
• tools/{pony-doc,pony-lint,pony-lsp}/_unreachable.pony: same fprintf/stderr one-touch fix as step 6.
• tools/pony-lsp/main.pony: @pony_os_stdout return → UnsafePointer[U8]; @_fileno stream → UnsafePointer[U8] tag (reads only — cap stays tag).
• Verify (delete the tool test binaries first and confirm the Built ...-tests line appears, so the run is against fresh builds): make test-pony-compiler && make test-pony-lint && make test-pony-lsp && make test-pony-doc.

Step 8 — example, documentation, release notes

• New example examples/ffi-buffer-fill/: a C function fills a Pony Array[U8] through cpointer(), demonstrating the honest-cap FFI declaration (Pointer[U8] ref), the viewpoint-adapted accessor, the P-out shape, and the P-iso-local shape — companion .c file and per-example README in the style of ffi-struct/; entry added to examples/README.md. make test-examples excludes ffi-* examples (they need the companion shared library), so it's verified by a manual gcc + ponyc build, documented in the example README like the other two.
• packages/builtin/asio_event.pony: docstring on AsioEventID documenting the exception — the runtime mutates event internals through this tag handle; tag is required because events are delivered via behaviors; synchronization is owned by the runtime; do not copy this pattern for ordinary foreign handles.
• Release notes (individual file under .release-notes/): the breaking change — cpointer() now returns a viewpoint-adapted pointer, with a before/after example for the iso-receiver pattern; the FileInfo.mode type change; the guidance change ("FFI declarations for C-written memory take ref; cstring() is for read-only use").

Step 9 — full validation

make build
make test-core
make test-stdlib-release
make test-examples
make test-pony-compiler && make test-pony-lint && make test-pony-lsp && make test-pony-doc

(the tool test binaries deleted first, as above). Windows-only branches (if windows declarations and sites in directory.pony, _pipe.pony, _process.pony, file_path.pony, _to_string.pony, the lsp main) can't be typechecked on Linux — the draft PR leans on Windows CI for those, and I expect at least one CI iteration there.

Step 10 — bookkeeping

• Issues for the out-of-scope findings: a capsicum CapRights0 class-layout fragility the audit turned up (C reads/writes 16 bytes through addressof _r0, relying on field adjacency in a class); File._pending_writev storing raw cpointer() results without anchoring the ByteSeq for GC across partial writes; the Windows LocalFree leak; the pony_compiler findings (including five C-signature mismatches and a source_contents() use-after-free hazard the audit caught incidentally) and its cap design round.
• A compiler issue: teach auto-receiver-recovery to accept arrow-typed returns whose adapted capability is sendable (an iso receiver calling the new cpointer() would get tag back, restoring direct read use on iso receivers). Its absence is what forces the chop/unchop rewrites and the iso read-site bindings.
• The lead-section gap is this discussion's open design question; whatever answer emerges — a language change for the synchronous case, or the owned-C shim in open question 4 — unblocks the recvfrom carve-out.

What stays imperfect, on purpose

• pony_os_recvfrom.from stays NetAddress tag — the one knowingly dishonest parameter left. One call fills two objects that must exit at different caps; today's language cannot express that call site honestly, and the in-language workarounds are patches. Tracking comment at the declaration points here.
• IOCP retention: on Windows, recv/recvfrom write through the passed pointers after the call returns until completion. No call-boundary cap expresses retention; ref makes the at-call claim honest. Noted in code comments and above.
• The escape hatches stay open: addressof, .usize() laundering, USize(0)-as-NULL through the void*-rule, and _unsafe() + from_cpointer adoption (which chop legitimately depends on for its sound-by-disjointness zero-copy split). All audited uses are authorized; closing the hatches is future language/lint work.
• The void*-rule still skips cap checks for any remaining [None] params (e.g. PeekNamedPipe's always-NULL buffer). Element types were tightened where C actually writes; always-NULL and read-only [None] params were left alone.

Open questions

1. The lead-section gap: is a language-level answer for the synchronous case worth pursuing, and what shape should it take? That's the big one, and it's why this is a discussion and not just a PR.
2. The asio exception (Divergence 7): the one place "C mutates ⇒ ref" knowingly doesn't apply, because sendability is load-bearing for event delivery. Better shapes welcome.
3. The pony_compiler deferral (Divergence 8): 33 findings that amount to "the AST wrapper's type story needs a redesign," punted to a dedicated round rather than cap-flipped onto a wrong architecture. If some mechanical subset should ride along now, make the case.
4. A runtime shim could remove the recvfrom carve-out without a language change. FFI can't return tuples (not ABI-safe across platforms), but it can return a struct with multiple items. pony_os_recvfrom is ours: a variant that allocates the buffer C-side with pony_alloc and returns {char* buf, size_t len} for Pony to adopt (the established pony_os_realpath/ponyint_unix_readdir idiom), with from consumed in as iso and handed back as a struct field (C already writes NetAddress through the existing ipaddress_t mirror — no new layout coupling), would make this round's only lie unnecessary. It costs one new runtime function and relaxes "no C changes" in exactly one spot we own. The same trick could replace the consume-dance at half a dozen more sites if taken further, at the price of more exported runtime surface — and it does nothing for the general problem, because the world's C still won't return our pointers. Worth it for the one function, or hold the line and let the carve-out stand until the language has an answer?

[SeanTAllen]

Working through the Windows IOCP read path step by step convinced me of a limit worth stating plainly: "honest caps" has a ceiling at the C boundary, and IOCP is the proof.

_queue_read on Windows calls pony_os_recv, which issues an overlapped WSARecv and returns immediately. The kernel writes into the buffer after the call returns, on its own schedule, and the byte count arrives later through the completion port as an asio event. Between enqueue and completion there are two parties with write access to the same memory: the kernel, holding a raw pointer no type system can see, and the actor, holding the buffer as iso, which entitles it to read and mutate that memory the whole time. Nothing but discipline keeps the actor's hands off. The buffer also has to stay alive for the kernel's whole window; today the iso field roots it incidentally. Drop the reference after enqueue and the GC will reclaim memory the kernel is still writing.

No capability can state this contract. ref says "mutates during the call." False: it mutates after. Consuming it in would say "gone forever." Also false, and it un-roots the buffer. The truthful contract has two halves: you may not touch this until the completion fires, and you must keep it alive until then. The first half is consume-shaped. The second half is the opposite of consume. And even if we wanted to express "ownership transfers at enqueue and comes back at completion," there is no carrier for the return trip: completions arrive as _event_notify(event, flags, arg), an event id and two U32s. No object can ride that.

The lesson I take from this: capabilities describe what Pony references may do. They don't describe C. When we say a parameter capability is "honest," the most we can actually mean is "this truthfully bounds what Pony can expect during the call." For synchronous, non-retaining C, which is nearly everything in the audit, that is a real and useful honesty, and it's what this plan delivers. For retained-pointer C like overlapped I/O, full honesty is unrepresentable. The window between return and completion is governed by the field that roots the buffer, the discipline that doesn't touch it, and the happens-before edge the completion message provides when it finally arrives. That's the same safety story every IOCP program in C or C# has. It's contained: users of TCPConnection and UDPSocket never see the hole. But it's containment by convention, the same trust class as _unsafe() inside builtin, not safety by construction.

So the plan's treatment of these sites is the most-true cap plus a written admission. The buffer parameters go to ref, which is fully honest for the POSIX path sharing the same declaration and true-but-incomplete for Windows, and a comment at the enqueue sites names the retention window. Leaving them tag because full honesty is out of reach anyway would be less honest, not more: it throws away the call-window truth on POSIX, where tag is exactly the miscompile class from #4925, and it claims sendability on memory the kernel is actively writing.

The only way to turn the discipline into enforcement is to take the buffer away from Pony for the window: the runtime owns and roots in-flight buffers, and completions come back through a path that can carry an object. That's a redesign of how asio notifications work. Possible. Its own round, if anyone ever wants it.

[SeanTAllen]

Following up on the IOCP ceiling above: there are two ways out of it, and both are proven in other ecosystems.

1. Avoid completions for sockets entirely: talk to AFD and get readiness, like epoll. WinSock is a layer over the Auxiliary Function Driver, and IOCTL_AFD_POLL against \Device\Afd delivers readiness notifications — the epoll model, on Windows. That's how wepoll works, how libuv's poll watchers work, and what Rust's mio switched to; tokio on Windows hasn't used IOCP-as-completions for socket data in years. Under readiness, every read is a synchronous nonblocking call into our buffer. The kernel never holds a pointer past a call's return, the retention window ceases to exist, and the same honest caps that cover POSIX cover Windows with no exception to document. Our Windows asio backend would work the way epoll and kqueue backends already do instead of being the odd one out. The cost: AFD is undocumented NT surface (load-bearing for years across wepoll, libuv, and mio, but undocumented) and it's a rewrite of the iocp backend. Honestly, this might work better for us all around.

2. Keep completions and make the ownership transfer real: the kernel gives us back the buffer. This is the contract IOCP actually has — ownership leaves at enqueue and should come back at completion. Expressed honestly: enqueue consumes the buffer (iso parameter), the runtime owns and roots it while it's in flight, and the completion hands it back. Rust's io_uring runtimes landed on exactly this shape after concluding that borrowed buffers can't be made sound against completion-based I/O — tokio-uring's read takes a Vec<u8> by value and the completion returns (result, buffer). For us it needs the two pieces the runtime doesn't have today: a completion path that can carry an object back (_event_notify carries an event id and two U32s) and runtime rooting of in-flight buffers. A real asio redesign, but it turns today's discipline into enforcement.

The first option removes the problem. The second makes the problem's true semantics expressible. Either one retires the "contained unsafety" admission for these sites.