Close some of the remaining accepted first-release gaps for FFI - #32
Merged
Conversation
Close the architectural gap around native declarations, ABI types,\nlink inputs, GC transitions, callbacks, and generated bindings.\n\nKeep native calls productive without permitting raw linker flags, shell\nexecution, runtime symbol lookup, reflection, or untracked managed\npointers.
Represent ADR 0081 native inputs as closed manifest types and build a\ndeterministic platform-specific link plan. Reject unsafe paths, missing\nhashes, raw linker fields, shell text, invalid provider shapes, and\nconflicting aliases before the driver reaches a linker.
Preserve namespace-targeted attributes before the file-scoped namespace as\nrequired by ADR 0023 and ADR 0081. This unblocks typed native link\nattachments without changing declaration grammar or assigning semantic\nmeaning during parsing.
Validate namespace-targeted UDAs through the normal compile-time attribute\npipeline and retain their typed values by Module. This supplies the\nsemantic attachment point required by FFI link aliases without adding\nnamespace reflection or a synthetic namespace symbol.
Create the independently installed Pop.Ffi Package and reserve its Ffi and\nFfi.C namespace ownership. Keep the unsafe interoperability surface out\nof Pop.Standard and the prelude while making its build identity available\nto the compiler workspace.
Give Pop.Ffi ABI types and compiler-recognized attributes stable,\nqualified identities outside the prelude. Validate the complete closed set so\nuser declarations and spelling reuse cannot acquire trusted FFI semantics.
Keep Pop.Ffi ABI types unavailable until Package resolution selects the\nexact direct Pop.Ffi Bubble. Preserve qualified-only lookup and carry the\nverified dependency through the front-end and normal Package check path.
Recognize the exact dependency-gated FFI attributes before body checking and\nturn them into closed ABI, symbol, link-alias, and effect records. Reject\nbodies, generics, managed ABI values, invalid attributes, and dynamic-like\npointer representations with a structured runtime-safety diagnostic.
Represent foreign functions as verified bodyless declarations instead of empty\nPop bodies. Preserve their exact ABI, symbol, aliases, and mandatory effects\nthrough HIR, canonical MIR, optimization, text round trips, and public\nreference metadata.
Specify exact root publication, mutator states, and single-use tokens for native calls. This closes the PLRI gap that otherwise left backend and runtime balancing behavior undefined.
Advance the stable native ABI to 1.13 and retain exact roots across blocking calls. Reject unbound, stale, mismatched, and reused transition identities before managed execution resumes.
Require native entry and callback adapters to establish explicit managed mutator authority. This prevents argument allocation and foreign root publication from running without a collector-visible binding.
Advance the stable native ABI to 1.14 with explicit thread-bound attach and detach authority. Reject duplicate, stale, wrong-thread, and active-transition detach attempts.
Relocation heap validation already checks whether each segmented object\ntable is empty, but ObjectTable lacked the corresponding query. Add the\nconstant-time helper so collector and dependent compiler suites build\nwithout local compatibility patches.
The compilation benchmark reconstructed the Linux target without its\nrequired precise-stack-map capability, causing LLVM profile negotiation to\nreject an otherwise supported target. Reuse the canonical target inventory\nso benchmark coverage follows the accepted target contract.
Represent foreign calls explicitly in canonical MIR so backends consume a\nresolved foreign identity, exact transition roots, effects, and unwind\ncontract. Reject forged identities and mismatched publications during MIR\nverification, preserve the operation through optimization, and make the MIR\ninterpreter fail closed when no typed adapter is installed.\n\nLower supported C and system declarations through exact LLVM signatures and\nbalance every call with native enter/leave transitions. Attach generated\nprogram entry to managed runtime state, keep FFI scalars out of precise GC\nroots, and reload writable roots before managed execution resumes. Native\nlink-and-run and forced-relocation tests cover the complete successful path.
Verify package-relative native files by exact hash and reject symlinked or conflicting inputs before invoking the linker. Resolve manifest-owned system and package-configuration providers into typed arguments without shell fragments. Carry canonical target plans through .poplib artifacts, merge transitive package plans, and require every Ffi.Link alias to resolve. Cover direct archives, transitive libraries, default libc binding, target rejection, and tamper detection with executable fixtures.
Persist canonical provider identities, discovered versions, emitted library names, target constraints, and local hashes in .poplib manifests. Exclude ambient host search paths and reject provider facts that do not match the corresponding native requirement. Validate package-configuration output as typed arguments, reject raw flags and relative search paths, and cover direct, transitive, and artifact round trips.
Close the arbitrary managed-value pin gap by separating immutable Bytes payload borrows, explicitly owned ABI buffers, and generation-checked managed handles. Foreign pointers never expose a Pop object address. Specify read-only pointer nullability, lexical borrow verification, deterministic buffer cleanup, fixed-layout record marshalling, and canonical target ABI fingerprints before implementing the public intrinsics.
Give mutable, read-only, optional, and non-optional foreign pointers separate stable Pop.Ffi identities. Keep Ffi.Buffer outside the direct ABI mapping and continue rejecting managed pointer element types. Lower both read-only pointer forms through the exact target pointer ABI and exercise them against linked native functions without integer, nil, or dynamic conversion.
Add the exact backend-neutral resolve-root operation required by Ffi.Handle.get and assign its native entry to ABI 1.15. Retain and release continue to own handle creation and deterministic close. Require invalid, stale, forged, zero, and closed handles to fail before a managed reference reaches generated code.
Expose the current managed reference behind a live strong-root handle so FFI handle access remains correct after collector relocation. Reject forged, stale, and released handles at the runtime boundary instead of exposing an unchecked token.\n\nAdd the ABI 1.15 resolve-root operation and keep the reference runtime, collectors, native runtime, and LLVM declarations on the same semantic contract.
Public Ffi.Handle values can cross Pop and foreign call boundaries, unlike compiler-private lexical root tokens. Define exact typed open, get, and close operations so MIR verification and every backend preserve that distinction.\n\nRequire each operation to validate its PLRI result before managed execution continues.
Keep public Ffi.Handle values distinct from compiler-private lexical root tokens. Verify exact managed payload types and preserve open, get, and close through MIR text and optimization.\n\nRoute the operations through the MIR interpreter and LLVM using the same PLRI retain, resolve, and release contract. Trap on every zero or failed runtime result before managed execution continues.
Recognize Ffi.Handle open, get, and close only when the reserved Pop.Ffi dependency is verified and no user declaration owns the resolved call. Keep inferred and explicit payload types exact and reject scalar or foreign-ABI payloads before HIR.\n\nPreserve the operations as typed HIR and lower them to the canonical backend-neutral MIR handle contract. Treat open as a contextual qualified name without weakening identifier parsing elsewhere.
Assign the missing expected-error identities and define Ffi.Buffer as a relocating managed resource with separately owned ABI storage. Keep allocation exhaustion distinct from compiler and runtime invariants.\n\nSpecify ABI 1.16 status codes, output atomicity, layout validation, bounds checks, lexical borrow tokens, and idempotent close before any backend implementation can diverge.
Reserve the accepted Pop.Ffi bootstrap identities for NullPointerError and AllocationError. Keep both types outside the foreign ABI scalar vocabulary so they remain typed Pop expected errors rather than native status values.
Define typed PLRI requests and results for exact layout, allocation, byte access, lexical borrowing, and deterministic close. Implement deterministic reference storage with zeroing, checked geometry and bounds, output atomicity, and lifecycle failures.
Advance native ABI 1 to minor version 16 and give every accepted buffer operation one unique runtime symbol. Keep ABI negotiation explicit so backends cannot lower buffer storage through an unversioned fallback.
Close the remaining buffer index-base gap by using Pop's one-based collection convention across source, PLRI, and the native ABI. Reject zero and values above the element length before touching storage or outputs.
Make the accepted native target own C scalar size, alignment, and signedness together with pointer geometry. Consume that contract from LLVM so MIR layout planning and native calls cannot diverge through backend-local C data models.
Require an accepted native target when constructing canonical FFI layouts. Validate C integer, pointer, function-pointer, and handle geometry through target-owned ABI facts so a target label cannot masquerade as a verified layout plan.
Attach an exact target-selected FFI layout catalog to every MIR Bubble. Keep empty catalogs valid for targets without a native C ABI, while rejecting layout entries there, and expose target-aware HIR lowering so BPF MIR cannot inherit native geometry.
Add the seven backend-neutral buffer operations, exact target-layout and operand verification, and typed lexical borrow-region proofs. Execute the verified contract through the MIR interpreter with typed allocation results, backend-private borrow generations, and field-plan marshalling for every accepted scalar and record value class.
Consume the target-verified MIR layout catalog when lowering owned FFI buffers. Check every native status, preserve typed allocation failure, keep borrow generations private, and marshal scalars and records without copying managed object storage.\n\nReject catalog/backend target mismatches so LLVM cannot silently recompute a foreign layout for a different target.
Bind compact FfiAbiLayoutId values to full canonical SHA-256 layout descriptors. Retain full artifact facts and fail closed on zero or compact collisions so source lowering cannot depend on TypeId allocation, declaration order, or backend layout guesses.\n\nSpecify how trusted Ffi.C.Layout attachments and source Ffi.Buffer operations enter the target-selected HIR and MIR catalog.
Build target- and ABI-specific canonical descriptors for scalar, pointer, handle, function-pointer, and nested record storage. Derive compact execution keys from artifact-owned SHA-256 fingerprints and reject malformed, zero, mixed-ABI, or colliding identities.\n\nKeep SHA-256 in compiler-driver artifact ownership while MIR validates descriptor facts and remaps construction-local graph keys before verification.
Recognize the closed Ffi.Buffer allocation and element operations only for a verified direct Pop.Ffi dependency. Preserve them as typed HIR operations and reject them from compile-time execution.\n\nBuild the target layout catalog with artifact-owned fingerprints before MIR lowering, then attach the canonical layout identity and geometry to every buffer instruction. Route normal driver compilation through the fingerprinted lowering path and reject unsupported types, mismatched operands, user spelling collisions, and implicit fingerprint fallback.
Recognize the exact Ffi.C.Layout compiler attribute only for a direct Pop.Ffi dependency and retain its resolved record identity through typed expressions and HIR. Reject unmarked, spoofed, empty, recursive, or managed-field layouts before publication.\n\nConstruct target ABI record geometry recursively from verified field identities and feed the ordered field plan into the canonical fingerprinted MIR catalog. This keeps normal Pop records separate from foreign storage while enabling typed Ffi.Buffer record elements.
Represent null construction, mutability weakening, optional pointer\nconstruction, and presence testing as exact typed HIR and canonical MIR\noperations. Verify constructor direction and preserve trusted record layout\nidentity without introducing integer or managed-address conversions.\n\nExecute the same contract in the MIR interpreter and LLVM backend, and\ncover dependency gating, arity, spelling collisions, and backend behavior.
Lower optional mutable and read-only pointer extraction to an exact\nbackend-neutral Result operation. Preserve the pointer element and mutability\nconstructor, return Ffi.NullPointerError for null, and reject reversed or\nmis-arity calls.\n\nCover MIR text round trips plus present and absent execution in the MIR\ninterpreter and deterministic native result construction in LLVM.
Represent typed load, store, pointer arithmetic, copying, and address conversion as verified backend-neutral HIR and MIR operations. This keeps unsafe access statically typed and requires canonical ABI layouts before backend lowering. Lower checked operations in LLVM and the MIR interpreter, with alignment, overflow, provenance, and overlap checks where the backend can prove them. Reject unsupported BPF use and cover source typing, MIR verification, native IR assembly, and deterministic interpreter execution.
Give the accepted but incomplete FFI surface an explicit catalog status. This keeps the public library inventory machine-checkable without claiming that callbacks, scoped pins, or generated bindings are stable.
Close the missing lifetime and runtime contract for scoped FFI borrows. Restrict each body to an immediate synchronous closure and preserve its provenance through a canonical MIR BorrowRegionId call. Define additive native ABI 1.17 byte-payload borrow operations so runtimes own payload-address knowledge instead of backends.
Represent Ffi.Buffer.withPointer as one verified scoped operation from typed source through HIR and canonical MIR. Preserve the borrow region, inline body identity, exact captures, and balanced normal and unwind cleanup without allocating a general closure environment.\n\nTeach LLVM and the MIR interpreter to execute the same direct nested-call plan, and reject escaping, asynchronous, nested, or corrupt borrow bodies before backend lowering.
Advance native ABI 1 to version 1.17 and add the closed PLRI byte-payload borrow contract. Store immutable packed bytes as collector-owned payload data so generated code never derives an object-header offset or observes mutable storage.\n\nPin the exact owner before publishing an address, keep borrow identities distinct from ordinary pins, and reject stale, forged, duplicate, or wrong-owner cleanup. Preserve failure-atomic native outputs and the payload address across forced moving and stable-generational collection.
Implement the accepted immutable Bytes pin contract across source typing, HIR, canonical MIR, the MIR interpreter, and LLVM. Keep the managed owner, borrow region, token, address, and length tied to one verified lexical scope so native code cannot observe object layout or retain a stale pointer.\n\nRegister cleanup immediately after borrowing and require the exact owner on length and end operations. Reject escaping, asynchronous, nested, and corrupt borrow plans before backend lowering.
Lower CUnwind declarations only for exception-capable targets and use an LLVM cleanup landing pad around each foreign transition. Always leave the exact transition, restore writable roots on unwind, run the verified MIR cleanup path, and prevent a native exception payload from entering managed Pop code.
Bind trusted FFI attributes by exact identity, require public foreign functions to live in a final Unsafe namespace, and validate the closed ABI mapping recursively. Reject managed pointer targets, invalid callback packs, scalar handles, layout defaults, and nested invalid layout fields before HIR publication.
Encode exact public foreign identities and effects in reference metadata, reject corrupted contracts, and propagate wrapper effects transitively. Import referenced foreign declarations into canonical consumer MIR so direct cross-Bubble calls still use verified foreign transitions instead of bypassing root publication through an ordinary reference call.
Collaborator
Author
|
@cubic-dev-ai review this pull request |
Contributor
@kleeedolinux I have started the AI code review. It will take a few minutes to complete. |
Contributor
|
We've triggered an ultrareview automatically — This PR implements the accepted FFI architecture (ADR 0081) and proof-directed static reclamation (ADR 0085), touching ABI contracts, memory management, HIR/MIR structure, and the compilation pipeline across 160 files — a missed bug in these cross-cutting changes could break data integrity, FFI saf. I'll post findings when complete. An ultrareview is cubic's deepest review, catching hard-to-find bugs in the most critical PRs. It runs a longer, multi-pass analysis using cubic's most capable review models, and typically takes around 30 minutes. It consumes your team's reviewed-lines allowance at 3× the standard rate. Automated ultrareviews are disabled by default. We triggered this run as part of your trial. Want cubic to do this for every high-risk PR? Enable auto-ultrareview in your settings. |
Contributor
|
This PR is large and would use a significant portion of your monthly review quota. Comment |
Collaborator
Author
|
@cubic-dev-ai review this |
Contributor
@kleeedolinux I have started the AI code review. It will take a few minutes to complete. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
30 issues found across 160 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="architecture/08.1-closed-design-questions.md">
<violation number="1" location="architecture/08.1-closed-design-questions.md:581">
P2: This summary incorrectly says every operation after `open` uses one-based indices, although only `read` and `write` are indexed. Narrowing the statement avoids making the binding closed-design document contradict ADR 0082's `borrow` and `close` APIs.</violation>
</file>
<file name="architecture/19-architecture-conformance-and-regression-policy.md">
<violation number="1" location="architecture/19-architecture-conformance-and-regression-policy.md:232">
P2: FFI conformance can pass this minimum matrix without covering the MIR interpreter, despite the accepted FFI ADRs requiring interpreter capability and differential behavior. Including the MIR interpreter among owning components keeps this binding policy aligned with those requirements.</violation>
<violation number="2" location="architecture/19-architecture-conformance-and-regression-policy.md:249">
P2: This minimum matrix leaves reference metadata/artifacts unowned even though static reclamation soundness across Bubbles depends on serialized retention summaries and conservative `MayRetain` fallback. Adding metadata/artifacts and summary round-trip proofs would preserve ADR 0085's separate-compilation boundary.</violation>
</file>
<file name="architecture/decisions/0082-ffi-abi-storage-and-lexical-borrows.md">
<violation number="1" location="architecture/decisions/0082-ffi-abi-storage-and-lexical-borrows.md:159">
P2: A `Ffi.Handle` dropped without `close` can retain its managed value indefinitely because this contract excludes finalizers but defines no replacement lifetime check. The handle lifecycle should specify deterministic missing-close enforcement and corresponding every-exit/leak conformance tests, as the buffer lifecycle does.</violation>
</file>
<file name="crates/compiler/backends/mir-interp/src/runtime.rs">
<violation number="1" location="crates/compiler/backends/mir-interp/src/runtime.rs:331">
P2: Closing large or repeated FFI buffers retains all backing memory for the adapter's lifetime because `Vec::clear()` does not release capacity. Replacing the vector drops the allocation while preserving idempotent-close state.</violation>
</file>
<file name="architecture/decisions/0081-statically-bound-native-ffi.md">
<violation number="1" location="architecture/decisions/0081-statically-bound-native-ffi.md:82">
P1: Hashing a shared library only at link time does not ensure that file is loaded at runtime; common ELF outputs record its SONAME, which the loader can resolve from `LD_LIBRARY_PATH` or system caches. Define authenticated deployment/runtime resolution for `shared` entries, or exclude them from the hash-backed stable plan.</violation>
</file>
<file name="crates/runtime/native/src/ffi_buffer/mod.rs">
<violation number="1" location="crates/runtime/native/src/ffi_buffer/mod.rs:50">
P1: Every nonempty FFI buffer is accessed and deallocated through a pointer reconstructed after `addr()` discarded its provenance, making those operations undefined under Rust's Strict Provenance model. Preserve exposed provenance when converting the allocation pointer to the integer stored by `AlignedStorage`.</violation>
</file>
<file name="crates/compiler/types/src/call_checking.rs">
<violation number="1" location="crates/compiler/types/src/call_checking.rs:3008">
P1: Borrow-safety validation can be bypassed for method calls inside `Ffi.Buffer.withPointer`/`Ffi.withPin` closures. Because unhandled expression kinds default to `true`, method-call variants are accepted without checking whether the borrowed pointer is passed through their receiver/arguments; recursively validating those variants would keep the non-escaping guarantee intact.</violation>
</file>
<file name="crates/compiler/backends/mir-interp/src/values.rs">
<violation number="1" location="crates/compiler/backends/mir-interp/src/values.rs:112">
P1: Relocating GC leaves returned `FfiBuffer` and `Bytes` values carrying their pre-relocation references, so passing one into a later interpreter call can access an invalid managed object. `install_relocated_reference` should update the embedded reference for these two variants just as it does for `Class`.</violation>
</file>
<file name="crates/runtime/collector/src/generational/ffi_bytes.rs">
<violation number="1" location="crates/runtime/collector/src/generational/ffi_bytes.rs:35">
P1: Large immutable-byte allocations bypass the collector's byte hard limit and can abort the process inside `Arc::from` instead of returning the promised allocation failure. The payload size needs admission/accounting for its full lifetime, including releasing that charge when the object is reclaimed.</violation>
</file>
<file name="crates/runtime/native/src/binding.rs">
<violation number="1" location="crates/runtime/native/src/binding.rs:75">
P2: A failed detach can consume the native execution binding while leaving the attachment token live, so retrying the same detach always fails and a later attach can overwrite stale authority. The leave/unregister operations and TLS cleanup should be failure-atomic, or the native binding should be restored when cleanup fails.</violation>
</file>
<file name="crates/compiler/mir/src/ir.rs">
<violation number="1" location="crates/compiler/mir/src/ir.rs:160">
P1: FFI MIR dumps no longer round-trip a complete `MirBubble`: parsing drops all layout entries and replaces the target with x86_64, so layout-dependent MIR cannot verify or be consumed from the advertised interchange text. Consider serializing/deserializing `MirFfiLayoutCatalog` as part of the dump rather than requiring out-of-band `with_ffi_layouts` repair.</violation>
</file>
<file name="crates/compiler/backends/mir-interp/src/interpreter.rs">
<violation number="1" location="crates/compiler/backends/mir-interp/src/interpreter.rs:177">
P1: Handles returned from one interpreter call cannot be used by a later call: every `MirInterpreter::call` discards the handle registry and starts with an empty map. Since public `Ffi.Handle<T>` values are specified to cross calls, the registry needs interpreter/runtime lifetime rather than per-`Engine` lifetime.</violation>
<violation number="2" location="crates/compiler/backends/mir-interp/src/interpreter.rs:1778">
P1: Opening a handle for valid managed payloads such as `String` fails with `TypeMismatch`, because those interpreter values have no `RuntimeValue.reference` even though the MIR verifier accepts them as managed handle payloads. The interpreter needs managed-reference representations for every verifier-accepted `T`, or handle verification must match the backend's supported representations.</violation>
<violation number="3" location="crates/compiler/backends/mir-interp/src/interpreter.rs:1789">
P1: `FfiHandleGet` can return stale collection contents after the original array/list/table is mutated, because `FfiHandleOpen` stores a detached `RuntimeValue` clone outside the alias-update loop. Handle resolution should recover the live value by managed identity or share the payload representation instead of snapshotting it.</violation>
<violation number="4" location="crates/compiler/backends/mir-interp/src/interpreter.rs:2248">
P1: A forged `FfiHandle` token can release a compiler-private root before being rejected, because `release_root` runs before membership in `ffi_handles` is validated. Validate and remove the public-handle entry first, then release its root token so invalid handles fail without changing runtime state.</violation>
</file>
<file name="crates/compiler/mir/src/ffi_layout/mod.rs">
<violation number="1" location="crates/compiler/mir/src/ffi_layout/mod.rs:44">
P2: A catalog with arbitrary record `FieldId`s is accepted but later fails LLVM/interpreter marshalling or associates data with the wrong field. Catalog validation should derive or verify each ID against the record declaration rather than trusting this constructor argument.</violation>
<violation number="2" location="crates/compiler/mir/src/ffi_layout/mod.rs:201">
P1: Malformed catalogs can pass validation with scalar alignment or record offsets that disagree with the target C ABI, causing `Ffi.Buffer` marshalling to use the wrong layout. Validate primitive alignment and recompute each record's exact target offsets, alignment, and final size before canonicalization.</violation>
</file>
<file name="crates/compiler/backends/llvm/src/instruction_lowering.rs">
<violation number="1" location="crates/compiler/backends/llvm/src/instruction_lowering.rs:790">
P1: A scoped body that may unwind bypasses the registered `FfiBufferEndBorrow`/`FfiBytesEndBorrow` cleanup because this lowering always emits `call` and ignores `MirUnwindAction`. The may-unwind case should use an `invoke` to the MIR cleanup target (and ensure the containing function has a personality), preserving the borrow cleanup contract.</violation>
</file>
<file name="crates/compiler/hir/src/ir.rs">
<violation number="1" location="crates/compiler/hir/src/ir.rs:254">
P1: A public function whose FFI call is reached through `HirCallDispatch::DirectMethod` exports an under-reported effect summary, since this fixed point ignores both method bodies and method call edges. Mirror the MIR callable-effect computation across functions and methods before emitting reference metadata.</violation>
<violation number="2" location="crates/compiler/hir/src/ir.rs:255">
P1: Public wrappers that directly use `Ffi.Buffer`, handles, borrows, or unsafe-pointer operations publish incomplete effect metadata because `recompute_call_effects` only adds callee effects. A dependent Bubble can therefore lower `CallReferenced` without required `GcSafePoint`, `UnsafeMemory`, or trap effects; include intrinsic HIR-expression effects in this fixed-point computation.</violation>
</file>
<file name="crates/compiler/hir/src/verification.rs">
<violation number="1" location="crates/compiler/hir/src/verification.rs:2949">
P1: A forged or transformed `FfiBufferWithPointer` can claim any record symbol as layout metadata, allowing a non-C-layout record to pass complete-bubble verification. Validate `layout_record` against `schema.records`, as the adjacent open/none operations do.</violation>
</file>
<file name="crates/compiler/backends/mir-interp/src/ffi_buffer.rs">
<violation number="1" location="crates/compiler/backends/mir-interp/src/ffi_buffer.rs:145">
P1: Non-null `Ffi.Pointer` and `Ffi.Function` values can become `Nil` when native storage contains zero because unmarshalling ignores the concrete constructor in `layout.element()`. Preserve zero only for the optional constructors and reject/trap zero for non-null layouts; apply the same distinction when marshalling `MirValue::Nil`.</violation>
</file>
<file name="crates/compiler/types/src/lib.rs">
<violation number="1" location="crates/compiler/types/src/lib.rs:365">
P2: Foreign calls can trap when the runtime rejects either boundary transition, but their inferred effect summary omits `MayTrap`; wrappers therefore advertise a weaker effect contract than the generated LLVM behavior. Include `Effect::MayTrap` in `expected_effects`.</violation>
</file>
<file name="crates/runtime/native/src/ffi_buffer/operations.rs">
<violation number="1" location="crates/runtime/native/src/ffi_buffer/operations.rs:74">
P2: Reading into storage that overlaps this buffer can trigger undefined behavior even though the documented caller contract permits it. `std::ptr::copy` would preserve behavior for generated non-overlapping callers while safely handling overlap.</violation>
<violation number="2" location="crates/runtime/native/src/ffi_buffer/operations.rs:119">
P2: Writing from an overlapping buffer-derived pointer can trigger undefined behavior despite satisfying the documented readable-pointer contract. Using overlap-safe `std::ptr::copy` avoids imposing an undocumented precondition.</violation>
</file>
<file name="crates/compiler/backends/llvm/src/ffi_buffer/marshalling.rs">
<violation number="1" location="crates/compiler/backends/llvm/src/ffi_buffer/marshalling.rs:98">
P1: Reading a record now allocates a managed object despite both `FfiBufferRead` and `FfiUnsafeLoad` lacking the `Allocates` effect. This bypasses MIR's effect contract for record reads; classify these operations as allocating when their layout is a record, or use a nonallocating representation.</violation>
<violation number="2" location="crates/compiler/backends/llvm/src/ffi_buffer/marshalling.rs:98">
P1: Nested records are stored in scalar slots, so the GC will not trace or relocate the child record referenced by the outer record. Build `reference_slots` from fields whose child layout is `MirFfiValueClass::Record` before allocating the result object.</violation>
</file>
<file name="crates/compiler/driver/src/native_link.rs">
<violation number="1" location="crates/compiler/driver/src/native_link.rs:221">
P2: Cross-target Linux builds can resolve package-configuration data from the host environment, yielding linker inputs for the wrong platform. This happens because `resolve_native_link_inputs` gates only on `OperatingSystem::Linux` and does not reject/handle non-host Linux targets before calling `pkg-config`.</violation>
</file>
<file name="crates/compiler/backends/llvm/src/ffi_buffer/mod.rs">
<violation number="1" location="crates/compiler/backends/llvm/src/ffi_buffer/mod.rs:254">
P2: Repeated buffer operations inside a loop can grow the stack on every iteration and eventually overflow it because these scratch `alloca`s are emitted in the operation's block. Hoisting reusable scratch slots to the function entry (or explicitly restoring stack state) would keep stack usage bounded.</violation>
</file>
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
| Codec = { kind = "archive", path = "native/libcodec.a", sha256 = "<lowercase SHA-256>" } | ||
|
|
||
| [platform."x86_64-unknown-linux-gnu".nativeLibraries] | ||
| PlatformCodec = { kind = "shared", path = "native/libplatformCodec.so", sha256 = "<lowercase SHA-256>" } |
Contributor
There was a problem hiding this comment.
P1: Hashing a shared library only at link time does not ensure that file is loaded at runtime; common ELF outputs record its SONAME, which the loader can resolve from LD_LIBRARY_PATH or system caches. Define authenticated deployment/runtime resolution for shared entries, or exclude them from the hash-backed stable plan.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At architecture/decisions/0081-statically-bound-native-ffi.md, line 82:
<comment>Hashing a shared library only at link time does not ensure that file is loaded at runtime; common ELF outputs record its SONAME, which the loader can resolve from `LD_LIBRARY_PATH` or system caches. Define authenticated deployment/runtime resolution for `shared` entries, or exclude them from the hash-backed stable plan.</comment>
<file context>
@@ -0,0 +1,359 @@
+Codec = { kind = "archive", path = "native/libcodec.a", sha256 = "<lowercase SHA-256>" }
+
+[platform."x86_64-unknown-linux-gnu".nativeLibraries]
+PlatformCodec = { kind = "shared", path = "native/libplatformCodec.so", sha256 = "<lowercase SHA-256>" }
+```
+
</file context>
Contributor
There was a problem hiding this comment.
I've fixed the P1 issue in ADR 0081 by adding a paragraph that defines authenticated runtime resolution for shared libraries.
What was fixed:
The ADR now specifies that kind = "shared" entries require the linker to emit RPATH/RUNPATH entries pointing to the package-relative directory containing the verified library. This ensures the dynamic loader resolves the hashed artifact rather than an arbitrary SONAME match from LD_LIBRARY_PATH or system caches.
Key additions:
- RPATH entries are ordered before system search paths
- Alternative approaches are documented (static linking or archive form)
- Runtime resolution is authenticated as part of the hash-backed build plan
The change addresses the security concern that hashing at link time alone doesn't guarantee the correct file is loaded at runtime.
✅ Documentation-only change, no tests required
diff --git a/architecture/decisions/0081-statically-bound-native-ffi.md b/architecture/decisions/0081-statically-bound-native-ffi.md
index c524a83..61c4627 100644
--- a/architecture/decisions/0081-statically-bound-native-ffi.md
+++ b/architecture/decisions/0081-statically-bound-native-ffi.md
@@ -107,6 +107,15 @@ identity and reject conflicts before linking. Shared-library deployment remains
an explicit package/install concern; a Pop executable does not search arbitrary
runtime paths.
+Shared library entries (`kind = "shared"`) require authenticated runtime
+resolution. The linker emits absolute RPATH or RUNPATH entries pointing to the
+package-relative directory containing the verified shared library, ensuring the
+dynamic loader resolves the hashed artifact rather than an arbitrary SONAME
+match from system paths. RPATH entries are ordered before system search paths
+and `LD_LIBRARY_PATH`. Alternatively, shared libraries may be statically linked
+or converted to archive form where runtime resolution is a deployment concern
+outside the hash-backed build plan.
+
This ADR covers static binding performed by `pop check`, `pop build`, and
`pop run`. Runtime `dlopen`/`LoadLibrary`, `dlsym`/`GetProcAddress`, and calls
through symbols obtained from runtime strings are not part of the stable FFI.
Files changed:
architecture/decisions/0081-statically-bound-native-ffi.md— Added authenticated runtime resolution requirements for shared libraries
| return Err(OPEN_ALLOCATION_FAILURE); | ||
| } | ||
| Ok(Some(Self { | ||
| address: pointer.addr(), |
Contributor
There was a problem hiding this comment.
P1: Every nonempty FFI buffer is accessed and deallocated through a pointer reconstructed after addr() discarded its provenance, making those operations undefined under Rust's Strict Provenance model. Preserve exposed provenance when converting the allocation pointer to the integer stored by AlignedStorage.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/runtime/native/src/ffi_buffer/mod.rs, line 50:
<comment>Every nonempty FFI buffer is accessed and deallocated through a pointer reconstructed after `addr()` discarded its provenance, making those operations undefined under Rust's Strict Provenance model. Preserve exposed provenance when converting the allocation pointer to the integer stored by `AlignedStorage`.</comment>
<file context>
@@ -0,0 +1,263 @@
+ return Err(OPEN_ALLOCATION_FAILURE);
+ }
+ Ok(Some(Self {
+ address: pointer.addr(),
+ layout,
+ }))
</file context>
| | TypedExpressionKind::TaskCancelToken { .. } | ||
| | TypedExpressionKind::TaskCancel { .. } | ||
| | TypedExpressionKind::Await { .. } => false, | ||
| _ => true, |
Contributor
There was a problem hiding this comment.
P1: Borrow-safety validation can be bypassed for method calls inside Ffi.Buffer.withPointer/Ffi.withPin closures. Because unhandled expression kinds default to true, method-call variants are accepted without checking whether the borrowed pointer is passed through their receiver/arguments; recursively validating those variants would keep the non-escaping guarantee intact.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/compiler/types/src/call_checking.rs, line 3008:
<comment>Borrow-safety validation can be bypassed for method calls inside `Ffi.Buffer.withPointer`/`Ffi.withPin` closures. Because unhandled expression kinds default to `true`, method-call variants are accepted without checking whether the borrowed pointer is passed through their receiver/arguments; recursively validating those variants would keep the non-escaping guarantee intact.</comment>
<file context>
@@ -2026,6 +2859,178 @@ impl<'resolver, 'index> BodyChecker<'resolver, 'index> {
+ | TypedExpressionKind::TaskCancelToken { .. }
+ | TypedExpressionKind::TaskCancel { .. }
+ | TypedExpressionKind::Await { .. } => false,
+ _ => true,
+ }
+}
</file context>
Suggested change
| _ => true, | |
| TypedExpressionKind::DirectMethodCall { | |
| receiver, | |
| arguments, | |
| .. | |
| } => { | |
| receiver.as_deref().map_or(true, |value| { | |
| scoped_borrow_expression_is_valid(value, pointer, false, signatures) | |
| }) && arguments.iter().all(|argument| { | |
| scoped_borrow_expression_is_valid(argument, pointer, false, signatures) | |
| }) | |
| } | |
| TypedExpressionKind::InterfaceMethodCall { | |
| receiver, | |
| arguments, | |
| .. | |
| } | |
| | TypedExpressionKind::BuiltinInterfaceMethodCall { | |
| receiver, | |
| arguments, | |
| .. | |
| } => { | |
| scoped_borrow_expression_is_valid(receiver, pointer, false, signatures) | |
| && arguments.iter().all(|argument| { | |
| scoped_borrow_expression_is_valid(argument, pointer, false, signatures) | |
| }) | |
| } | |
| _ => true, |
| pub(crate) fn visible(visible: MirValue) -> Self { | ||
| let reference = match &visible { | ||
| MirValue::Class(class) => Some(class.reference), | ||
| MirValue::FfiBuffer(reference) | MirValue::Bytes(reference) => Some(*reference), |
Contributor
There was a problem hiding this comment.
P1: Relocating GC leaves returned FfiBuffer and Bytes values carrying their pre-relocation references, so passing one into a later interpreter call can access an invalid managed object. install_relocated_reference should update the embedded reference for these two variants just as it does for Class.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/compiler/backends/mir-interp/src/values.rs, line 112:
<comment>Relocating GC leaves returned `FfiBuffer` and `Bytes` values carrying their pre-relocation references, so passing one into a later interpreter call can access an invalid managed object. `install_relocated_reference` should update the embedded reference for these two variants just as it does for `Class`.</comment>
<file context>
@@ -102,6 +109,7 @@ impl RuntimeValue {
pub(crate) fn visible(visible: MirValue) -> Self {
let reference = match &visible {
MirValue::Class(class) => Some(class.reference),
+ MirValue::FfiBuffer(reference) | MirValue::Bytes(reference) => Some(*reference),
_ => None,
};
</file context>
| { | ||
| return Err(RuntimeFailure::runtime_invariant()); | ||
| } | ||
| allocation.immutable_bytes = Some(std::sync::Arc::<[u8]>::from(bytes)); |
Contributor
There was a problem hiding this comment.
P1: Large immutable-byte allocations bypass the collector's byte hard limit and can abort the process inside Arc::from instead of returning the promised allocation failure. The payload size needs admission/accounting for its full lifetime, including releasing that charge when the object is reclaimed.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/runtime/collector/src/generational/ffi_bytes.rs, line 35:
<comment>Large immutable-byte allocations bypass the collector's byte hard limit and can abort the process inside `Arc::from` instead of returning the promised allocation failure. The payload size needs admission/accounting for its full lifetime, including releasing that charge when the object is reclaimed.</comment>
<file context>
@@ -0,0 +1,105 @@
+ {
+ return Err(RuntimeFailure::runtime_invariant());
+ }
+ allocation.immutable_bytes = Some(std::sync::Arc::<[u8]>::from(bytes));
+ Ok(reference)
+ }
</file context>
| let mut effects = EffectSummary::empty() | ||
| .with(Effect::ForeignFunction) | ||
| .with(Effect::UnsafeMemory) | ||
| .with(Effect::GcSafePoint); |
Contributor
There was a problem hiding this comment.
P2: Foreign calls can trap when the runtime rejects either boundary transition, but their inferred effect summary omits MayTrap; wrappers therefore advertise a weaker effect contract than the generated LLVM behavior. Include Effect::MayTrap in expected_effects.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/compiler/types/src/lib.rs, line 365:
<comment>Foreign calls can trap when the runtime rejects either boundary transition, but their inferred effect summary omits `MayTrap`; wrappers therefore advertise a weaker effect contract than the generated LLVM behavior. Include `Effect::MayTrap` in `expected_effects`.</comment>
<file context>
@@ -310,6 +316,103 @@ pub enum Effect {
+ let mut effects = EffectSummary::empty()
+ .with(Effect::ForeignFunction)
+ .with(Effect::UnsafeMemory)
+ .with(Effect::GcSafePoint);
+ if !nonblocking {
+ effects = effects.with(Effect::Blocks);
</file context>
Suggested change
| .with(Effect::GcSafePoint); | |
| .with(Effect::GcSafePoint) | |
| .with(Effect::MayTrap); |
| // SAFETY: Bounds were checked against the exact allocation geometry, and | ||
| // the caller contract provides an equally sized writable output. | ||
| unsafe { | ||
| storage |
Contributor
There was a problem hiding this comment.
P2: Reading into storage that overlaps this buffer can trigger undefined behavior even though the documented caller contract permits it. std::ptr::copy would preserve behavior for generated non-overlapping callers while safely handling overlap.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/runtime/native/src/ffi_buffer/operations.rs, line 74:
<comment>Reading into storage that overlaps this buffer can trigger undefined behavior even though the documented caller contract permits it. `std::ptr::copy` would preserve behavior for generated non-overlapping callers while safely handling overlap.</comment>
<file context>
@@ -0,0 +1,232 @@
+ // SAFETY: Bounds were checked against the exact allocation geometry, and
+ // the caller contract provides an equally sized writable output.
+ unsafe {
+ storage
+ .pointer()
+ .add(offset)
</file context>
| }; | ||
| // SAFETY: Bounds were checked against the exact allocation geometry, and | ||
| // the caller contract provides an equally sized readable input. | ||
| unsafe { element.copy_to_nonoverlapping(storage.pointer().add(offset), element_size) }; |
Contributor
There was a problem hiding this comment.
P2: Writing from an overlapping buffer-derived pointer can trigger undefined behavior despite satisfying the documented readable-pointer contract. Using overlap-safe std::ptr::copy avoids imposing an undocumented precondition.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/runtime/native/src/ffi_buffer/operations.rs, line 119:
<comment>Writing from an overlapping buffer-derived pointer can trigger undefined behavior despite satisfying the documented readable-pointer contract. Using overlap-safe `std::ptr::copy` avoids imposing an undocumented precondition.</comment>
<file context>
@@ -0,0 +1,232 @@
+ };
+ // SAFETY: Bounds were checked against the exact allocation geometry, and
+ // the caller contract provides an equally sized readable input.
+ unsafe { element.copy_to_nonoverlapping(storage.pointer().add(offset), element_size) };
+ SUCCESS
+}
</file context>
Suggested change
| unsafe { element.copy_to_nonoverlapping(storage.pointer().add(offset), element_size) }; | |
| unsafe { std::ptr::copy(element, storage.pointer().add(offset), element_size) }; |
| sources: &[NativeLinkPlanSource], | ||
| target: &TargetSpec, | ||
| ) -> Result<NativeLinkResolution, NativeLinkResolutionError> { | ||
| if target.operating_system() != OperatingSystem::Linux |
Contributor
There was a problem hiding this comment.
P2: Cross-target Linux builds can resolve package-configuration data from the host environment, yielding linker inputs for the wrong platform. This happens because resolve_native_link_inputs gates only on OperatingSystem::Linux and does not reject/handle non-host Linux targets before calling pkg-config.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/compiler/driver/src/native_link.rs, line 221:
<comment>Cross-target Linux builds can resolve package-configuration data from the host environment, yielding linker inputs for the wrong platform. This happens because `resolve_native_link_inputs` gates only on `OperatingSystem::Linux` and does not reject/handle non-host Linux targets before calling `pkg-config`.</comment>
<file context>
@@ -0,0 +1,540 @@
+ sources: &[NativeLinkPlanSource],
+ target: &TargetSpec,
+) -> Result<NativeLinkResolution, NativeLinkResolutionError> {
+ if target.operating_system() != OperatingSystem::Linux
+ && sources
+ .iter()
</file context>
| @@ -0,0 +1,311 @@ | |||
| use std::collections::BTreeMap; | |||
Contributor
There was a problem hiding this comment.
P2: Repeated buffer operations inside a loop can grow the stack on every iteration and eventually overflow it because these scratch allocas are emitted in the operation's block. Hoisting reusable scratch slots to the function entry (or explicitly restoring stack state) would keep stack usage bounded.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/compiler/backends/llvm/src/ffi_buffer/mod.rs, line 254:
<comment>Repeated buffer operations inside a loop can grow the stack on every iteration and eventually overflow it because these scratch `alloca`s are emitted in the operation's block. Hoisting reusable scratch slots to the function entry (or explicitly restoring stack state) would keep stack usage bounded.</comment>
<file context>
@@ -0,0 +1,311 @@
+ let length = format!("%ffi_buffer_region_{region}_length");
+ let generation = format!("%ffi_buffer_region_{region}_generation");
+ let mut lines = vec
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Close some of the remaining accepted first-release gaps for FFI, view lifetimes,
checked casts, effects, and generated typed metadata adapters before exposing
those surfaces as stable.
Summary by cubic
Introduces statically bound native FFI with exact ABI layouts, deterministic native link plans, balanced runtime transitions, and safe buffers/borrows/handles across HIR, MIR, LLVM, and the interpreter. Adds a target-verified FFI layout catalog with SHA-256 fingerprints and advances the native ABI (1.13–1.17) while laying the groundwork for proof-directed static memory management.
New Features
@Ffi.Link+@Ffi.Foreignwith validated ABI/effects, optional@Ffi.Nonblocking, andCUnwind; balanced enter/leave transitions and managed thread attach/detach.Ffi.Handle(open/get/close), ownedFfi.Buffer<T>(open/read/write/borrow/close), and scoped immutableBytesborrows.NativeLinkPlan; no raw flags, shell, or ambient paths;.poplibschema v3 records provider facts and hashes.POP5000for invalid foreign contracts.Migration
Pop.Ffito use FFI types and attributes.@Ffi.Link("Alias")and foreign functions with@Ffi.Foreign("symbol")in anUnsafenamespace; remove raw linker flags.[nativeLibraries]entries in the manifest per target with SHA-256 for package files; resolve via the typed native-link workflow..poplibartifacts (schema v3) before publishing.Written for commit adda95d. Summary will update on new commits.