feat: cgo Go bindings

[terylt]

Summary

Complete CPEX Rust plugin runtime with Go SDK bindings via cgo. Plugins are written in Rust for memory safety and security guarantees; Go applications embed the runtime through a clean SDK that handles MessagePack serialization and
opaque handle management.

Rust Core (crates/cpex-core, cpex-sdk)

• Plugin trait, typed hooks, 5-phase executor (sequential, transform, audit, concurrent, fire-and-forget)
• Plugin manager with YAML config, factory registry, and route resolution
• 11 extension types with mutability tiers: immutable (Arc), monotonic (MonotonicSet), guarded (WriteToken)
• Capability-gated extension filtering per plugin
• COW extensions: Extensions.cow_copy() → OwnedExtensions for zero-copy reads, clone-on-write modifications
• CMF message types: Message, 12 ContentPart variants, domain objects (ToolCall, Resource, PromptRequest, media
  sources)
• MessageView for zero-copy policy evaluation
• 184 unit tests

C FFI (crates/cpex-ffi)

• extern "C" surface: manager lifecycle, hook invocation, background tasks, memory management
• Payload type registry (u8 discriminator) for efficient deserialization
• Per-manager tokio runtime for async execution from synchronous cgo calls
• cpex_load_config for split creation/config-loading flow

Go SDK (go/cpex)

• PluginManager: lifecycle, RegisterFactories(callback), LoadConfig, Initialize, InvokeByName, Shutdown
• Invoke[P](): typed generic invoke with automatic payload/extension deserialization
• Full extension types matching Rust (Meta, Security, HTTP, Delegation, Agent, Request, MCP, Completion, Provenance,
  LLM, Framework)
• CMF types: Message, MessagePayload, all 12 ContentPart variants with custom msgpack marshaling
• 25 tests covering lifecycle, invocation, context threading, CMF round-trips

Go Demo Examples (examples/go-demo)

• Generic demo: 3 plugins (identity checker, PII guard, audit logger) with routing rules and policy groups
• CMF demo: 2 plugins (tool policy, header injector) with typed messages, security labels, and capability-gated
  header injection
• Separate cpex-demo-ffi Rust crate keeps demo plugins out of the core FFI library
• Factory registration via RegisterFactories callback pattern

Closes: #18

[araujof]

Thanks for this PR, @terylt !

Here are a few auto-reviewed findings that we can discuss and address.

P0 — Critical

#	File	Issue	C
1	crates/cpex-ffi/src/lib.rs:343	Aliased &mut via concurrent goroutines — cpex_invoke calls mgr.as_mut() creating &mut CpexManagerInner. Two goroutines calling InvokeByName concurrently produce overlapping &mut references to the same allocation, violating Rust's aliasing invariant. LLVM may miscompile based on noalias guarantees. Fix: change to *const and use mgr.as_ref() since PluginManager.invoke_by_name takes &self.	90
2	crates/cpex-ffi/src/lib.rs:394	Panic unwinds across FFI boundary — No catch_unwind wraps any block_on call. A plugin panic propagates through extern "C", which is undefined behavior (process abort on Rust 1.71+, stack corruption on earlier versions). Applies to lines 249, 270, 394, 478. Fix: wrap each block_on in `std::panic::catch_unwind(AssertUnwindSafe(

---

P1 — High

#	File	Issue	C
3	go/cpex/manager.go:361	Wait() ignores cpex_wait_background return code — On failure (-1), output pointers are uninitialized. C.GoBytes(nil, 0) is UB/panic. Fix: check rc != 0 before reading output pointers; return a sentinel error.	100
4	go/cpex/manager.go:101	Finalizer + Shutdown() race -> double-free — No synchronization between GC finalizer goroutine and explicit Shutdown(). Both can see handle != nil concurrently. runtime.SetFinalizer is not cleared in Shutdown(). Fix: use sync.Mutex or atomic.Pointer for handle access; call runtime.SetFinalizer(mgr, nil) inside Shutdown().	100
5	go/cpex/manager.go:293	BackgroundTasks holds stale manager pointer after Shutdown() — bg.mgr is copied from m.handle at invoke time. After Shutdown() frees the Rust object, bg.Wait() dereferences freed memory. Fix: store a *PluginManager reference instead of copying the raw C handle, so Wait() can check mgr.handle == nil.	85
6	crates/cpex-ffi/src/lib.rs:394	No timeout on block_on at FFI boundary — A plugin that blocks an OS thread (not yields) hangs the calling Go goroutine indefinitely. Per-plugin tokio timeouts only work for cooperative async. Fix: wrap block_on body in tokio::time::timeout(Duration::from_secs(WALL_CLOCK_LIMIT), ...).	100
7	go/cpex/manager.go:33	Duplicate CGO declarations in ffi.go and manager.go — All 12 C function signatures declared in both files. A signature change requires coordinated edits or the build silently accepts a stale prototype. Fix: remove the CGO comment block from manager.go; CGO resolves declarations package-wide from ffi.go.	100
8	go/cpex/ffi.go:16	CFLAGS -I references non-existent include/ directory — No header file exists at crates/cpex-ffi/include/. Misleads contributors and masks the absence of a generated C header. Fix: remove the -I line or add cbindgen to generate a header.	100

---

P2 — Moderate

#	File	Issue	C
9	crates/cpex-ffi/src/lib.rs:112	alloc_bytes UB on zero-length data — std::alloc::alloc with size=0 Layout is undefined. Fix: add if data.is_empty() { return (ptr::null_mut(), 0); }	75
10	crates/cpex-ffi/src/lib.rs:119	Integer truncation data.len() as c_int — Payloads >2GB cause wrap to negative, corrupting the free length. Fix: add bounds check before cast.	75
11	crates/cpex-ffi/src/lib.rs:467	cpex_wait_background leaks bg_handle when mgr is NULL — Returns -1 without consuming the handle, but Go nils it. Fix: consume bg_handle before returning -1 on null mgr.	75
12	go/cpex/manager.go:242	Context table irrecoverably lost on invoke failure — Handle nil'd before FFI call; if Rust fails after consuming, the state is gone. Fix: only nil the handle after confirming rc == 0, or have Rust write context_table_out even on partial failure.	80
13	go/cpex/cmf.go:356-579	11 decode helpers silently drop fields — decodeVideoSource/decodeAudioSource miss DurationMs; decodeResourceRef misses RangeStart/RangeEnd; decodeResource misses Blob/SizeBytes; decodePromptResult drops Messages entirely. Fix: replace manual helpers with re-encode-and-unmarshal pattern, or add missing field extractions.	100
14	go/cpex/types.go:182	Go CompletionExtension missing raw_format, created_at — Rust serializes both; Go silently drops them. Fix: add the fields with matching msgpack tags.	100
15	go/cpex/types.go:156	Go ToolMetadata missing output_schema, annotations — Silent data loss on Rust->Go. Fix: add the fields.	100
16	go/cpex/types.go:130	Go AgentExtension missing conversation field — ConversationContext not exposed in Go SDK. Fix: add ConversationContext struct and field.	100
17	go/cpex/cmf.go:159	DecodeMsgpack no default case — Unknown content_type variants decode to empty ContentPart; re-encoding loses the data. Fix: add default case that preserves raw content or returns an error.	100
18	go/cpex/manager.go:274	Generic "invoke failed" error with no diagnostic — Six distinct Rust failure paths all produce the same opaque Go error string. Fix: thread a char** error_msg_out through the FFI, or use discriminating integer error codes.	100

---

P3 — Low

#	File	Issue	C
19	go/cpex/cmf.go:523	decodeVideoSource/decodeAudioSource miss duration_ms — Add if ms, ok := m["duration_ms"].(uint64); ok { s.DurationMs = &ms } to both.	100
20	go/cpex/types.go:135	AgentExtension.Turn type mismatch: Go *int vs Rust Option<u32> — Go int is 64-bit; values >2^32 would overflow Rust. Fix: change to *uint32.	75
21	go/cpex/cmf.go:204	Constructor functions shadow ContentPart struct field names — ToolCallContent is both a field and a function. Fix: rename constructors to NewToolCallPart, etc.	75
22	crates/cpex-ffi/src/lib.rs:153	Hardcoded worker_threads(4) with no caller override — Each manager creates 4 OS threads regardless of workload. Fix: accept thread count as a parameter or use current_thread runtime.	75

---

Learnings & Past Solutions

No docs/solutions/ directory exists. Key patterns identified from source:

1. Payload type registry — uint8 discriminator requires 5-touch-point sync across Rust/Go
2. Per-manager tokio runtime — block_on from CGO is the only safe async bridge
3. Consumed-handle convention — nil Go handle immediately after FFI call
4. serde_bytes_opt — Required for Vec<u8> fields in MessagePack structs
5. ContentPart tagged union — Custom encoder/decoder required for Rust serde compatibility

---

Agent-Native Gaps

• All lifecycle errors collapse to identical opaque strings — agents cannot classify failures
• on_error: ignore/disable swallows execution errors silently; PipelineResult has no Errors field
• BackgroundTasks.Wait() returns []string instead of structured []PluginError
• Plugin introspection methods (plugin_names(), is_initialized()) not exposed through FFI
• PluginError type defined in types.go but never wired into any result struct

---

Residual risks:

• Go<->Rust MessagePack wire format not verified end-to-end for complex nested types
• tokio::runtime::Runtime created per manager; 100 managers = 400 OS threads with no cap
• Go GC pointer pinning during CGO calls relies on correct unsafe.Pointer usage patterns
• serialize_payload failure is silent — Go receives "no modification" when Rust fails to re-serialize

Testing gaps:

• No concurrent stress test (multiple goroutines + -race flag)
• No cross-language MessagePack round-trip verification (Go encode -> Rust decode and vice versa)
• No test for panic propagation through FFI boundary
• No test exercising modified-payload/modified-extensions deserialization (all tests use zero-plugin manager)
• No FFI crate unit tests at all (cpex-ffi/src/lib.rs has 0 #[cfg(test)])
• No test for BackgroundTasks.Wait() when cpex_wait_background returns -1
• No test for operations-after-shutdown (nil handle guard paths)
• No test for invalid payload_type values
• No round-trip tests for video, audio, prompt_request, prompt_result, or resource_ref content parts

---

Recommendations

Fix order:

1. P0#1 — Change cpex_invoke to take *const and use mgr.as_ref() (PluginManager.invoke_by_name takes &self)
2. P0#2 — Wrap all block_on calls in std::panic::catch_unwind
3. P1#3 — Check cpex_wait_background return code before reading output pointers
4. P1#4 — Add sync.Mutex or atomic.Pointer to PluginManager; clear finalizer in Shutdown()
5. P1#5 — Store *PluginManager reference in BackgroundTasks instead of raw C handle copy
6. P1#6 — Add wall-clock timeout to block_on calls
7. P1#7-8 — Consolidate CGO declarations into one file; remove stale -I path
8. P2 findings — Address field gaps, decode helpers, and error propagation

[araujof]

LGTM

Nice work, @terylt !

Fully Addressed

#	Finding	Status
P0#1	Aliased &mut via concurrent goroutines	Fixed — all calls now use mgr.as_ref() (immutable). Comment at L655 explicitly documents the refactor.
P0#2	Panic unwinds across FFI boundary	Fixed — catch_unwind(AssertUnwindSafe(...)) wraps all block_on calls. Dedicated ErrCpexPanic sentinel.
P1#3	Wait() ignores return code	Fixed — checks rc != 0 before reading output pointers, returns errorFromRC.
P1#4	Finalizer + Shutdown race / double-free	Fixed — sync.RWMutex guards handle access; Shutdown() takes write lock and calls runtime.SetFinalizer(m, nil).
P1#5	BackgroundTasks holds stale pointer	Fixed — stores *PluginManager reference, takes mgr.mu.RLock() in Wait(), checks mgr.handle == nil.
P1#6	No timeout on block_on	Fixed — FFI_WALL_CLOCK_TIMEOUT (60s) via tokio::time::timeout wraps every async call. Dedicated ErrCpexTimeout.
P1#8	Stale -I include/ path	Fixed — removed from ffi.go.
P2#9	alloc_bytes UB on zero-length	Fixed — early return (ptr::null_mut(), 0) for empty data.
P2#10	Integer truncation len as c_int	Fixed — bounds check against c_int::MAX before cast.
P2#11	cpex_wait_background leaks bg_handle on null mgr	Fixed — drop(Box::from_raw(bg_handle)) on the null-mgr path.
P2#12	Context table lost on invoke failure	Fixed — handle only nil'd after confirming rc == 0. Comment explains the one unavoidable edge case (post-invoke serialization OOM).
P2#13	Decode helpers silently drop fields	Fixed — uses decodeAs[T] generic pattern with msgpack struct tags; DurationMs added to VideoSource/AudioSource.
P2#14	CompletionExtension missing fields	Fixed — RawFormat and CreatedAt fields added.
P2#15	ToolMetadata missing fields	Fixed — OutputSchema and Annotations added.
P2#16	AgentExtension missing conversation	Fixed — ConversationContext struct and field added.
P2#17	DecodeMsgpack no default case	Fixed — default preserves rawMap for round-trip fidelity.
P2#18	Generic "invoke failed" error	Fixed — errors.go defines 7 sentinel errors; errorFromRC maps each FFI code to a typed, errors.Is-compatible error.
P3#19	Missing duration_ms in video/audio	Fixed — DurationMs *uint64 in both structs.
P3#20	Turn type mismatch (*int vs u32)	Fixed — changed to *uint32.
P3#22	Hardcoded worker_threads(4)	Fixed — ConfigureRuntime(workerThreads int) exposes caller control.

NOT Addressed (with explanation)

#	Finding	Status
P1#7	Duplicate CGO declarations	Kept intentionally — comment in manager.go:43-56 explains that CGO does NOT merge declarations across files' preambles (the reviewer's suggestion was tested and doesn't work). Documented as a known limitation with future remediation paths (consolidate to single file or use cbindgen header).
P3#21	Constructor names shadow field names	Not addressed — no rename of ToolCallContent() etc. to NewToolCallPart(). Low priority; likely a deliberate style choice.