Native Ready Reference Foundation

Native-Ready Reference Foundation

“Native-ready” describes the clarity of the current reference behavior. It does not mean that a native processor exists or that one is committed to the current roadmap.

Current state

Python remains:

• the implemented AISMixer runtime;
• the behavioural reference implementation;
• the sole production data-plane processor implementation;
• the owner of the current data-plane semantics; and
• the basis for regression and future differential testing.

The cumulative progression is:

• Campaign A — behavioural semantics;
• Campaign B — explicit state and limits;
• Campaign C — immutable and parse-once representation;
• Campaign D — a synchronous processor and explicit runtime boundary; and
• Campaign E — compiled numeric routing, exact output bytes, ordered OutputBatch results, and unified numeric egress.

Campaign C established immutable IngressFrame transport, explicit PayloadTextMode, bytes-native scanning into immutable half-open spans, parse-once ParsedSentence metadata, and production entry through feed_parsed_outcome(). Campaign D then established the stable Python-side DataPlaneProcessor contract, PythonDataPlaneProcessor as its sole production and reference implementation, explicit ingress fan-in, processor, and egress stages, an ordered completion barrier, and process-local task supervision. Campaign E made the processor snapshot target-only, compiled operator-facing target names to dense numeric egress IDs, made OutputBatch the public ordered result, and unified production dispatch through Forwarder.send_to_ids().

That production entry shares one lifecycle implementation with the supported legacy string feed_outcome() API. The Python assembler still materializes and stores sentence strings. Each emitted sentence now crosses the processor boundary as exact immutable bytes after one UTF-8 encoding. Together, the five campaigns reduce the behavior that future work would have to infer from incidental code structure.

Implemented Python-side processor boundary

core/data_plane.py defines immutable ProcessingSnapshot, ProcessorOutput, and OutputBatch values, DeduplicationMode, and the synchronous DataPlaneProcessor.process(frame, snapshot) protocol. One call accepts one immutable ingress frame and one immutable target-only processing snapshot and returns one ordered OutputBatch.

The snapshot contains only the routing generation, deduplication mode, and already resolved numeric target IDs. It contains no routing table, compiled routing program, queue, transport, or runtime object. Each ProcessorOutput contains the exact immutable bytes for one completely formatted sentence and its explicit ordered numeric target IDs; an empty target tuple is valid.

The processor contract contains no asyncio task ownership and has no socket, queue, transport, forwarder, multiprocessing, IPC, native binding, or private completion acknowledgement. The runtime processor stage acquires the snapshot and invokes the processor; the egress stage asynchronously dispatches the returned batch in order through Forwarder.send_to_ids(). A non-empty OutputBatch crosses a private completion barrier, and the processor stage does not process the next ingress frame until that batch's egress dispatch succeeds; this barrier remains runtime orchestration, not part of the processor contract.

PythonDataPlaneProcessor is synchronous and process-local. It owns the assembler, deduplicator, multipart TAG contexts, and processing configuration, but it does not own the runtime stages, routing-state replacement, network transports, or task supervision.

Current comparison surface

The tested Python behaviour exposes observable reference points that a future second implementation can compare. The stable processor contract now provides the primary Python-side call and result boundary:

Surface	Examples
Contract	One IngressFrame and target-only ProcessingSnapshot to one ordered OutputBatch
Output	Exact immutable sentence bytes, explicit numeric targets, order, and outbound TAG metadata
Representation	Direct frames, valid legacy-event adaptation, scanner spans, and parsed metadata
Routing	One compiled numeric target match per accepted frame, with the resolved IDs frozen into the snapshot
Deduplication	Global or target-scoped accept/suppress decisions
Assembly	Lifecycle status, ordered completion, and discarded keys
Metadata	Selected TAG s, minimum valid multipart c, and preserved or generated g
No-output behavior	Invalid input, pending fragments, conflicts, no-route results, and duplicate suppression
State inspection	Contract-relevant immutable statistics snapshots

The canonical behavioural contract defines the exact Python reference semantics.

These are current reference and regression surfaces. No second processor implementation exists, so differential native execution and cross-implementation test results remain future work.

Staged evolution

Campaign A: behavioural semantics
        |
Campaign B: explicit state and limits
        |
Campaign C: immutable and parse-once representation
        |
Campaign D: synchronous processor and explicit runtime boundary
        |
Campaign E: numeric routing, byte output, and unified egress
        |
Campaign F: worker readiness within the current process
        |
later process architecture and later native implementation

Campaign F covers bounded queues and backpressure, per-instance processor-state ownership, lifecycle and reset semantics, routing-snapshot handoff semantics, and a metrics boundary. It does not add actual worker or coordinator processes, multiprocessing, IPC, cross-process supervision, native bindings, or an ABI.

Only later process-architecture work may introduce coordinator, ingress-worker, or egress-worker processes and their IPC, supervision, distribution, restart, and recovery policies. A separate later native track may place another processor behind the established contracts and add differential-conformance execution. The stable Python-side contract does not itself select or implement a native API, ABI, binding strategy, or processor.

Current non-claims

• No native processor exists.
• No native processor API or ABI has been defined.
• No C or C++ API has been selected.
• No binding or foreign-function technology has been selected.
• No differential native tests or differential native execution currently run.
• The Python assembler still materializes and stores sentence strings even though ProcessorOutput.message is exact immutable bytes.
• End-to-end zero-copy processing is not claimed.
• No benchmark, throughput, latency, memory, or other performance result follows from Campaign C, D, or E.
• No multiprocessing coordinator or worker-process architecture is implemented.
• No IPC or cross-process routing-snapshot distribution is implemented.
• No compatibility promise extends beyond the documented and tested behavioural surface.

Why the preparation matters

Explicit boundaries allow implementation work to be incremental:

• State can remain with its current owner until a deliberate boundary is introduced.
• A candidate implementation can be checked behavior-by-behavior rather than replacing the complete service at once.
• Routing, assembly, deduplication, metadata, and secure state have testable lifecycle edges.
• Injected clocks and deterministic generators make time- and ID-sensitive comparisons reproducible.
• Immutable statistics provide point-in-time state observations without mutating the system under test.
• Any future process separation would require separately specified ownership, cross-process supervision, restart/recovery, routing-snapshot distribution, and IPC boundaries. Campaign F prepares worker-facing boundaries but does not implement that separation.

The preparation is therefore an architectural option, not a feature claim.

Related pages

• Architecture Overview
• State, Lifecycle, and Limits
• Behavioural Contract
• Architecture Evolution
• Roadmap