Native Ready Reference Foundation

Native-Ready Reference Foundation

“Native-ready” describes the clarity of the current reference behavior. It does not mean that a native processor exists or that one is committed to the current roadmap.

Current state

Python remains:

• the implemented aismixer runtime;
• the behavioural reference implementation;
• the sole production data-plane processor implementation;
• the owner of the current data-plane semantics; and
• the basis for regression and future differential testing.

The cumulative progression is:

• Campaign A — behavioural semantics;
• Campaign B — explicit state and limits;
• Campaign C — immutable and parse-once representation;
• Campaign D — a synchronous processor and explicit runtime boundary;
• Campaign E — compiled numeric routing, exact output bytes, ordered OutputBatch results, and unified numeric egress; and
• Campaign F — bounded and observable Worker Readiness within one process.

Campaign C established immutable IngressFrame transport, explicit PayloadTextMode, bytes-native scanning into immutable half-open spans, parse-once ParsedSentence metadata, and production entry through feed_parsed_outcome(). Campaign D then established the stable Python-side DataPlaneProcessor contract, PythonDataPlaneProcessor as its sole production and reference implementation, explicit ingress fan-in, processor, and egress stages, an ordered completion barrier, and process-local task supervision. Campaign E made the processor snapshot target-only, compiled operator-facing target names to dense numeric egress IDs, made OutputBatch the public ordered result, and unified production dispatch through Forwarder.send_to_ids().

Campaign F then added private bounded ingress queues, bounded processing admission, the bounded egress handoff, capacity-safe snapshot binding, processor-instance reset ownership, pull-based runtime statistics, and the interactive operator shell. It preserved the sole Python processor and the single-process runtime.

That production entry shares one lifecycle implementation with the supported legacy string feed_outcome() API. The Python assembler still materializes and stores sentence strings. Each emitted sentence now crosses the processor boundary as exact immutable bytes after one UTF-8 encoding. Together, the six campaigns reduce the behavior that future work would have to infer from incidental code structure.

What native-ready means

For v0.2.0, native-ready means the Python reference now has:

• explicit immutable ingress, snapshot, work-item, and output boundaries;
• an explicit synchronous processor contract;
• explicit mutable-state ownership and deterministic reset semantics;
• compiled numeric target routing and an exact bytes output boundary;
• bounded queue and handoff semantics with observable backpressure; and
• differential-testable Python reference behavior.

It does not mean that C or C++ code, a native processor, bindings, an ABI, worker processes, or measured performance improvements exist.

Implemented Python-side processor boundary

core/data_plane.py defines immutable ProcessingSnapshot, ProcessingWorkItem, ProcessorOutput, and OutputBatch values, DeduplicationMode, and the synchronous DataPlaneProcessor.process(frame, snapshot) protocol. One call accepts one immutable ingress frame and one immutable target-only processing snapshot and returns one ordered OutputBatch.

The snapshot contains only the routing generation, deduplication mode, and already resolved numeric target IDs. It contains no routing table, compiled routing program, queue, transport, or runtime object. Each ProcessorOutput contains the exact immutable bytes for one completely formatted sentence and its explicit ordered numeric target IDs; an empty target tuple is valid.

The processor contract contains no asyncio task ownership and has no socket, queue, transport, forwarder, multiprocessing, IPC, native binding, or private completion acknowledgement. Fan-in processing admission acquires the snapshot after capacity becomes available and binds it with the exact frame into one immutable ProcessingWorkItem; the serial processor stage then invokes the processor. A frame still waiting for capacity may see a later routing update, while an admitted work item stays fixed.

The egress stage asynchronously dispatches the returned batch in order through Forwarder.send_to_ids(). A non-empty OutputBatch crosses a bounded one-batch egress handoff and private completion barrier, and the processor stage does not process the next admitted work item until that batch's local egress dispatch succeeds. Later work may already be admitted and bound. This barrier remains runtime orchestration, not part of the processor contract or a remote-delivery acknowledgement.

PythonDataPlaneProcessor is synchronous and process-local. One runtime owns one long-lived instance, serially consumed by one processor stage. It owns the assembler, deduplicator, SourceState, multipart TAG contexts, processing configuration/helpers, and processor metrics, but it does not own the runtime stages, routing-state replacement, network transports, or task supervision. Its synchronous reset clears retained processing state in a defined, fail-fast, non-transactional order while retaining cumulative metrics. Reset is not an aismixerctl command.

Current comparison surface

The tested Python behaviour exposes observable reference points that a future second implementation can compare. The stable processor contract now provides the primary Python-side call and result boundary:

Surface	Examples
Contract	One IngressFrame and target-only ProcessingSnapshot to one ordered OutputBatch
Output	Exact immutable sentence bytes, explicit numeric targets, order, and outbound TAG metadata
Representation	Direct frames, valid legacy-event adaptation, scanner spans, and parsed metadata
Routing	One compiled numeric target match at processing admission, with the resolved IDs frozen into the work item
Deduplication	Global or target-scoped accept/suppress decisions
Assembly	Lifecycle status, ordered completion, and discarded keys
Metadata	Selected TAG s, minimum valid multipart c, and preserved or generated g
No-output behavior	Invalid input, pending fragments, conflicts, no-route results, and duplicate suppression
State inspection	Contract-relevant immutable statistics snapshots

The canonical behavioural contract defines the exact Python reference semantics.

These are current reference and regression surfaces. No second processor implementation exists, so differential native execution and cross-implementation test results remain future work.

Staged evolution

Campaign A: behavioural semantics
        |
Campaign B: explicit state and limits
        |
Campaign C: immutable and parse-once representation
        |
Campaign D: synchronous processor and explicit runtime boundary
        |
Campaign E: numeric routing, byte output, and unified egress
        |
Campaign F: bounded, observable worker readiness within the current process
        |
later process architecture and later native implementation

Campaign F completed bounded queues and backpressure, per-instance processor-state ownership, lifecycle and reset semantics, admission-time routing-snapshot handoff, runtime traffic accounting, and immutable pull-based metrics. It did not add actual worker or coordinator processes, multiprocessing, IPC, cross-process supervision, distributed metrics, native bindings, or an ABI.

Only later process-architecture work may introduce coordinator, ingress-worker, processor-worker, or egress-worker processes and their IPC, supervision, distribution, restart, and recovery policies. A separate later native track may place another processor behind the established contracts and add differential-conformance execution. The stable Python-side contract does not itself select or implement a native API, ABI, binding strategy, or processor.

Current non-claims

• No native processor exists.
• No native processor API or ABI has been defined.
• No C or C++ API has been selected.
• No binding or foreign-function technology has been selected.
• No differential native tests or differential native execution currently run.
• The Python assembler still materializes and stores sentence strings even though ProcessorOutput.message is exact immutable bytes.
• End-to-end zero-copy processing is not claimed.
• No benchmark, throughput, latency, memory, or other performance result follows from Campaign C, D, E, or F.
• No multiprocessing coordinator or worker-process architecture is implemented.
• No IPC or cross-process routing-snapshot distribution is implemented.
• No compatibility promise extends beyond the documented and tested behavioural surface.

Why the preparation matters

Explicit boundaries allow implementation work to be incremental:

• State can remain with its current owner until a deliberate boundary is introduced.
• A candidate implementation can be checked behavior-by-behavior rather than replacing the complete service at once.
• Routing, assembly, deduplication, metadata, and secure state have testable lifecycle edges.
• Injected clocks and deterministic generators make time- and ID-sensitive comparisons reproducible.
• Immutable statistics provide point-in-time state observations without mutating the system under test.
• Any future process separation would require separately specified ownership, cross-process supervision, restart/recovery, routing-snapshot distribution, and IPC boundaries. Campaign F prepared worker-facing boundaries but did not implement that separation.

The preparation is therefore an architectural option, not a feature claim.

Related pages

• Architecture Overview
• State, Lifecycle, and Limits
• Behavioural Contract
• Runtime Observability
• Architecture Evolution
• Roadmap