Architecture Evolution

Architecture Evolution

This page records why the architecture changed. It is historical context, not a second roadmap or behavioural specification.

Original baseline

The AISMixer project began with aismixer, a practical Python stream service centered on:

• network ingress;
• extraction and normalization of AIS NMEA text;
• multipart assembly;
• NMEA TAG handling;
• near-real-time deduplication; and
• UDP forwarding.

Logical source-ID and operator-facing named-target routing, immutable routing snapshots, and the optional local control plane were added later. Legacy broadcast behavior was retained for configurations without an active routing table.

Campaign A: behavioural contract

Campaign A preserved Python as the reference implementation while making observable processing behavior explicit.

Its main architectural outcomes were:

• a normative behavioural contract;
• precise ingress and extraction boundaries;
• explicit multipart ordering, duplicate, conflict, and completion behavior;
• clear separation of routing identity and emitted TAG metadata;
• group-atomic multipart deduplication;
• one immutable routing decision per accepted string event;
• deterministic clock and generator seams for tests; and
• a defined surface for future differential comparison.

Campaign A did not select a native API or change the data-plane implementation language.

Campaign B: explicit state and limits

Campaign B focused on state as an architectural responsibility rather than an incidental collection of containers.

Deduplication

• One authoritative process-local owner.
• Monotonic TTL with exact boundary behavior.
• Duplicate non-refresh.
• Optional instance-wide capacity.
• Expiry before deterministic oldest-live eviction.
• Explicit reset and immutable statistics.

Multipart assembly and TAG context

• Explicit per-generation assembler state.
• Unique-progress timestamps and exact duplicate non-refresh.
• Single-sentence and invalid/limit paths isolated from multipart state.
• Optional fragment and pending-group limits.
• Deterministic lifecycle outcomes and discarded keys.
• Multipart TAG s, c, and g cleanup synchronized with assembler conflict, expiry, capacity, completion, and reset boundaries.

Secure ingress

• One SecureState owner for replay records, sessions, and per-session nonces.
• Separation of protocol wall time from local monotonic lifecycle time.
• Hard replay, session, and nonce capacities.
• Deterministic expiry, replacement, and eviction behavior.
• Traffic-driven cleanup and restart loss made explicit.
• Immutable statistics with mutually exclusive removal accounting.

Closure baseline

The source repository records a Campaign B audit baseline in its canonical behavioural contract. That snapshot confirms a point in project history; its test counts are not permanent compatibility guarantees.

Campaign C: data representation

Campaign C changed the production representation path while preserving the Campaign A and B observable and lifecycle semantics:

• built-in UDP and UDPSEC producers enqueue immutable IngressFrame values, while valid legacy IngressEvent inputs cross one compatibility adapter;
• each accepted frame receives one routing snapshot and, only for an active table, one match against frame.source_id;
• the scanner works on bytes and returns immutable half-open spans;
• ParsedSentence carries parse-once fragment and relevant TAG metadata; and
• production assembly enters through feed_parsed_outcome(), while the valid legacy string feed_outcome() API converges on the same lifecycle.

The Python assembler still materializes and stores sentence strings. At the Campaign C closure point, output was also string-based and the synchronous processor contract later delivered by Campaign D did not yet exist. Campaign C added no native API or ABI, native code, end-to-end zero-copy processing, benchmarks, or multiprocessing.

Campaign D: processor and runtime boundary

Campaign D made synchronous processing and asynchronous runtime ownership explicit without changing the established observable semantics.

D1 — Processor contract

Campaign D introduced immutable processing-snapshot and processor-output values and the synchronous DataPlaneProcessor.process(frame, snapshot) protocol. One accepted IngressFrame and one immutable snapshot produced one complete ordered processor result. Campaign E later refined the snapshot, target, byte, and public batch representations without changing the synchronous call.

The contract owns no asyncio tasks, sockets, queues, transports, forwarders, multiprocessing, IPC, or native dependency. The completion acknowledgement used by runtime orchestration is therefore not part of the public processor contract.

D2 — Python reference processor

PythonDataPlaneProcessor became the sole production data-plane processor and the behavioural Python reference implementation. Its long-lived, process-local instance owns:

• the assembler, which separately owns its multipart fragment groups;
• the deduplicator;
• multipart output TAG s, c, and g contexts keyed by AssemblyKey; and
• processing configuration and synchronous processing effects.

It does not own ingress queues, asyncio tasks, sockets, forwarders, mutable routing-state replacement, processes, IPC, or native bindings.

D3 — Explicit runtime stages

Campaign D established the stage sequence:

ingress producers
    -> ingress fan-in
    -> processor stage
    -> PythonDataPlaneProcessor
    -> complete ordered processor result
    -> private completion barrier
    -> egress stage
    -> UDP forwarders

At the Campaign D closure, fan-in transported input without processing it. The processor stage rejected unsupported items before snapshot acquisition, then captured one snapshot and called the processor once per accepted frame. Empty output completed locally; a non-empty batch crossed the private egress handoff, and the processor stage did not consume the next frame until that batch completed. Campaign F later moved compatibility handling and snapshot binding to capacity-safe fan-in admission while preserving serialized processor calls.

The egress stage dispatches outputs sequentially in result order. A dispatch failure prevents later outputs in that batch and prevents later-frame processor runahead, but it does not roll back completed processor effects or already dispatched outputs. No later frame is processed after a processor or egress failure. The barrier supplies ordering, not transactional delivery, retry, rollback, replay, or durable acknowledgement.

D4 — Process-local supervision

One fail-fast supervisor owns the essential plain UDP, UDPSEC, ingress fan-in, processor-stage, and egress-stage tasks. Fan-in privately owns and supervises its per-input readers. The first real failure propagates; unexpected normal return or internal cancellation becomes a role-named runtime failure. External cancellation remains CancelledError. Siblings and private readers are cancelled and awaited, and their outcomes are retrieved.

An empty fan-in stays idle until cancellation. Partial task-creation failure cleans already-created tasks and closes the rejected coroutine. UDPSEC closes its owned socket after bind failure, runtime failure, or cancellation.

This is single-process asyncio task supervision. It defines termination and cleanup, not coordinator/worker supervision, worker restart policy, automatic service restart, delivery retry, or durable recovery.

D5 — Closure audit

Campaign D was closed in the main repository by ceae5d44a9c5e1b97d5878497ef6432a31b2895c (Document Campaign D closure). The audited implementation snapshot was d35de4d84233b27e8541f0cc1b5c041ad464dbc2 (Supervise runtime task lifecycle). Exact historical validation remains in the source repository rather than being duplicated here.

Campaign E: numeric and byte egress foundation

Campaign E completed the Python/native-ready data-plane foundation:

• Forwarder assigns every configured destination an immutable dense, zero-based numeric egress ID, including unnamed legacy destinations;
• routing candidates resolve operator-facing target names and compile a complete numeric target-only plan before installation;
• each accepted frame receives a target-only ProcessingSnapshot containing the routing generation, deduplication mode, and resolved numeric target IDs;
• DataPlaneProcessor.process() returns an ordered OutputBatch;
• every ProcessorOutput carries exact immutable bytes and explicit ordered numeric target IDs; and
• production egress dispatches outputs and their destinations sequentially through Forwarder.send_to_ids().

The complete OutputBatch still exists before the first asynchronous dispatch, and the non-empty-batch completion barrier still prevents later-frame processor runahead. This is send ordering, not confirmed network delivery.

Campaign E added no native implementation, API, ABI, binding, multiprocessing, IPC, coordinator, or worker-process architecture, and it made no performance claim.

Architectural result

The current Python reference now makes representation, processor, and runtime boundaries explicit and these questions answerable for each contract-relevant state item:

Who owns it?
What is its identity?
Which clock applies?
When does it expire?
What is its capacity?
What refreshes it?
Why was it removed?
Is it durable or shared?

This is the basis for the State, Lifecycle, and Limits model and the Native-Ready Reference Foundation.

Campaign F: Worker Readiness

The maintained Roadmap describes remaining staged work without promised dates.

Campaign F closed on 2026-08-09. It completed the Worker Readiness foundation inside the existing single Python process:

• one private 1024-item ingress queue for each input, shared 1024-item processing admission, and a one-batch egress queue;
• wait-based process-local backpressure, without a stage-level drop-on-full branch;
• capacity-safe binding of each frame and immutable processing snapshot into a ProcessingWorkItem only after processing capacity becomes available;
• one explicitly owned PythonDataPlaneProcessor instance with synchronous, ordered reset semantics;
• immutable pull-based queue, processor, egress-operation, input-traffic, and output-traffic statistics; and
• read-only runtime statistics plus interactive operation through aismixerctl.

Campaign F refined the earlier snapshot boundary: a frame waiting for processing capacity has captured no routing state and may see a later routing replacement, while an admitted work item retains its bound generation and target IDs. It also kept the Campaign D ordered processor-to-egress barrier: later work may be admitted, but the next processor call does not begin until the current non-empty batch completes its awaited local egress work.

These are bounded and observable in-process boundaries that can later become worker boundaries. Campaign F did not introduce coordinator, ingress-worker, processor-worker, or egress-worker processes, multiprocessing, IPC, cross-process supervision or metrics aggregation, native bindings, or an ABI.

Later architecture

Later Process Architecture

Only later process-architecture work may introduce a coordinator process, dedicated ingress, processor, and egress workers, cross-process supervision and failure handling, IPC, routing-snapshot distribution, restart or recovery policy, and metrics aggregation. None is implemented today, and this Roadmap direction is not a fixed final architecture contract.

Later Native Implementation

The behavioural contract and DataPlaneProcessor boundary can support a later native processor and differential-conformance execution. No native processor, API, ABI, or binding has been selected or implemented, and no benchmark or performance claim exists until measured.

Other future tracks include optional routing-state restoration, safe configuration reload, rollback history, exported or persistent monitoring, additional egress and control transports, and maritime security or feed-quality research. These are directions, not implemented guarantees.

Sources

• Normative behavioural contract
• Project roadmap
• Trilingual README