Architecture Evolution

Architecture Evolution

This page records why the architecture changed. It is historical context, not a second roadmap or behavioural specification.

Original baseline

AISMixer began as a practical Python stream service centered on:

• network ingress;
• extraction and normalization of AIS NMEA text;
• multipart assembly;
• NMEA TAG handling;
• near-real-time deduplication; and
• UDP forwarding.

Logical source_id and target_id routing, immutable routing snapshots, and the optional local control plane were added later. Legacy broadcast behavior was retained for configurations without an active routing table.

Campaign A: behavioural contract

Campaign A preserved Python as the reference implementation while making observable processing behavior explicit.

Its main architectural outcomes were:

• a normative behavioural contract;
• precise ingress and extraction boundaries;
• explicit multipart ordering, duplicate, conflict, and completion behavior;
• clear separation of routing identity and emitted TAG metadata;
• group-atomic multipart deduplication;
• one immutable routing decision per accepted string event;
• deterministic clock and generator seams for tests; and
• a defined surface for future differential comparison.

Campaign A did not select a native API or change the data-plane implementation language.

Campaign B: explicit state and limits

Campaign B focused on state as an architectural responsibility rather than an incidental collection of containers.

Deduplication

• One authoritative process-local owner.
• Monotonic TTL with exact boundary behavior.
• Duplicate non-refresh.
• Optional instance-wide capacity.
• Expiry before deterministic oldest-live eviction.
• Explicit reset and immutable statistics.

Multipart assembly and TAG context

• Explicit per-generation assembler state.
• Unique-progress timestamps and exact duplicate non-refresh.
• Single-sentence and invalid/limit paths isolated from multipart state.
• Optional fragment and pending-group limits.
• Deterministic lifecycle outcomes and discarded keys.
• Multipart TAG s, c, and g cleanup synchronized with assembler conflict, expiry, capacity, completion, and reset boundaries.

Secure ingress

• One SecureState owner for replay records, sessions, and per-session nonces.
• Separation of protocol wall time from local monotonic lifecycle time.
• Hard replay, session, and nonce capacities.
• Deterministic expiry, replacement, and eviction behavior.
• Traffic-driven cleanup and restart loss made explicit.
• Immutable statistics with mutually exclusive removal accounting.

Closure baseline

The source repository records a Campaign B audit baseline in its canonical behavioural contract. That snapshot confirms a point in project history; its test counts are not permanent compatibility guarantees.

Campaign C: data representation

Campaign C changed the production representation path while preserving the Campaign A and B observable and lifecycle semantics:

• built-in UDP and UDPSEC producers enqueue immutable IngressFrame values, while valid legacy IngressEvent inputs cross one compatibility adapter;
• each accepted frame receives one routing snapshot and, only for an active table, one match against frame.source_id;
• the scanner works on bytes and returns immutable half-open spans;
• ParsedSentence carries parse-once fragment and relevant TAG metadata; and
• production assembly enters through feed_parsed_outcome(), while the valid legacy string feed_outcome() API converges on the same lifecycle.

The Python assembler still materializes and stores sentence strings, and output remains string-based. At the Campaign C closure point, the synchronous processor contract later delivered by Campaign D did not yet exist. Campaign C also added no native API or ABI, native code, end-to-end zero-copy processing, benchmarks, or multiprocessing.

Campaign D: processor and runtime boundary

Campaign D made synchronous processing and asynchronous runtime ownership explicit without changing the established observable semantics.

D1 — Processor contract

core/data_plane.py defines immutable ProcessingSnapshot and ProcessorOutput values, the RoutingDisposition enum, and the synchronous DataPlaneProcessor.process(frame, snapshot) protocol. One accepted IngressFrame and one immutable snapshot produce one complete ordered tuple[ProcessorOutput, ...].

The contract owns no asyncio tasks, sockets, queues, transports, forwarders, multiprocessing, IPC, or native dependency. The completion acknowledgement used by runtime orchestration is therefore not part of the public processor contract.

D2 — Python reference processor

PythonDataPlaneProcessor became the sole production data-plane processor and the normative Python reference implementation. Its long-lived, process-local instance owns:

• the assembler, which separately owns its multipart fragment groups;
• the deduplicator;
• multipart output TAG s, c, and g contexts keyed by AssemblyKey; and
• processing configuration and synchronous processing effects.

It does not own ingress queues, asyncio tasks, sockets, forwarders, mutable routing-state replacement, processes, IPC, or native bindings.

D3 — Explicit runtime stages

The one production path is now:

ingress producers
    -> ingress fan-in
    -> processor stage
    -> PythonDataPlaneProcessor
    -> complete ordered ProcessorOutput tuple
    -> private completion barrier
    -> egress stage
    -> UDP forwarders

Fan-in transports input without processing it. The processor stage rejects unsupported items before snapshot acquisition, then captures one snapshot and calls the processor once per accepted frame. Empty output completes locally; a non-empty batch crosses the private egress handoff, and the processor stage does not consume the next frame until that batch completes.

The egress stage sends outputs sequentially in tuple order. A send failure prevents later outputs in that batch and prevents later-frame processor runahead, but it does not roll back completed processor effects or already sent outputs. No later frame is processed after a processor or egress failure. The barrier supplies ordering, not transactional delivery, retry, rollback, replay, or durable acknowledgement.

D4 — Process-local supervision

One fail-fast supervisor owns the essential plain UDP, UDPSEC, ingress fan-in, processor-stage, and egress-stage tasks. Fan-in privately owns and supervises its per-input readers. The first real failure propagates; unexpected normal return or internal cancellation becomes a role-named runtime failure. External cancellation remains CancelledError. Siblings and private readers are cancelled and awaited, and their outcomes are retrieved.

An empty fan-in stays idle until cancellation. Partial task-creation failure cleans already-created tasks and closes the rejected coroutine. UDPSEC closes its owned socket after bind failure, runtime failure, or cancellation.

This is single-process asyncio task supervision. It defines termination and cleanup, not coordinator/worker supervision, worker restart policy, automatic service restart, delivery retry, or durable recovery.

D5 — Closure audit

Campaign D was closed in the main repository by ceae5d44a9c5e1b97d5878497ef6432a31b2895c (Document Campaign D closure). The audited implementation snapshot was d35de4d84233b27e8541f0cc1b5c041ad464dbc2 (Supervise runtime task lifecycle). Its historical full-suite result was 1248 passed, 18 skipped, 0 failed from 1266 collected tests. These values are closure evidence, not a promise that future suite counts remain unchanged.

Architectural result

The current Python reference now makes representation, processor, and runtime boundaries explicit and these questions answerable for each contract-relevant state item:

Who owns it?
What is its identity?
Which clock applies?
When does it expire?
What is its capacity?
What refreshes it?
Why was it removed?
Is it durable or shared?

This is the basis for the State, Lifecycle, and Limits model and the Native-Ready Reference Foundation.

Future direction

The maintained Roadmap describes possible staged work without dates:

• a coordinator process plus dedicated ingress and egress worker processes;
• cross-process supervision and worker restart/recovery policy;
• IPC distribution of routing snapshots;
• optional routing-state restoration or persistence research;
• safe configuration reload or watch behavior;
• rollback history and richer operational observability;
• additional egress and control transports;
• maritime security, spoof/anomaly, and feed-quality research.

Already implemented within the current single process are explicit stage ownership, the synchronous processor contract, the processor-to-egress completion barrier, and fail-fast task supervision. Still future are a coordinator, ingress and egress worker processes, cross-process supervision, worker restart/recovery, IPC, and routing-snapshot distribution.

Separately, the behavioural contract and DataPlaneProcessor boundary supply the observable Python reference for possible future differential conformance. A native implementation, native API or ABI, bindings, bytes-native assembler/output state, differential native execution, and benchmarks remain deferred.

These are directions, not implemented guarantees. In particular, there is no multiprocessing coordinator, IPC state sharing, persistent routing, automatic reload, spoof detector, or native processor today.

Sources

• Normative behavioural contract
• Project roadmap
• Trilingual README