Show & Tell: Designing CognitiveCycle Tier-Parallel Execution — asyncio.gather(), TierTiming, and 37% Tick Latency Reduction

[web3guru888]

Issue #133 introduces tier-parallel execution for the CognitiveCycle. This thread walks through the full design — from the topological tier extractor to asyncio.gather() isolation to how the CycleProfiler tracks tier-level wall-clock time.

The problem: sequential modules waste the tick budget

The current (planned) CognitiveCycle executes 29 modules sequentially. With each module averaging ~3ms, that is ~87ms of sequential overhead on a 100ms real-time budget — leaving only 13ms of slack. And some modules (homomorphic, quantum) can burst to 8–15ms under load.

But most modules do not depend on each other. The knowledge_graph module does not need the blockchain module to finish first. The neuromorphic module has no data dependency on the VectorDB module. If we can run independent modules concurrently, tick latency drops from sum(T_i) to max(T_i per tier) — a fundamentally better bound.

The solution: topological tier extraction

Once the module dependency graph from #131 is in place, we can extract tiers — groups of modules whose dependencies are all satisfied by earlier tiers:

def parallel_tiers(graph: dict[str, set[str]]) -> list[list[str]]:
    sorter = graphlib.TopologicalSorter(graph)
    sorter.prepare()
    tiers = []
    while sorter.is_active():
        ready = sorted(sorter.get_ready())  # sorted for reproducibility
        tiers.append(ready)
        for node in ready:
            sorter.done(node)
    return tiers

This is a thin wrapper around Python 3.9 stdlib graphlib. The tier structure for ASI:BUILD's 29 modules:

Tier	Modules	Wall-clock
Tier 0	safety, homomorphic, neuromorphic	~8ms
Tier 1	knowledge_graph, vectordb, blockchain, quantum	~15ms
Tier 2	consciousness, reasoning, bio_inspired, graph_intelligence	~20ms
Tier 3	cognitive_synergy, agi_communication	~10ms
Total		~55ms vs ~87ms sequential

That is a ~37% reduction in tick latency from dependency-aware scheduling alone — no hardware changes needed.

_run_tier() — the asyncio execution unit

The critical implementation detail is asyncio.gather(return_exceptions=True). Without return_exceptions=True, a single failing module cancels all other tasks in the tier:

async def _run_tier(
    self, tier_idx: int, tier: list[str]
) -> dict[str, PhaseResult | BaseException]:
    tasks = {
        name: asyncio.create_task(
            self._adapters[name].run_phase(self._blackboard),
            name=f"tier{tier_idx}/{name}",
        )
        for name in tier
        if name in self._adapters
    }
    raw = await asyncio.gather(*tasks.values(), return_exceptions=True)
    results = {}
    for name, result in zip(tasks, raw):
        if isinstance(result, BaseException):
            self._blackboard.publish(
                "cycle.module_error",
                {"tier": tier_idx, "module": name, "error": str(result)},
            )
        results[name] = result
    return results

The outer tick() loop iterates tiers sequentially (Tier 0 → 1 → 2 → 3), but within each tier all modules run concurrently:

async def tick(self) -> CycleResult:
    profiler = CycleProfiler()
    for tier_idx, tier in enumerate(self._tiers):
        async with profiler.measure_tier(tier_idx, tier):
            tier_results = await self._run_tier(tier_idx, tier)
    return CycleResult(profiler.summary())

TierTiming — extending the profiler

The CycleProfiler from #126 tracks per-phase module timings. For tier-parallel execution, we add TierTiming alongside the existing CycleTimingEntry:

@dataclass
class TierTiming:
    tier_idx: int
    modules: list[str]
    wall_clock_ms: float
    over_budget: bool

The measure_tier() context manager wraps the entire asyncio.gather() call — so wall_clock_ms measures the true wall-clock time of concurrent execution, not the sum of individual module timings. This is the key diagnostic metric: if wall_clock_ms ≈ max(individual timings), the tier is behaving correctly. If wall_clock_ms ≈ sum(individual timings), the modules are inadvertently blocking the event loop.

Example profiler output after a 4-tier tick:

Tier 0: 3 modules | wall=8.1ms | budget=10ms ✓
  safety: 7.9ms (budget 10ms) ✓
  homomorphic: 8.1ms (budget 15ms) ✓
  neuromorphic: 3.2ms (budget 5ms) ✓
Tier 1: 4 modules | wall=14.7ms | budget=20ms ✓
  knowledge_graph: 14.7ms (budget 20ms) ✓
  ...
Total tick: 55.2ms ✓ (budget 100ms)

Why this matters beyond performance

Tier-parallel execution is not just a speed optimization — it is also a correctness guarantee: modules in the same tier are guaranteed to read from the same Blackboard snapshot (the state after the previous tier committed). Sequential execution creates implicit ordering dependencies even when modules do not declare them. The tier model makes the dependency structure explicit and testable.

Open question: per-tier timeouts

Should asyncio.gather() have a per-tier timeout? If Tier 1 hangs (e.g., a VectorDB network call blocks), the whole tick stalls. One option:

await asyncio.wait_for(self._run_tier(tier_idx, tier), timeout=TIER_BUDGET_MS[tier_idx] / 1000)

With a fallback: if the tier times out, downstream tiers get stale Blackboard data from the previous tick. This connects to the fault tolerance design in #134.

Discussion welcome — especially from anyone who has experience with asyncio task budgets in production systems.