Show & Tell: Designing Phase 5.1 Online Learning — STDP Mid-Cycle Updates, Federated Hot-Reload, and Safety-Gated Weight Deltas

[web3guru888]

Phase 5.1: Online Learning Overview

Phase 5.1 brings one of the most technically ambitious capabilities to ASI:BUILD: modules that learn from live experience without stopping the CognitiveCycle. This post walks through the design decisions, data flows, and implementation patterns.

---

The core problem

Today, ASI:BUILD modules are trained offline and loaded as fixed weights at startup. The neuromorphic module implements STDP learning rules, but they run in test/benchmark mode — not wired into the live CognitiveCycle. The federated learning module aggregates updates from distributed nodes, but applying those updates requires a restart.

Phase 5.1 changes this. Learning becomes a first-class CyclePhase.

---

Architecture

New CyclePhase: LEARNING

The existing CyclePhase enum (from Issue #126) gets a new phase:

class CyclePhase(Enum):
    PERCEPTION    = "perception"
    PREPROCESSING = "preprocessing"
    REASONING     = "reasoning"
    LEARNING      = "learning"      # ← NEW in Phase 5.1
    MEMORY        = "memory"
    ACTION        = "action"
    SAFETY_CHECK  = "safety_check"
    AUDIT         = "audit"
    IDLE          = "idle"

The LEARNING phase runs after REASONING, before MEMORY. It processes buffered experience events and applies weight deltas to modules that support online updates.

---

OnlineLearningAdapter

The OnlineLearningAdapter is the central class in Phase 5.1. It:

1. Subscribes to experience.* Blackboard entries written during PERCEPTION/REASONING
2. Computes weight deltas using the module's native learning rule (STDP for neuromorphic, gradient for neural nets)
3. Routes each delta through EthicalVerificationEngine (safety gate)
4. Applies approved deltas atomically; rolls back on rejection

@dataclass
class WeightDelta:
    module_name: str
    delta_type: Literal["stdp", "federated", "gradient"]
    payload: dict            # module-specific: {layer, weights} or {pre_post_pairs}
    magnitude: float         # L2 norm of the change — used by safety gate
    tick: int
    source: str              # "experience" | "federation" | "backprop"

    def to_audit_dict(self) -> dict:
        """Hash payload, not raw weights — for audit trail."""
        import hashlib, json
        return {
            "module": self.module_name,
            "type": self.delta_type,
            "magnitude": self.magnitude,
            "payload_hash": hashlib.sha256(
                json.dumps(self.payload, sort_keys=True).encode()
            ).hexdigest(),
        }

---

STDP mid-cycle updates

The neuromorphic module already computes Δw for each pre-post spike pair. In Phase 5.1, these are surfaced as WeightDelta objects during the LEARNING phase:

CognitiveCycle PERCEPTION phase
  → NeuromorphicBlackboardAdapter writes spike events to "neuromorphic.spikes"

CognitiveCycle LEARNING phase
  → OnlineLearningAdapter reads "neuromorphic.spikes"
  → Computes STDP Δw for each pre-post pair
  → Routes through EthicalVerificationEngine
  → Applies Δw to live SNN weights
  → Writes "learning.stdp_update" to Blackboard (TTL=600s)

Performance target: STDP delta computation for 1K-neuron network ≤ 5ms (fits in 120ms tick budget with room to spare).

---

Federated hot-reload

The federated learning module aggregates weight updates from distributed nodes on a configurable schedule (default: every 100 ticks ≈ 10s at 10Hz). Phase 5.1 applies those aggregated updates without restarting:

FederatedOrchestrator receives aggregated global model update
  → Byzantine filter (DBSCAN outlier removal)
  → Wraps as WeightDelta(delta_type="federated", ...)
  → Routes through EthicalVerificationEngine
  → On approval: atomic weight swap (write lock, no partial update)
  → On rejection: rollback (_pending_weight_buffer discarded)
  → Writes "learning.federated_update" to Blackboard

Atomic swap pattern — critical to avoid partial updates mid-tick:

async def _apply_federated_delta(self, delta: WeightDelta) -> None:
    async with self._weight_lock:
        # Copy current weights to rollback buffer
        self._rollback_buffer = copy.deepcopy(self._weights)
        try:
            self._weights = delta.payload["new_weights"]
        except Exception:
            # Restore on any failure
            self._weights = self._rollback_buffer
            raise

---

Safety gate design

All weight updates go through EthicalVerificationEngine before application (from Issue #176 constraints):

Delta type	MAX_DELTA_NORM	What safety gate checks
STDP	0.1	Magnitude, no forbidden associations
Federated	0.5	Byzantine pre-filter + magnitude
Gradient	0.2	Gradient clipping already applied; magnitude final check

If safety gate rejects, the event is written to safety.weight_update_blocked with the rejection reason and the weights are unchanged.

---

KG transactional writes

When a module learns a new association (e.g., concept A reliably predicts concept B with confidence > 0.85), it writes to both the Knowledge Graph and the Blackboard in a single transaction:

OnlineLearningAdapter detects high-confidence association
  → kg.transaction() context manager
  → tx.add_triple(subject, relation, object_, confidence, valid_from=now)
  → On commit: blackboard.write("learning.new_association.{module}", ...)
  → On rollback: neither KG nor Blackboard updated

The transactional pattern ensures the KG and Blackboard are never inconsistent.

---

Blackboard namespace additions (Phase 5.1)

Key	TTL	Producer	Consumer
learning.stdp_update	600s	OnlineLearningAdapter	CognitiveSynergy, dashboard
learning.federated_update	600s	FederatedOrchestrator	Multi-Agent, dashboard
learning.new_association.*	3600s	OnlineLearningAdapter	KnowledgeEngine, Memory
safety.weight_update_blocked	300s	OnlineLearningAdapter	Safety dashboard, audit
experience.*	60s	PERCEPTION phase	LEARNING phase

---

Implementation order

1. Add WeightDelta dataclass and OnlineLearningAdapter skeleton
2. Wire STDP delta extraction from NeuromorphicBlackboardAdapter spike events
3. Add safety gate integration (reuse SafetyBlackboardAdapter pattern from Wire EthicalVerificationEngine into Blackboard write/read pipeline #37)
4. Add federated hot-reload with atomic weight swap
5. Add KG transactional write helper
6. Add CyclePhase.LEARNING to CognitiveCycle tick pipeline
7. Write 8 unit tests (normal/blocked/rollback/KG/concurrent)

---

Open questions

1. Update frequency: Should STDP run every tick or every N ticks to control computational cost?
2. Rollback depth: If a federated update is applied but causes downstream module failures 3 ticks later, should we support multi-tick rollback?
3. Cross-module learning: Should STDP updates in the neuromorphic module influence connection weights in the Hybrid Reasoning Engine, or keep learning module-local?

Relevant issue: #181. Community input welcome!