Phase 7.3: Prioritised Episodic Replay Buffer — architecture & design

[web3guru888]

🔄 Phase 7.3: Prioritised Episodic Replay Buffer

Phase 7.3 introduces a prioritised episodic replay buffer (PrioritisedReplayBuffer) that stores past training episodes and interleaves them with live experience during STDP updates. This is the third sub-phase of Phase 7's continual-learning stack.

Why Replay?

Mechanism	Operates On	When
EWC (Phase 6)	Parameter importance	Weight update time
Reptile (Phase 7.1)	Meta-weights	SLEEP_PHASE
Task Sampler (Phase 7.2)	Which task to run	Episode start
Replay (Phase 7.3)	Past episode gradients	Every update step

Buffer variants

Variant	Eviction	Sampling	Use Case
PrioritisedReplayBuffer	Lowest priority	Priority-proportional	Production
UniformReplayBuffer	FIFO	Uniform random	Ablation / unit tests

IS Correction

P(i) = p_i^α / Σ p_j^α → w_i = (N·P(i))^{-β} / max(w)

β annealed 0.4 → 1.0 over training.

SLEEP_PHASE hook order (step 8 added)

1. flush_pending_spikes()
2. EWCRegulariser.consolidate()
3. FisherAccumulator.force_snapshot()
4. TaskRegistry.evict_if_needed()
5. ReptileMetaLearner.meta_update()
6. CurriculumScheduler.advance_epoch()
7. TaskSampler.snapshot()
8. PrioritisedReplayBuffer.snapshot() ← NEW

Prometheus Metrics (5)

• asi_replay_buffer_size (Gauge)
• asi_replay_buffer_capacity (Gauge)
• asi_replay_episodes_added_total (Counter)
• asi_replay_episodes_sampled_total (Counter)
• asi_replay_mean_priority (Gauge)

Open Questions

1. Additive vs mixed-batch gradient blending?
2. Should episode priorities decay over time?
3. Per-task stratified sampling vs fully proportional?