Skip to content

P0- We have a fundamental misunderstanding of the design and it lead to drift. Everything is blocked until this is resolved. #54

@devlux76

Description

@devlux76

CORTEX Architecture Alignment & Documentation Reconstruction

Objective

The current repository has diverged from the intended architecture of CORTEX due to an early conceptual collapse between medoids and Metroids. This drift propagated through DESIGN.md, PLAN.md, and implementation assumptions.

The result is that the system now resembles a vector neighbor graph, while the original architecture is a dialectical epistemic exploration system.

Your task is to restore architectural correctness by rewriting documentation and auditing the codebase against the intended design.

Do not modify code in this pass.

Instead:
1. Rewrite DESIGN.md to accurately describe the architecture.
2. Update PLAN.md to align with the corrected design.
3. Perform a comprehensive code review identifying architectural drift.
4. Rewrite TODO.md so implementation tasks match the true design.

Architectural Overview

CORTEX is a distributed epistemic memory system designed to enable:
• structured recall
• epistemically honest exploration
• knowledge gap detection
• collaborative learning through P2P exchange

The architecture is inspired by biological cognition and is composed of three major subsystems.

HIPPOCAMPUS

Hebbian Influenced Parametric Projection Over Clustered Autoassociative Memory Patterns to Unify Systems

Responsibilities:
• embedding generation
• hierarchical clustering
• medoid discovery
• Hebbian association edges
• append-only vector storage

HIPPOCAMPUS handles fast ingestion of experience.

CORTEX

Responsible for recall, structured exploration, and knowledge discovery.

This subsystem constructs Metroids, performs dialectical search, and detects knowledge boundaries.

DAYDREAMER

Idle-time consolidation process responsible for:
• LTP/LTD edge adjustments
• cluster stabilization
• prototype recomputation
• experience replay
• community detection

DAYDREAMER maintains long-term memory structure.

Critical Conceptual Distinction

The repository currently conflates three separate mathematical constructs:

Concept Meaning
Medoid Actual memory node representing cluster center
Centroid Mathematical average of vectors
Metroid Structured dialectical search probe

The current documentation incorrectly treats Metroid as a synonym for medoid-derived neighbors.

This is incorrect and must be corrected.

Correct Definition of a Metroid

A Metroid is a structured search probe used for epistemically balanced exploration of a topic.

It is constructed as:

Metroid = { m1, m2, c }

Where:
• m1 = thesis medoid representing the topic
• m2 = antithesis medoid discovered through constrained search
• c = centroid between them

The Metroid is built through recursive Matryoshka embedding exploration.

Metroid Construction Algorithm
1. Select the topic medoid (m1).
2. Freeze the protected lower embedding dimensions.
3. Within the remaining dimensions, search for the nearest medoid representing semantic opposition.

This medoid becomes:

m2

4.	Compute centroid:

c = centroid(m1, m2)

5.	Prefer the centroid as the search center to prevent semantic drift.
6.	Repeat the process while progressively unwinding Matryoshka embedding layers.
7.	Stop once reaching the protected dimension.

The protected dimension ensures semantic invariants remain stable.

Example:

Query:

How can I help pepperoni stick to my pizza?

Without dimensional protection, vector similarity could drift toward adhesives and glue.

Protected dimensions maintain the culinary context, preventing such semantic failures.

Dialectical Exploration

This algorithm ensures each search explores:
• supporting ideas (thesis)
• opposing ideas (antithesis)
• balanced conceptual center (centroid)

This prevents confirmation bias and enables epistemically balanced reasoning.

Knowledge Gap Detection

If at any stage of Metroid construction no suitable antithesis medoid can be found within constraints:

knowledge_gap = true

The system must conclude it does not possess sufficient knowledge to answer.

This triggers a P2P curiosity request.

The node broadcasts the Metroid probe representing the boundary of its knowledge.

Peers receiving this probe search their own graphs and may respond with relevant fragments.

This mechanism enables distributed learning without hallucination.

Required Documentation Corrections

  1. Rewrite DESIGN.md

DESIGN.md must clearly describe:
• HIPPOCAMPUS ingestion architecture
• medoid discovery
• Metroid construction
• Matryoshka dimensional unwinding
• dialectical search
• knowledge gap detection
• P2P curiosity requests

The document must allow a new engineer to understand and implement the architecture correctly.

  1. Update PLAN.md

After DESIGN.md is corrected, ensure implementation phases match the architecture.

Verify that the plan includes:
• MetroidBuilder component
• dimensional unwinding search
• antithesis discovery
• centroid stabilization
• knowledge gap detection
• P2P curiosity broadcasting

Restructure phases if necessary.

  1. Comprehensive Code Review

Perform a full repository review and identify where implementation diverges from the architecture.

For each divergence provide:
• file
• component
• current behavior
• intended behavior
• required correction

Do not modify code during this pass.

  1. Rewrite TODO.md

Ensure TODO tasks align with the corrected architecture.

Include tasks for:
• MetroidBuilder implementation
• Matryoshka search logic
• dialectical search pipeline
• knowledge gap detection
• P2P curiosity requests
• DAYDREAMER consolidation tasks

Expected Deliverables

Produce the following artifacts:
1. Corrected DESIGN.md
2. Updated PLAN.md
3. Repository-wide architectural review report
4. Updated TODO.md

No code changes should be committed during this task.

Goal

Restore architectural fidelity so CORTEX functions as a distributed epistemic memory system capable of dialectical knowledge exploration and collaborative learning, rather than a simple vector retrieval engine.

Metadata

Metadata

Labels

P0: criticalCritical path — blocks all dependent workbugSomething isn't workingdocumentationImprovements or additions to documentationlayer: ciCI/CD pipeline and build toolinglayer: computeWebGPU / WebGL / WebNN / WASM vector backendslayer: cortexRetrieval orchestration (rank → expand → order)layer: daydreamerBackground consolidation (LTP/LTD, recalc)layer: documentationAPI docs, developer guide, architecture diagramslayer: embeddingsEmbedding providers and resolverlayer: foundationCore types, model profiles, cryptolayer: hippocampusIngest orchestration (chunk → embed → persist)layer: storageOPFS vector store and IndexedDB metadata storelayer: testingTest coverage and integration tests

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions