feat(P2): Daydreamer background consolidation, community detection, and P2P sharing#71
Merged
feat(P2): Daydreamer background consolidation, community detection, and P2P sharing#71
Conversation
…Sharing modules Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
…bility Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Subtask P2 issues for better tracking
feat(P2): Daydreamer background consolidation, community detection, and P2P sharing
Mar 13, 2026
Closed
Contributor
There was a problem hiding this comment.
Pull request overview
Implements major Daydreamer background-maintenance modules and introduces a first pass at privacy-safe P2P sharing (curiosity probes + subgraph exchange), along with required MetadataStore API extensions and comprehensive Vitest coverage.
Changes:
- Add Daydreamer modules: idle scheduling, Hebbian edge updates, full neighbor-graph recalculation, prototype recomputation, and community detection via label propagation.
- Add sharing modules: eligibility classification, curiosity probe broadcasting, subgraph export/import, and peer exchange; centralize sharing schema in
sharing/types.ts. - Extend
MetadataStorewith volume/shelf enumeration and single-edge deletion; update IndexedDB backend and tests accordingly.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| TODO.md | Marks P2 Daydreamer + sharing tasks as completed. |
| PLAN.md | Updates implementation status tables/exit criteria for Daydreamer + sharing. |
| core/types.ts | Extends MetadataStore with getAllVolumes, getAllShelves, and deleteEdge. |
| storage/IndexedDbMetadataStore.ts | Implements getAllVolumes, getAllShelves, and deleteEdge for IndexedDB backend. |
| core/crypto/uuid.ts | Adds randomUUID() helper with platform + fallback implementations. |
| daydreamer/IdleScheduler.ts | Adds cooperative idle-time scheduler with priority queue. |
| daydreamer/HebbianUpdater.ts | Adds LTP/LTD/pruning and triggers salience recompute + promotion sweep. |
| daydreamer/FullNeighborRecalc.ts | Adds dirty-volume full neighbor recomputation bounded by a budget. |
| daydreamer/PrototypeRecomputer.ts | Adds recomputation of volume/shelf prototypes and attempts to update hotpath. |
| daydreamer/ClusterStability.ts | Adds label propagation community detection + helper detectors. |
| sharing/types.ts | Introduces canonical types for probes, fragments, slices, and transport messages. |
| sharing/EligibilityClassifier.ts | Adds deterministic eligibility rules to block PII/credentials/etc. |
| sharing/SubgraphExporter.ts | Exports eligibility-filtered slices using BFS expansion. |
| sharing/SubgraphImporter.ts | Imports fragments/slices with schema validation + identity stripping. |
| sharing/PeerExchange.ts | Adds opt-in proactive slice exchange over transport. |
| sharing/CuriosityBroadcaster.ts | Adds rate-limited probe queue + fragment handler dispatch. |
| tests/sharing/*.test.ts | Adds tests for eligibility, broadcaster, and subgraph exchange round-trip. |
| tests/daydreamer/*.test.ts | Adds tests for idle scheduler, Hebbian updater, full recalc, prototypes, and cluster stability. |
| tests/integration/Daydreamer.test.ts | Adds integration coverage for Daydreamer passes using fake-indexeddb. |
| tests/SalienceEngine.test.ts | Updates MetadataStore mock to satisfy new interface methods. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements all P2 milestone tasks: cooperative background memory consolidation (Daydreamer), label-propagation community detection with quota enforcement, and privacy-safe P2P curiosity/sharing infrastructure.
MetadataStore interface extensions
getAllVolumes(),getAllShelves()— needed by FullNeighborRecalc and PrototypeRecomputer to enumerate hierarchydeleteEdge(from, to)— required for Hebbian pruningcore/crypto/uuid.ts—randomUUID()withcrypto.randomUUID()+ fallbackDaydreamer modules (
daydreamer/)IdleScheduler— cooperative task loop viarequestIdleCallback/setImmediate; priority queue, FIFO tie-break, error-isolated per taskHebbianUpdater— LTP (strengthenEdges), LTD+pruning (decayAndPrune); enforcesmaxDegree; triggersbatchComputeSalience+runPromotionSweepfor changed nodesFullNeighborRecalc— pairwise cosine recalc for dirty volumes; pair budget bounded bycomputeCapacity(graphMass)per idle cycle (O(√(t log t))); clears dirty flag, recomputes saliencePrototypeRecomputer— volume medoid selection (minimises avg pairwise distance) and centroid, then shelf routing prototypes; appends to VectorStore; runs tier-quota sweepClusterStability— deterministic label propagation on MetroidNeighbor graph; persistscommunityId→PageActivity; exportsdetectOversizedCommunities/detectEmptyCommunitiesSharing modules (
sharing/)types.ts—CuriosityProbe(with requiredmimeType+modelUrnfields),GraphFragment,SubgraphSlice,EligibilityDecisionEligibilityClassifier— regex-based PII/credential detection (credentials, credit card, IBAN, SSN, email, medical); deterministic with typed reason codes;classifyPage/filterEligibleSubgraphExporter— BFS expansion from seed nodes, eligibility filter, creator identity stripped, provenance tagged;exportForProbe+exportForExchangeSubgraphImporter— schema validation, sender identity stripping, safe page+edge import;importFragment/importSliceCuriosityBroadcaster— rate-limited probe queue; dispatches incominggraph_fragmentresponses to registered handlerPeerExchange— opt-in signed subgraph exchange; inject anyP2PTransportimplementationTests
93 new tests across 8 test files (
tests/daydreamer/,tests/sharing/,tests/integration/Daydreamer.test.ts). Integration test validates Williams Bound invariant (resident count ≤ H(t)) holds after multiple Daydreamer passes.Original prompt
💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.