Skip to content

P0-G: Implement SalienceEngine — hotpath admission, promotion & eviction#44

Merged
devlux76 merged 13 commits intomainfrom
copilot/implement-salience-engine
Mar 13, 2026
Merged

P0-G: Implement SalienceEngine — hotpath admission, promotion & eviction#44
devlux76 merged 13 commits intomainfrom
copilot/implement-salience-engine

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 13, 2026

SalienceEngine is the decision-making layer for hotpath admission, needed by ingest, query, and Daydreamer before any of those modules can be implemented.

New modules

  • core/HotpathPolicy.ts — Williams Bound policy foundation. computeCapacity (H(t)), computeSalience (sigma formula), deriveTierQuotas / deriveCommunityQuotas (largest-remainder allocation), and frozen DEFAULT_HOTPATH_POLICY with structured HotpathPolicy interface containing nested salienceWeights and tierQuotaRatios objects (c=0.5, alpha=0.5, beta=0.3, gamma=0.2, tier ratios).

  • core/SalienceEngine.ts — Core salience computation (computeNodeSalience, batchComputeSalience), admission gating (shouldPromote), eviction targeting (selectEvictionTarget with deterministic tie-breaking by entityId), and lifecycle helpers (bootstrapHotpath, runPromotionSweep) with community-aware quota enforcement. Tier and community quotas are enforced regardless of overall capacity remaining, preventing monopolization during ramp-up. Uses in-memory entry caching during sweep loops to avoid repeated store reads.

Type & storage extensions

  • core/types.ts — Added PageActivity, HotpathEntry, TierQuotas, TierQuotaRatios, SalienceWeights; extended MetadataStore with hotpath methods (including removeHotpathEntry) and page activity methods.

  • storage/IndexedDbMetadataStore.ts — DB schema v2: hotpath_index store (keyed by entityId, indexed by tier) and page_activity store. Implements all new MetadataStore methods including removeHotpathEntry and evictWeakest with deterministic tie-breaking by entityId.

Tests

  • tests/SalienceEngine.test.ts — 40 tests covering: H(t) monotonicity/sublinearity/edge-case safety, sigma formula correctness, bootstrap fills to exactly H(t), steady-state promotion only when candidate beats weakest, eviction targets exactly the weakest resident, community quotas prevent single-community monopolization, tier quotas prevent hierarchy-level domination (even when overall capacity has room), deterministic eviction under identical state.

  • tests/HotpathPolicy.test.ts — Updated to use structured DEFAULT_HOTPATH_POLICY interface with nested salienceWeights and tierQuotaRatios.

Naming conventions

All variable names and comments use English names (alpha, beta, gamma, sigma) instead of Greek letter Unicode characters. Unicode special characters (arrows, mathematical symbols) replaced with ASCII equivalents throughout.

import { computeNodeSalience, bootstrapHotpath, runPromotionSweep } from "./core/SalienceEngine";
import { DEFAULT_HOTPATH_POLICY } from "./core/HotpathPolicy";

// Compute salience for a single page
const salience = await computeNodeSalience(pageId, metadataStore, DEFAULT_HOTPATH_POLICY);

// Bootstrap: greedily fill hotpath to H(t)
await bootstrapHotpath(metadataStore, DEFAULT_HOTPATH_POLICY, candidatePageIds);

// Steady-state: promote if salience > weakest resident in same tier/community bucket
await runPromotionSweep(candidateIds, metadataStore, DEFAULT_HOTPATH_POLICY);
Original prompt

This section details on the original issue you should resolve

<issue_title>P0-G: Salience Engine (BLOCKS: hotpath promotion in ingest and Daydreamer)</issue_title>
<issue_description>Why: The SalienceEngine is the decision-making layer for hotpath admission. It is needed by ingest (new page admission), query (hit-count update), and Daydreamer (post-LTP/LTD sweeps). Implementing it before ingest ensures promotion logic is correct from the first page written.

  • P0-G1: Implement core/SalienceEngine.ts

    • computeNodeSalience(pageId: Hash, metadataStore: MetadataStore): Promise<number> — fetch PageActivity and incident Hebbian edges; apply σ formula via HotpathPolicy
    • batchComputeSalience(pageIds: Hash[], metadataStore: MetadataStore): Promise<Map<Hash, number>> — efficient batch version
    • shouldPromote(candidateSalience: number, weakestResidentSalience: number, capacityRemaining: number): boolean — admission gating
    • selectEvictionTarget(tier: HotpathEntry['tier'], communityId: string | undefined, metadataStore: MetadataStore): Promise<Hash | undefined> — find weakest resident in tier/community bucket
  • P0-G2: Implement promotion/eviction lifecycle helpers in core/SalienceEngine.ts

    • bootstrapHotpath(metadataStore: MetadataStore, policy: HotpathPolicy): Promise<void> — fill hotpath greedily by salience while resident count < H(t)
    • runPromotionSweep(candidateIds: Hash[], metadataStore: MetadataStore, policy: HotpathPolicy): Promise<void> — steady-state: promote if salience > weakest in same tier/community bucket; evict weakest on promotion
  • P0-G3: Add SalienceEngine test coverage (tests/SalienceEngine.test.ts)

    • Bootstrap fills hotpath to exactly H(t) given enough candidates
    • Steady-state promotes only when candidate beats the weakest resident
    • Steady-state evicts exactly the weakest resident (not a random entry)
    • Community quotas prevent a single community from consuming all page-tier slots
    • Tier quotas prevent one hierarchy level from dominating
    • Eviction is deterministic under the same state

Exit Criteria: SalienceEngine module passes all tests; promotion/eviction lifecycle is correct and deterministic.</issue_description>

Comments on the Issue (you are @copilot in this section)


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Copilot AI and others added 3 commits March 13, 2026 05:35
Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
- Add PageActivity, HotpathEntry, TierQuotas, SalienceWeights, TierQuotaRatios to core/types.ts
- Extend MetadataStore interface with hotpath/activity methods
- Create core/HotpathPolicy.ts: computeCapacity, computeSalience, deriveTierQuotas, deriveCommunityQuotas
- Create core/SalienceEngine.ts: computeNodeSalience, batchComputeSalience, shouldPromote, selectEvictionTarget, bootstrapHotpath, runPromotionSweep
- Extend IndexedDbMetadataStore with hotpath_index and page_activity stores
- Add 39 tests in tests/SalienceEngine.test.ts covering all requirements

Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
…est, add clarity comments

Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement Salience Engine for hotpath promotion P0-G: Implement SalienceEngine — hotpath admission, promotion & eviction Mar 13, 2026
Copilot AI requested a review from devlux76 March 13, 2026 05:45
@devlux76 devlux76 marked this pull request as ready for review March 13, 2026 05:54
Copilot AI review requested due to automatic review settings March 13, 2026 05:54
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces the “hotpath” salience policy + engine layer (Williams Bound) and wires it into storage via new MetadataStore capabilities, with accompanying unit tests to validate capacity/salience math and promotion/eviction behavior.

Changes:

  • Add core/HotpathPolicy.ts (capacity, salience, tier/community quota derivation) and core/SalienceEngine.ts (salience computation + promotion/eviction lifecycle).
  • Extend core/types.ts MetadataStore with hotpath index + page activity APIs, and implement them in storage/IndexedDbMetadataStore.ts with an IndexedDB schema bump to v2.
  • Add comprehensive unit tests in tests/SalienceEngine.test.ts for policy math and lifecycle behavior.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
core/types.ts Adds PageActivity, HotpathEntry, quota/weight types, and new MetadataStore hotpath/activity methods.
core/HotpathPolicy.ts Defines default hotpath policy and implements capacity/salience/quota derivation functions.
core/SalienceEngine.ts Implements salience computation and hotpath promotion/eviction logic (including community-aware logic).
storage/IndexedDbMetadataStore.ts Bumps IndexedDB schema to v2 and adds persistence for hotpath index + page activity.
tests/SalienceEngine.test.ts Adds unit tests for policy + engine behavior and determinism.
.gitignore Ignores package-lock.json.

Comment thread core/SalienceEngine.ts Outdated
Comment thread core/SalienceEngine.ts Outdated
Comment thread core/SalienceEngine.ts Outdated
Comment on lines +313 to +318
// Tier is at quota — promote only if beats weakest in tier
const weakestId = await selectEvictionTarget(tier, communityId, metadataStore);
if (weakestId === undefined) continue;

const weakestEntry = tierEntries.find((e) => e.entityId === weakestId);
const weakestSalience = weakestEntry?.salience ?? 0;
Comment thread core/SalienceEngine.ts
Comment thread core/SalienceEngine.ts Outdated
Comment thread tests/SalienceEngine.test.ts Outdated
@devlux76
Copy link
Copy Markdown
Owner

@copilot apply changes based on the comments in this thread

devlux76 and others added 6 commits March 13, 2026 00:12
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
… cache entries, fix tests

- Enforce tier quotas regardless of capacityRemaining (review comment 2)
- Enforce community quotas regardless of capacityRemaining (review comment 1)
- When tier is full, evict from full tier not scoped to communityId (review comment 3)
- Treat candidate community as at least size 1 for quota derivation (review comment 4)
- Cache hotpath entries in-memory during sweep loop (review comment 5)
- Fix tautological test assertion to verify against actual tier quota (review comment 6)
- Add new community condition: communityCount > 0 so new communities can earn first slot

Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 13, 2026

@copilot apply changes based on the comments in this thread

All 6 review comments addressed in commit fd76cd9:

  1. Community quotas enforced regardless of capacityRemaining — check runs when tierFull, not only when capacity is exhausted
  2. Tier quotas enforced regardless of capacityRemainingtierFull boolean gates eviction independent of overall capacity
  3. Tier-level eviction not scoped to communityIdfindWeakestIn(tierEntries, undefined) selects from the entire tier so new communities can displace weak entries
  4. New communities can earn first slotgetCommunityDistribution uses Math.max(1, actualCount) for quota derivation; communityCount > 0 guard prevents the vacuous 0>=0 from blocking new communities
  5. In-memory cache — single initial getHotpathEntries() read; cachedEntries updated in-place per mutation
  6. Non-tautological assertion — asserts entries.length <= sweepPageQuota computed from same graphMass the sweep uses, with explicit capacity assumption checks

@devlux76
Copy link
Copy Markdown
Owner

@copilot There was apparently some overlap between this and another issue, as a result there was a merge conflict. I attempted to fix it manually but now there are checks failing and this is complex enough, it might be worth a top to bottom redraft of the effected files. While you're in there, please ensure all non-comment characters are called by name, not by greek math symbols. Gamma, beta, rho are much better variable names than their greek letter counterparts.

Copilot AI and others added 2 commits March 13, 2026 06:40
…ith English names

Clean redraft of HotpathPolicy.ts, types.ts, IndexedDbMetadataStore.ts to
resolve broken merge artifacts (duplicate functions, interfaces, syntax errors).

- core/HotpathPolicy.ts: unified HotpathPolicy interface with structured
  salienceWeights/tierQuotaRatios, computeCapacity with optional c param,
  computeSalience using named weights (alpha/beta/gamma not Greek letters),
  deriveTierQuotas, deriveCommunityQuotas with min-1 guarantee
- core/types.ts: single set of PageActivity, HotpathEntry, TierQuotas,
  TierQuotaRatios, SalienceWeights; MetadataStore with removeHotpathEntry
- storage/IndexedDbMetadataStore.ts: single applyUpgrade, single set of
  hotpath methods with removeHotpathEntry and deterministic eviction
- tests/HotpathPolicy.test.ts: updated to structured DEFAULT_HOTPATH_POLICY
- All Greek symbol variables and special Unicode chars replaced with ASCII
  equivalents (sigma, alpha, beta, gamma, ->, ~=)

All 192 tests pass. Build and lint clean.

Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 13, 2026

@copilot There was apparently some overlap between this and another issue, as a result there was a merge conflict. I attempted to fix it manually but now there are checks failing and ...

Done in 5e70e57 and 6dec6fa. All three merge-conflicted files (core/HotpathPolicy.ts, core/types.ts, storage/IndexedDbMetadataStore.ts) redrafted top-to-bottom — duplicate functions, interfaces, and syntax errors from the merge are resolved. Also updated tests/HotpathPolicy.test.ts for the restructured DEFAULT_HOTPATH_POLICY API.

All Greek/Unicode symbols replaced with English names throughout: sigma, alpha, beta, gamma for variables; ASCII -> and ~= for arrows in comments. Build, lint, and all 192 tests pass.

Copilot AI changed the title P0-G: Implement SalienceEngine — hotpath admission, promotion & eviction P0-G: Implement SalienceEngine — hotpath admission, promotion &amp; eviction Mar 13, 2026
@devlux76 devlux76 merged commit 394d6ab into main Mar 13, 2026
2 checks passed
@devlux76 devlux76 deleted the copilot/implement-salience-engine branch March 13, 2026 07:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P0-G: Salience Engine (BLOCKS: hotpath promotion in ingest and Daydreamer)

3 participants