Skip to content

feat(genome): demand-aligned-recall PR-3c — trait impl + CandidateSource seam#1374

Merged
joelteply merged 1 commit into
canaryfrom
feat/demand-aligned-recall-trait-impl-pr3c
May 17, 2026
Merged

feat(genome): demand-aligned-recall PR-3c — trait impl + CandidateSource seam#1374
joelteply merged 1 commit into
canaryfrom
feat/demand-aligned-recall-trait-impl-pr3c

Conversation

@joelteply
Copy link
Copy Markdown
Contributor

@joelteply joelteply commented May 17, 2026

Summary

PR-3c of demand-aligned-recall. Wires DemandAlignedRecall trait impl on LocalDemandAlignedRecall + introduces CandidateSource trait as the seam between the ranking engine and the substrate candidate sources.

PR-3d will wrap the working-set-manager (my #1362's bus hook) as a CandidateSource impl; PR-3c stays substrate-agnostic so the seam is independently reviewable.

What lands

  • CandidateSource trait — async fn fetch(query, context) -> Vec<CandidateArtifact>. Send + Sync + async_trait for tokio. Object-safe.
  • LocalDemandAlignedRecall.source: Option<Arc<dyn CandidateSource>> — optional injection.
  • with_source(source) + with_config_and_source(weights, half_life, source) constructors.
  • DemandAlignedRecall trait impl on LocalDemandAlignedRecall:
    • recall(query, context) — fetches via source, scores via rank() with SystemTime::now() (rank() stays pure with explicit now_ms for replay determinism)
    • replay(trace) — returns typed RecallError::ScopeUnreachable with "RecallTraceStore (sentinel PR); not yet implemented in PR-3c". Typed refusal beats silent empty pool. When sentinel ships RecallTraceStore, this test flips to expect Ok(pool).

Design choices

  • Source is Option, not required. None = empty-pool mode (legitimate "no candidates locally; try federation" signal). Some = recall() dispatches.
  • recall() reads SystemTime::now at the trait entry. Internal rank() keeps explicit now_ms for replay determinism. Decouples live-recall vs replay paths cleanly.
  • PR-3c scope: no scope filtering, no freshness enforcement, no budget filtering. The CandidateSource does query-aware pruning in its fetch(); PR-3d's working-set walker filters by RecallScope::Local. Future PRs add the rest.

Test plan

  • cargo test --lib --features metal,accelerate genome::recall_impl — 18/18 pass (13 PR-3b + 5 PR-3c):
    • recall_dispatches_through_dyn_demand_aligned_recall — Arc<dyn> object-safety
    • recall_without_source_returns_empty_pool_not_error — empty-pool contract
    • recall_with_source_dispatches_to_fetch_and_ranks — fetch call count + candidate-in-pool round-trip
    • with_config_and_source_preserves_all_three
    • replay_returns_typed_not_implemented_refusal_in_pr3c — pins the typed refusal so sentinel PR has a regression check to flip
  • No regressions across other 2802 lib tests
  • Pre-push gate clean

Stack

🤖 Generated with Claude Code

@claude
feat(genome): demand-aligned-recall PR-3c — trait impl + CandidateSou…
078d076 
…rce seam

PR-3c of demand-aligned-recall. Wires `DemandAlignedRecall` trait
impl on `LocalDemandAlignedRecall` + introduces `CandidateSource`
trait as the seam between the ranking engine and the substrate
candidate sources. PR-3d will wrap the working-set-manager
(#1362's bus hook) as a CandidateSource impl; PR-3c stays
substrate-agnostic.

Why this split

PR-3c locks the source seam first. PR-3d adds the working-set
walker as one impl; future PRs add the genome catalog walker +
federation peer source. Each is independently testable.

What lands

- CandidateSource trait — async fn fetch(query, context) ->
  Vec<CandidateArtifact>. Send + Sync + async_trait for tokio.
  Object-safe; PR-3d's working-set walker is one impl.
- LocalDemandAlignedRecall.source: Option<Arc<dyn CandidateSource>>
  — optional injection. None = empty-pool mode (legitimate "no
  candidates locally; try federation" signal). Some = trait
  impl's recall() dispatches to source.fetch() then rank().
- with_source(source) constructor.
- with_config_and_source(weights, half_life, source) constructor
  for governor-driven config + source wiring.
- DemandAlignedRecall trait impl on LocalDemandAlignedRecall:
  - recall(query, context) — fetches via source, scores via rank()
    with SystemTime::now() (rank() stays pure with explicit
    now_ms threading for replay determinism)
  - replay(trace) — returns typed RecallError::ScopeUnreachable
    with "RecallTraceStore (sentinel PR); not yet implemented in
    PR-3c". Per never-swallow-errors: typed refusal beats silent
    empty pool. When sentinel ships RecallTraceStore, this test
    flips to expect Ok(pool).

Design choices

- Source is Option, not required. The no-source path returns
  empty — useful for unit tests that don't need substrate +
  diagnostic tooling that wants a recall engine without
  candidate plumbing.
- `recall()` reads SystemTime::now at the trait entry. The
  internal rank() still takes explicit now_ms; replay
  determinism preserved at the pure layer, live recall at the
  trait layer. This is the cleanest decoupling I could find that
  satisfies both spec asks.
- PR-3c scope: no scope filtering, no freshness enforcement, no
  budget filtering. The CandidateSource does query-aware pruning
  in its fetch(); PR-3d's working-set walker filters by
  RecallScope::Local. Future PRs add the rest.

Tests

5 new tests on the PR-3c surface:
- recall_dispatches_through_dyn_demand_aligned_recall — Arc<dyn>
  object-safety
- recall_without_source_returns_empty_pool_not_error — empty-pool
  contract
- recall_with_source_dispatches_to_fetch_and_ranks — fetch call
  count + candidate-in-pool round-trip
- with_config_and_source_preserves_all_three
- replay_returns_typed_not_implemented_refusal_in_pr3c — pins the
  typed refusal so sentinel PR has a regression check to flip

18/18 pass on genome::recall_impl (13 PR-3b + 5 PR-3c). No
regressions across other 2802 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my genome stack
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- THIS PR — DAR PR-3c: trait impl + CandidateSource seam
- NEXT — DAR PR-3d: WorkingSetCandidateSource wrapping #1362's
  bus hook + concrete walker for the persona's working set

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply merged commit 6dadf0d into canary May 17, 2026
3 checks passed
@joelteply joelteply deleted the feat/demand-aligned-recall-trait-impl-pr3c branch May 17, 2026 04:04
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3d — WorkingSetCandidateSource (…
90e0896 
…#1378)

The architectural payoff of the genome stack lands here. A persona's
page_in calls populate the working set (#1355); this source reads
that same working set to surface "what's already hot" candidates
that LocalDemandAlignedRecall (#1372 + #1374) ranks via the scoring
function (#1371).

End-to-end loop closed:
  page_in(persona, page) → WorkingSet.pages updated → bus publishes
  PageFault (#1362) → recall(query, ctx) → working_set_snapshot →
  CandidateArtifact per resident page → rank() → RankedPool

What lands

- WorkingSetCandidateSource struct holding
  Arc<LocalWorkingSetManager>
- CandidateSource::fetch impl that:
  - reads persona's working_set_snapshot
  - returns empty Vec on unregistered persona (no error — cold-
    start signal callers may try federation)
  - translates each ResidentPage → CandidateArtifact with
    ResidencyHint::Hot { role } (resident = hot by definition)
  - preserves PageKind for downstream sub-pool partitioning
  - sets NEUTRAL_FACTOR_STUB (0.5) for semantic / outcome_history
    / provenance_trust factors (dedicated integrations land in
    separate PRs)
- NEUTRAL_FACTOR_STUB public constant for the contract

Design choices

- Snapshot the working set via the manager's working_set_snapshot
  helper (cloned) rather than holding the RwLock across the fetch
  await. Same pattern as #1362's bus_arc hook.
- Object-safe: works through Arc<dyn CandidateSource> per PR-3c's
  contract.
- All resident pages map to Hot residency. PR-3e (or a separate
  catalog walker PR) will add Local{role=Bench/Cold/Frozen} for
  candidates outside the working set but resident in the genome
  catalog.
- Stub-0.5 factors documented inline + via NEUTRAL_FACTOR_STUB
  constant. When the embedding / sentinel / trust integrations
  land, they replace the stubs without re-touching this file.

What is deliberately deferred

- Genome catalog walker (Bench/Cold/Frozen tier sources) — needs
  the catalog module
- Federation peer source — needs federation registry
- Embedding integration (semantic factor) — separate Lane H slice
- Sentinel outcome lookup (outcome_history factor) — sentinel PR
- Trust registry lookup (provenance_trust factor) — separate PR

Tests

7 new tests, all end-to-end with real LocalWorkingSetManager +
page_in calls:
- fetch_unregistered_persona_returns_empty_not_error
- fetch_registered_empty_working_set_returns_empty
- fetch_after_page_in_returns_resident_pages_as_hot_candidates —
  the payoff test
- translation_preserves_page_kind_for_sub_pool_partitioning —
  layer → layers, expert → experts, engram → engrams
- translation_uses_neutral_factor_stubs_for_non_tier_factors —
  pins the contract so embedding-integration PRs flip it
- source_is_object_safe_for_arc_dyn_dispatch — through PR-3c's
  Arc<dyn CandidateSource>
- end_to_end_page_in_then_recall_returns_ranked_pool — full
  pipeline: page_in → WorkingSetCandidateSource ::fetch →
  LocalDemandAlignedRecall::recall → RankedPool with the
  paged-in artifacts ranked correctly

7/7 pass. No regressions across other 2822 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- THIS PR — DAR PR-3d: WorkingSetCandidateSource (the payoff)

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3e — CompositeCandidateSource (#…
ae7db5f 
…1380)

Combines multiple CandidateSource impls into one, with optional
deduplication by artifact id. Sets up the extensibility seam so
future PRs (genome catalog walker, federation peer source,
must-include resolver) add sources without re-wiring
LocalDemandAlignedRecall.

What lands

- CompositeCandidateSource { sources, dedup }
- DedupPolicy::None — return all candidates from all sources (a
  single artifact may appear N times if N sources surface it).
  Useful for audit-trail callers.
- DedupPolicy::ByArtifactId — keep first occurrence per (kind,
  artifact_id) tuple in source-iteration order. Most callers want
  this (prevents double-counting a resident page that also
  surfaces via federation lookup).
- CandidateSource::fetch impl: fans out to all sources
  concurrently via futures::future::join_all, merges, dedups.
- new(sources, dedup) + with_default_dedup(sources) constructors.
- source_count() + dedup_policy() inspector methods.

Design choices

- futures::future::join_all for fan-out (concurrent, unbounded).
  Acceptable for ≤5 sources currently; federation peer counts may
  need bounding later — when that happens, this fn changes
  internals without breaking the trait.
- Dedup is configurable per composite. Most production wiring
  uses ByArtifactId; replay traces may use None for audit fidelity.
- Different PageKind with same artifact_id treated as distinct
  candidates (a layer-page reference and an engram-page reference
  happen to share the underlying artifact id; recall keeps them
  separate so the sub-pool partitioning is correct).
- Composite itself is object-safe — composites of composites
  valid for future hierarchical wiring.

What is deliberately deferred

- Source priority ordering — first-hit-wins per dedup. A future
  PR may add weighted merging.
- Per-source error isolation — fetch returns Vec, not Result. The
  underlying trait method also returns Vec; widening the trait
  would be a separate concern.
- Bounded concurrent fan-out — join_all is unbounded. Fine for
  the current source count; needs revisit when federation peers
  scale.

Tests

9 new tests pin the composite's behaviors:
- empty_composite_returns_empty_vec — no-error empty contract
- single_source_composite_passes_through — degenerate case
- fan_out_invokes_every_source_exactly_once — per-call accounting
- merge_preserves_source_iteration_order — dedup correctness
  depends on this
- dedup_none_preserves_all_duplicates
- dedup_by_artifact_id_keeps_first_occurrence_only
- dedup_treats_different_page_kinds_as_distinct
- with_default_dedup_uses_by_artifact_id
- composite_is_object_safe_as_dyn_candidate_source

9/9 pass. No regressions across other 2834 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- #1378 — DAR PR-3d: WorkingSetCandidateSource
- THIS PR — DAR PR-3e: CompositeCandidateSource (extensibility seam)
- NEXT — DAR PR-3f or later: catalog walker + federation source +
  must-include resolver, all composing through this PR's seam

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@joelteply joelteply mentioned this pull request May 17, 2026
Merged
3 tasks
joelteply added a commit that referenced this pull request May 17, 2026
@joelteply @claude
feat(genome): demand-aligned-recall PR-3f — MustIncludeCandidateSource (
c8b11d9 
#1382)

Resolves CapabilityQuery.must_include hard pins as candidates per
GENOME-FOUNDRY-SENTINEL Part 7: "Hard pins — recall MUST include
these in the RankedPool even if their score is low. Used for
persona-private LoRA layers and sticky engrams."

Plays through the composite seam shipped in PR-3e: wired AFTER a
resident source like WorkingSetCandidateSource with ByArtifactId
dedup, must-include items that ARE resident get the resident
source's Hot residency + factor data; must-include items NOT
resident get this source's NotResident placeholder (still ranked,
just lower combined score).

What lands

- MustIncludeCandidateSource — zero-state unit struct (no Arc state
  needed; the source is pure-function over the query)
- CandidateSource::fetch impl that:
  - reads query.must_include Vec<ArtifactRef>
  - maps each variant (LoRALayer / MoEExpert / Engram) to a
    CandidateArtifact with the appropriate PageKind
  - marks every must-include candidate as ResidencyHint::
    NotResident { acquirable_from: SentinelRefinement }
  - uses NEUTRAL_FACTOR_STUB (0.5) for the three non-tier factors,
    same convention as WorkingSetCandidateSource (PR-3d)

Recommended composite wiring

  let composite = CompositeCandidateSource::with_default_dedup(vec![
      Arc::new(WorkingSetCandidateSource::new(mgr)),     // Hot first
      Arc::new(MustIncludeCandidateSource::new()),       // Pins
      // future: catalog walker, federation source
  ]);

Spec contract met: every hard-pinned artifact surfaces in the
RankedPool; if it's resident, it gets full residency-aware score;
if not, it still appears (at lower combined) so composition can
see "this was pinned but isn't here yet — schedule the foundry."

Tests

6 new tests:
- empty_must_include_returns_empty_candidates (no-error empty
  contract)
- variant_mapping_preserves_page_kind (LoRALayer/MoEExpert/Engram
  variants → PageKind mapping)
- must_include_marks_candidates_as_not_resident
- factors_use_neutral_stubs_consistent_with_working_set_source
- source_is_object_safe_for_dyn_dispatch
- composite_with_dedup_resident_wins_must_include_for_pinned_hot_
  artifact — the architectural payoff: resident pin keeps Hot,
  non-resident pin gets NotResident, both appear in merged Vec

6/6 pass. No regressions across other 2873 lib tests.

Stack

- #1346 / #1353 / #1355 / #1358 / #1362 — my working-set-manager
- #1366 — DAR PR-1: pure types
- #1367 + #1370 — DAR PR-2: trait + composite types
- #1371 — DAR PR-3a: scoring function + per-factor curves
- #1372 — DAR PR-3b: LocalDemandAlignedRecall ranking engine
- #1374 — DAR PR-3c: trait impl + CandidateSource seam
- #1378 — DAR PR-3d: WorkingSetCandidateSource (working-set source)
- #1380 — DAR PR-3e: CompositeCandidateSource (extensibility seam)
- THIS PR — DAR PR-3f: MustIncludeCandidateSource (hard-pin source)

Co-authored-by: Test <test@test.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

size: L

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@joelteply