feat(rag): Phase 1.5 — ChatRAGBuilder consumer ordering (#918)

[joelteply]

Builds on PR #920 (Phase 1 — composer-side stable-first ordering). This is the consumer side that completes the prefix-reuse story end-to-end.

What changes

ChatRAGBuilder.buildContext section 2.4 reorders three injections:

1. Tool definitions moved from end → start (after identity). They're INVARIANT — belong in the byte-stable prefix region.
2. Generic source loop unchanged, already iterates Map in insertion order which equals tier-sorted order from Phase 1 (extractFromComposition inserts in result.sections order; that array was tier-sorted by RAGComposer).
3. HumanPresenceTracker injection moved from start → end. Presence is volatile (changes when users switch rooms) — must live in suffix.

Final assembly order:

identity (INVARIANT)
→ tool definitions (INVARIANT)
→ loop in tier order (remaining INVARIANT → SEMI_STABLE → VOLATILE)
→ human presence (VOLATILE)

Why

PR #920 made the section list byte-deterministic. But the consumer (this builder) was injecting volatile content (human presence) BEFORE the tier-sorted loop and INVARIANT content (tool defs) AFTER. Result: the assembled prompt string still had non-stable bytes in the prefix region — Phase 1 alone wasn't enough for actual prefix-reuse.

This commit makes the assembled string byte-identical-prefix across requests for the same persona+recipe. Combined with future Phase 2 (per-persona DMR slot pinning), llama-server/DMR's prefix-KV-cache reuse fires for real and the ~70× prompt-eval speedup actualizes.

Verification

• tsc clean.
• One file changed: system/rag/builders/ChatRAGBuilder.ts, +40 −19.
• Stacks on top of PR feat(rag): Phase 1 — stable-first ordering for prefix-reuse (#918) #920 (feature/prefix-reuse-and-multimodal). Merge after feat(rag): Phase 1 — stable-first ordering for prefix-reuse (#918) #920 lands.
• Runtime test (still gate-blocked by Personas go silent after first response wave — Rust full_evaluate gate or InferenceCoordinator slot leak #919 silence-after-first-wave until PR fix(coord): InferenceCoordinator queues instead of denying — fixes #919 #921 lands): sha256(prompt[:invariant_len]) should be identical across consecutive turns of the same persona.

Sequencing

1. PR fix(coord): InferenceCoordinator queues instead of denying — fixes #919 #921 lands (queue fix for Personas go silent after first response wave — Rust full_evaluate gate or InferenceCoordinator slot leak #919) — unblocks runtime testing
2. PR feat(rag): Phase 1 — stable-first ordering for prefix-reuse (#918) #920 lands (Phase 1)
3. This PR lands (Phase 1.5) — completes the consumer side
4. memento's ModelMetadata refactor: declarative struct, no Option<>, adapter queries its own source #917 lands → unblocks Phase 4
5. Phase 2 (slot pinning) → and the ~70× actually shows up live

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR advances the prompt prefix-reuse work for Chat RAG by introducing tier-aware section ordering and then reordering consumer-side prompt assembly to keep volatile bytes in the suffix (to enable KV-cache prefix reuse).

Changes:

• Add PromptTier and thread tier metadata through RAGSource/RAGSection, injecting tier in RAGComposer and sorting sections deterministically by (tier, sourceName).
• Update all RAG sources to declare a tier and to return Omit<RAGSection, 'tier'> so the composer is the single tier authority.
• Reorder ChatRAGBuilder injections (tool definitions earlier; human presence later) and throttle codebase indexing embedding batches to reduce startup event-loop starvation.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/system/rag/builders/ChatRAGBuilder.ts	Reorders prompt injections to improve stable-prefix behavior (tool defs earlier; presence later).
src/system/rag/services/CodebaseIndexer.ts	Reduces embedding batch size and adds inter-batch pause to yield the event loop.
src/system/rag/shared/RAGComposer.ts	Injects tiers into sections and sorts sections deterministically by tier + name.
src/system/rag/shared/RAGSource.ts	Adds tier to RAGSource/RAGSection, changes load/fromBatchResult return types, and re-exports PromptTier.
src/system/rag/shared/RAGTypes.ts	Introduces PromptTier and documents tier ordering contract.
src/system/rag/sources/ActivityContextSource.ts	Declares tier and updates load() return type to omit tier.
src/system/rag/sources/CodeToolSource.ts	Declares tier and updates load()/helpers to omit tier.
src/system/rag/sources/CodebaseSearchSource.ts	Declares tier and updates load() return type to omit tier.
src/system/rag/sources/ConversationHistorySource.ts	Declares tier and updates load()/helpers to omit tier.
src/system/rag/sources/DocumentationSource.ts	Declares tier and updates load() return type to omit tier.
src/system/rag/sources/GlobalAwarenessSource.ts	Declares tier and updates load()/fromBatchResult()/helpers to omit tier.
src/system/rag/sources/GovernanceSource.ts	Declares tier and updates load() return type to omit tier.
src/system/rag/sources/LiveRoomAwarenessSource.ts	Declares tier and updates load()/helpers to omit tier.
src/system/rag/sources/MediaArtifactSource.ts	Declares tier and updates load() return type to omit tier.
src/system/rag/sources/OpenProposalsSource.ts	Declares tier and updates EMPTY_SECTION/load() to omit tier.
src/system/rag/sources/PersonaIdentitySource.ts	Declares tier and updates load()/helpers to omit tier.
src/system/rag/sources/ProjectContextSource.ts	Declares tier and updates cached section types + load()/helpers to omit tier.
src/system/rag/sources/SemanticMemorySource.ts	Declares tier and updates load()/fromBatchResult()/helpers to omit tier.
src/system/rag/sources/SentinelAwarenessSource.ts	Declares tier and updates load() return type to omit tier.
src/system/rag/sources/SocialMediaRAGSource.ts	Declares tier and updates load()/helpers to omit tier.
src/system/rag/sources/ToolDefinitionsSource.ts	Declares tier and updates load()/helpers to omit tier.
src/system/rag/sources/ToolMethodologySource.ts	Declares tier and updates load() return type to omit tier.
src/system/rag/sources/VoiceConversationSource.ts	Declares tier and updates load()/helpers to omit tier.
src/system/rag/sources/WidgetContextSource.ts	Declares tier and updates load()/helpers to omit tier.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The file imports and re-exports PromptTier as a runtime symbol (import { PromptTier } ... + export { PromptTier } ...), but PromptTier is a const enum (erased at emit). In ESM this can cause a hard runtime failure when the re-exported binding doesn’t exist. Safer options: (a) make PromptTier a normal enum, or (b) remove the runtime re-export and have consumers import type { PromptTier } / use string literals.

Suggested change

	import type { RAGBuildOptions, LLMMessage, RAGArtifact, PersonaMemory, PersonaIdentity, RecipeStrategy } from './RAGTypes';
	import { PromptTier } from './RAGTypes';
	// Re-export so source files only need one import
	export { PromptTier } from './RAGTypes';
	import type { RAGBuildOptions, LLMMessage, RAGArtifact, PersonaMemory, PersonaIdentity, RecipeStrategy, PromptTier } from './RAGTypes';
	// Re-export so source files only need one import
	export type { PromptTier } from './RAGTypes';

[Copilot]

ToolDefinitionsSource is marked PromptTier.INVARIANT, but its output explicitly depends on context.options.currentMessage (contextual group selection + group hints). That makes the section change turn-to-turn, which defeats the stable-prefix goal and violates the tier contract. Either make tool definitions truly invariant (no currentMessage-dependent selection) or split into an invariant “tool catalog/specs” source and a volatile “tool hints for this turn” source (and mark this one SEMI_STABLE/VOLATILE accordingly).

[Copilot]

The PromptTier doc says “Within each tier, sources are sorted by name (alphabetical)”, but ChatRAGBuilder now explicitly hoists tool-definitions ahead of other INVARIANT sources. Either update this doc to reflect the actual consumer-side ordering rules, or move all ordering decisions into the composer so the documented contract matches the emitted prompt order.

Suggested change

	* Within each tier, sources are sorted by name (alphabetical) so the byte
	* order is fully deterministic.
	* Within a tier, source order must be deterministic. Consumers may apply
	* tier-specific ordering rules before any fallback alphabetical ordering; for
	* example, `tool-definitions` is hoisted ahead of other INVARIANT sources.

[Copilot]

PromptTier is declared as a const enum, but it’s also imported/re-exported as a value (export { PromptTier } ...). Since const enums are erased in JS output, this can break ESM/bundler consumers with “module does not provide an export named 'PromptTier'”. Consider switching PromptTier to a normal export enum (or as const object + union type), or make the re-export/imports type-only and stop re-exporting it as a runtime value.

[joelteply]

Reverted via #926 — bisecting a silence regression observed on clean main after tonight's chain merge (#921 → #923 → #920 → #922). #922 was the most recent merge so reverted first. If retest after revert shows main responds, this PR's logic introduced the regression and the consumer-side reordering needs rework before re-merging. Original commits still in git history ().