feat(orchestration): plan template caching for LLM planner cost reduction (#1856)#2068
Merged
feat(orchestration): plan template caching for LLM planner cost reduction (#1856)#2068
Conversation
github-actions
bot
added
documentation
bug-ops
added a commit
that referenced
this pull request
Mar 20, 2026
Resolves CI build failure in PR #2068 where blake3::hash() was used without importing the blake3 crate. Local builds passed due to transitive imports, but clean CI builds revealed the missing import.
Store completed TaskGraph plans as PlanTemplate skeletons in SQLite, matched via in-process cosine similarity on goal embeddings. On a cache hit, templates are adapted via a single LLM call instead of full re-decomposition, reducing planner cost for repeated or similar goals. Key implementation details: - BLAKE3 goal hash for deduplication with INSERT OR REPLACE semantics - Unicode-aware goal normalization (trim + collapse whitespace + lowercase) - Embedding stored as LE-encoded f32 BLOB (bytemuck-free, chunk-based decode) - Embedding model invalidation on PlanCache::new() startup - TTL + LRU two-phase eviction - plan_with_cache() wrapper with graceful degradation on any cache error - PlanCacheConfig::validate() with bounds checked on config load - Migration 040_plan_cache.sql with UNIQUE constraint on goal_hash
Resolves CI build failure in PR #2068 where blake3::hash() was used without importing the blake3 crate. Local builds passed due to transitive imports, but clean CI builds revealed the missing import.
The cast_signed() method doesn't exist in Rust's standard library. Replaced with standard as i64 cast, with clippy allow for safe Unix timestamp conversion that won't overflow for centuries.
…egration test The plan caching feature added `plan_cache: PlanCacheConfig` to `OrchestrationConfig`. The integration test's `default_config()` helper constructed the struct via an exhaustive literal and did not include the new field, causing E0063 in CI (which compiles `--tests`) while local `--lib --bins` runs were unaffected. Use struct update syntax (`..OrchestrationConfig::default()`) to fill any default fields so future additions do not require updating this helper.
bug-ops
force-pushed
the
issue-1856-plan-caching
branch
from
7954e10 to
88fa5d5
Compare
This was referenced Mar 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TaskGraphplans asPlanTemplateskeletons in SQLite (plan_cachetable, migration 040)Planner::decompose()Implementation
ch.to_lowercase()iteration)INSERT OR REPLACE ON CONFLICT(goal_hash)flat_map(|f| f.to_le_bytes())/chunks_exact(4)+f32::from_le_bytes(no bytemuck)PlanCache::new(), rows with a differentembedding_modelare deletedmax_templates) on every store[orchestration.plan_cache]section withenabled,similarity_threshold,ttl_days,max_templates;PlanCacheConfig::validate()enforces sane bounds on config loadFiles changed
crates/zeph-orchestration/src/plan_cache.rs— new (1000 lines, 23 tests)crates/zeph-memory/migrations/040_plan_cache.sql— new migrationcrates/zeph-config/src/experiment.rs—PlanCacheConfig+validate()crates/zeph-config/src/loader.rs— callplan_cache.validate()when enabledcrates/zeph-orchestration/src/planner.rs— exposeconvert_response_pub()for adapt_plancrates/zeph-orchestration/src/lib.rs— re-export public APIcrates/zeph-orchestration/Cargo.toml— add blake3, sqlx depsTest plan
cargo +nightly fmt --check— cleancargo clippy -p zeph-orchestration -p zeph-config -- -D warnings— cleancargo nextest run -p zeph-orchestration -p zeph-config— 232 passed, 0 skippedconfig/testing.tomlandorchestration.plan_cache.enabled = trueto verify end-to-end cache hit logging