Skip to content

feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes - #245

Open
iceteaSA wants to merge 2 commits into
cortexkit:masterfrom
iceteaSA:cache-ttl-never
Open

feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes#245
iceteaSA wants to merge 2 commits into
cortexkit:masterfrom
iceteaSA:cache-ttl-never

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Problem

cache_ttl assumes idle > TTL means the provider evicted the prompt cache, so the next prefix rebuild is free. Two consumers act on that assumption:

  1. the scheduler converts a defer pass into an execute pass (scheduler.ts), and
  2. mustMaterialize fires the ttl_idle HARD fold (via hardCacheExpired in transform.ts).

On lanes kept warm by an external keepwarm mechanism (prewarm proxies, dedicated cache-keep tools that re-warm the provider cache out-of-band), the assumption is wrong: the cache never goes cold, MC's lastResponseTime goes stale anyway, and both consumers false-positive. MC then initiates a rebuild it believes is free that is actually a full paid cache-write — measured 450-560K tokens per fold on large sessions. There was no way to express "this lane never goes cold": the config requires a duration.

Fix

cache_ttl: "never" (case-insensitive, works as the string form or any per-model value):

  • parseCacheTtl("never") returns Infinity; both consumers go inert through the existing comparisons (elapsed > Infinity / elapsed >= Infinity are never true) — no new branches in the hot path.
  • The Rust scheduler mirrors the sentinel with u64::MAX (its predicates already use saturating_sub, so no overflow path).
  • Status surfaces render honestly instead of a bogus countdown: /ctx-status shows never expires (always-warm lane); the TUI sidebar gets a JSON-safe cacheNeverExpires flag on StatusDetail (Infinity does not survive JSON-RPC); Pi's status dialog now uses the shared parseCacheTtl (it previously used a private fallback parser that would have shown a 5m countdown).
  • hardCacheExpired is extracted into a pure computeHardCacheExpired helper so the "never" -> Infinity -> false chain has direct unit coverage instead of only flag-consumption coverage.

Docs: CONFIGURATION.md describes the sentinel, what it disables, and the tradeoff — after a genuinely cold start (e.g. the keepwarm process died), the free-fold window is not detected on such lanes; mutations then apply at the execute threshold.

Verification

  • plugin: 2995 pass / 0 fail; pi-plugin: 666 pass / 0 fail; tsc --noEmit clean on both
  • new tests red-verified: reverting the sentinel fails the scheduler tests and the computeHardCacheExpired test
  • Rust: sentinel + predicate + scheduler-decision tests added in scheduler.rs's test module (note: I could not run cargo test locally — the crates workspace has path deps outside this repo; the change is a 3-line early return mirroring the TS logic)
  • schema + generated docs regenerated (build-schema / build-config-docs drift tests green); check:tui-compiled green

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Adds cache_ttl: "never" to disable the idle‑TTL heuristic on always‑warm lanes, preventing false executes and expensive cache rebuilds. Also fixes status serialization and restores invalid‑TTL diagnostics.

  • New Features

    • parseCacheTtl("never") returns Infinity (TS) / u64::MAX (Rust) so idle‑execute and ttl_idle never fire; computeHardCacheExpired exported for consistent checks with onInvalid.
    • Status UIs show “never expires” using a JSON‑safe cacheNeverExpires flag; Pi now uses the shared parser; docs and schema updated.
  • Bug Fixes

    • Status RPC keeps cacheRemainingMs numeric (0) when TTL is “never” and keys on cacheNeverExpires.
    • Transform restores invalid‑TTL outcomes and session logs via the onInvalid callback.

Written for commit 222c184. Summary will update on new commits.

Review in cubic

Greptile Summary

Adds and consistently surfaces the always-warm cache_ttl: "never" configuration.

  • Maps the sentinel to non-expiring TTL values in the TypeScript and Rust schedulers.
  • Prevents idle-TTL scheduling and hard-fold triggers for always-warm lanes.
  • Preserves invalid-TTL diagnostics in the extracted hard-expiry helper.
  • Adds a JSON-safe RPC flag and updates status interfaces, tests, schema, and documentation.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain.

Important Files Changed

Filename Overview
packages/plugin/src/plugin/rpc-handlers.ts Represents non-expiring cache status with a JSON-safe boolean while keeping numeric RPC fields finite.
packages/plugin/src/hooks/magic-context/transform.ts Extracts hard cache-expiry computation, supports the non-expiring sentinel, and preserves invalid-TTL diagnostics.
packages/plugin/src/features/magic-context/scheduler.ts Parses the case-insensitive sentinel as an infinite TTL so existing comparisons suppress idle execution.
crates/mc-module/src/scheduler.rs Mirrors non-expiring TTL semantics with u64::MAX and adds predicate and scheduler coverage.
packages/plugin/src/shared/rpc-types.ts Extends status details with the optional JSON-safe non-expiring-cache indicator.
packages/pi-plugin/src/dialogs/status-dialog.ts Uses shared TTL parsing and renders always-warm lanes without a misleading countdown.

Reviews (3): Last reviewed commit: "fix(scheduler): keep cacheRemainingMs JS..." | Re-trigger Greptile

Comment thread packages/plugin/src/plugin/rpc-handlers.ts Outdated
Comment thread packages/plugin/src/hooks/magic-context/transform.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/plugin/src/hooks/magic-context/transform.ts
Comment thread packages/plugin/src/plugin/rpc-handlers.ts Outdated
@iceteaSA

Copy link
Copy Markdown
Contributor Author

Note on the red Check (dashboard): it's pre-existing on master, not introduced here — this PR touches no dashboard files. The same 2 ConfigEditor ⇄ schema parity failures reproduce on a pristine upstream/master checkout at v0.33.0 (48ab531d):

- []
+ [
+   "experimental.mural.enabled",
+   "experimental.mural.model",
+   "fail_closed_blocking",
+   "pi.subagent_extensions",
+ ]

The v0.33.0 schema added those four leaves without updating the dashboard's RENDERED_PREFIXES/OMITTED_BY_DESIGN lists, and experimental.mural.* also trips the "nothing re-introduces the dead experimental.* namespace" test. Happy to send a separate fix PR for the parity lists if useful — the right classification (render vs omit-by-design, and whether mural should live under experimental.) is a maintainer call.

Tehan added 2 commits July 26, 2026 08:47
Lanes kept warm by external keepwarm mechanisms (prewarm proxies,
dedicated cache-keep tools) re-warm the provider prompt cache
out-of-band, so MC's idle>TTL heuristic false-positives: both TTL
consumers (scheduler idle-execute and the ttl_idle m[0] fold) initiate
a rebuild believed free that is actually a full paid cache-write
(measured 450-560K tokens on large sessions).

cache_ttl: "never" (string or per-model value, case-insensitive)
disables both consumers: parseCacheTtl returns Infinity, so
elapsed>ttl / elapsed>=ttl never fire. Rust scheduler mirrors the
sentinel with u64::MAX (predicates already use saturating_sub).
Status surfaces render "never expires (always-warm lane)" instead of
a bogus countdown: /ctx-status, the TUI sidebar (JSON-safe
cacheNeverExpires flag on StatusDetail), and Pi's status dialog
(which previously parsed the TTL with a private fallback parser).
hardCacheExpired extracted to a pure computeHardCacheExpired helper
with direct coverage of the never-chain.

Tradeoff documented in CONFIGURATION.md: on a genuinely cold start
the free-fold window is not detected on such lanes; mutations then
apply at the execute threshold.
…TL diagnostics

- rpc-handlers: the cacheNeverExpires branch assigned Infinity to
  cacheRemainingMs; JSON.stringify converts Infinity to null over RPC,
  violating the numeric StatusDetail contract. Use 0 and let the
  cacheNeverExpires flag carry the semantics (the TUI keys on it first).
- computeHardCacheExpired: the extraction dropped the
  invalid-cache-ttl-fallback pass outcome and session log on parse
  failure. Add an onInvalid callback; the transform call site restores
  the exact pre-extraction record + log.
- test: seed last_response_time in the never-TTL status test — the
  guarded branch only runs when lastResponseTime > 0, so the assertion
  was vacuous without it (verified red: Infinity revert now fails it).
iceteaSA pushed a commit to iceteaSA/magic-context that referenced this pull request Jul 26, 2026
Brings the cache_ttl "never" sentinel (PR cortexkit#245) into the live union
branch so the always-warm lanes can drop the 999h workaround.

Resolutions:
- rpc-handlers.ts: both imports (getSkillMemoryStats + parseCacheTtl).
- rpc-handlers.test.ts / execute-status.test.ts: the PR branch was cut
  before external-memory made buildStatusDetail and executeStatus async,
  so its two new tests called them synchronously and asserted against a
  Promise. Added the awaits; the external-memory describe block also lost
  its closing braces to the merge and was restored.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant