diff --git a/README.md b/README.md index a3cf8d2..320cd0a 100644 --- a/README.md +++ b/README.md @@ -45,13 +45,13 @@ those forms. `--json` enables machine-readable NDJSON output for non-interactive usage (one complete JSON object per processed input line). -Preload options keep authoritative state and runtime continuation separate: -- `--initial-state-json` / `--initial-state-file` load authoritative state +Preload options keep saved rules separate from in-progress confirmation state: +- `--initial-state-json` / `--initial-state-file` load saved state (via exported state JSON). - `--initial-checkpoint-json` / `--initial-checkpoint-file` restore full - checkpoint continuation (authoritative state + pending clarification state). + continuation checkpoint (saved state + pending confirmation state). -REPL command-layer commands (host/controller layer, not engine directives): +REPL commands (controller layer, not engine directives): - `state` shows current authoritative state. - `preview ` runs deterministic dry-run without mutating live state. - `step ` is an explicit alias of normal bare-input step behavior. @@ -113,6 +113,20 @@ The idea is similar to a traditional compiler: user directives are translated in --- +## FAQ + +**Is this just prompt reinjection?** +Partly. Hosts still pass state to models as context. The difference is that +state is maintained by a deterministic engine with explicit update rules, +clarification behavior, and inspectable checkpoints. + +**Isn’t this just prompt engineering?** +It complements prompt engineering, but solves a different problem. Prompting +shapes model behavior; Context Compiler provides a deterministic state layer +that updates only through explicit directives. + +--- + ## 10-Second Example User sets a constraint once: @@ -181,7 +195,7 @@ Host Application ``` The compiler owns state updates and never calls the LLM. -The host decides whether to call the model based on the returned `Decision`. +Your app decides whether to call the model based on the returned `Decision`. --- @@ -227,8 +241,8 @@ Meaning: ### Controller API (Reusable Outside REPL) -These controller-layer APIs are public package exports and can be used directly -in host code (not just inside the REPL). +These controller APIs are public package exports and can be used directly +in app code (not just inside the REPL). | API | Description | |---|---| @@ -337,6 +351,18 @@ Use policies instead when the constraint is explicit and enforceable: - “prohibit introducing new external dependencies” - “use single-step preparation methods” +### Example domains + +Hosts define what policy items and premise mean in context. Common patterns: + +- safety-oriented constraints (for example, prohibited materials or tools) +- authority/evidence constraints (for example, cite only approved sources) +- software workflow constraints (for example, require `uv`, prohibit `npm`) +- accessibility/environment constraints (for example, no audio-only outputs) + +Context Compiler enforces explicit directive/state mechanics. Domain reasoning +still belongs to the host and model workflow. + --- ## Directive Examples @@ -381,6 +407,10 @@ For full directive grammar and edge-case behavior, see [DirectiveGrammarSpec.md] - [demos](demos/) — concrete scenarios showing how behavior differs with and without the compiler - [integrations](examples/integrations/) — production-style host integrations (OpenWebUI, LiteLLM, etc.) +Integration note: current OpenWebUI example pipes return deterministic local +acknowledgements for directive-only `update` decisions instead of forwarding +those turns to the downstream LLM. + --- ## Guarantees diff --git a/docs/DescriptionAndMilestones.md b/docs/DescriptionAndMilestones.md index 42c8e7a..f83ee5f 100644 --- a/docs/DescriptionAndMilestones.md +++ b/docs/DescriptionAndMilestones.md @@ -10,7 +10,7 @@ conversations, and state can conflict over time. This project adds a deterministic state layer that is independent of the model. The model handles interpretation and generation; the engine handles premise and policies. Only explicit user directives can change state. -By separating reasoning from state authority, the system improves reliability +By separating reasoning from state ownership, the system improves reliability without requiring model retraining. The system never derives authoritative state from model responses. The goal is not to make the model smarter, but to make interactions @@ -26,7 +26,7 @@ The state engine is the source of truth and is model-independent. Model output is never interpreted to derive or modify state. All state transitions originate from explicit user directives. -Behavioral details are authoritative in `docs/DirectiveGrammarSpec.md`. +Behavioral details are defined in `docs/DirectiveGrammarSpec.md`. ## Project Milestones @@ -45,7 +45,7 @@ The current authoritative state shape and directive semantics are defined in `Di - Apply explicit state changes as deterministic replacements - Block ambiguous updates until clarified - Maintain a source-of-truth state that does not depend on prior model wording -- Provide structured state for host-provided model context +- Provide structured state for app-provided model context **Deliverables:** @@ -53,7 +53,7 @@ The current authoritative state shape and directive semantics are defined in `Di - State data model (authoritative conversational state) - Deterministic update rules for explicit directives and clarification - Clarification mechanism for ambiguous mutations -- Context serialization interface (`export_json` / `import_json`, state → host application) +- Context serialization interface (`export_json` / `import_json`, state → app layer) - Reference integration harness (example host) - Tests: persistence and non-regression of deterministic state updates @@ -64,7 +64,7 @@ After correcting or constraining the assistant once, the behavior remains consis ### M3 — Cross-Session Recall (implemented, engine-level / host-enabled) **Goal** -Extend host-level workflows around persisted exported state safely and intentionally. +Extend app-level workflows around persisted exported state safely and intentionally. **Core capability:** @@ -77,17 +77,17 @@ Extend host-level workflows around persisted exported state safely and intention **Deliverables:** -- Host-side storage/recovery patterns built on the existing import/export API -- Host-side storage/recovery patterns for checkpoint object/checkpoint JSON continuation restore +- App-side storage/recovery patterns built on the existing import/export API +- App-side storage/recovery patterns for checkpoint object/checkpoint JSON continuation restore **User-visible outcome:** -When hosts persist exported state, assistants can carry decisions across sessions without reintroducing old conflicts. -Pending confirmation-required flows can be resumed when the host persists checkpoints. +When apps persist exported state, assistants can carry decisions across sessions without reintroducing old conflicts. +Pending confirmation-required flows can be resumed when the app persists checkpoints. `export_json()` / `import_json()` remain authoritative-state only. Checkpoint APIs are separate and represent runtime continuation. -Long-term memory remains a host persistence responsibility, not an engine-owned store. +Long-term memory remains an app persistence responsibility, not an engine-owned store. ### 0.6.x @@ -104,19 +104,15 @@ Make engine behavior inspectable and externally controllable without guessing. - State inspection - Deterministic dry-run / preview - Structural state diff -- Thin controller layer around step / preview / replay behavior +- Thin stateless controller layer around step / preview behavior - Machine-readable REPL JSON output containing: - - `decision` - - `prompt_to_user` - - `state` -- JSON input for initial state only: + - versioned one-object-per-line output (`output_version`) + - step / preview / state command result envelopes +- JSON preload for authoritative state and checkpoint continuation: - `--initial-state-json` - `--initial-state-file` -- REPL LLM fallback as explicit optional mode: - - `--with-llm-fallback` - - requires `--with-preprocessor` - - never implicit - - inspectable via preview / JSON output + - `--initial-checkpoint-json` + - `--initial-checkpoint-file` - Explicit preprocessor policy for multi-line, multi-sentence, and conversational-prefix input (for example `ok. prohibit peanuts`, `sure - use docker`, mixed conversational + directive content) that is rule-based, fixture-covered, and inspectable @@ -133,9 +129,11 @@ Make engine behavior inspectable and externally controllable without guessing. ### Post-0.7 Direction -- Profile commands and workflow conveniences +- 0.8 candidate direction: model-assisted state suggestions (inspectable, previewable, + and never directly mutating authoritative state) +- MCP adapter likely as a separate/later track after 0.8 direction is clearer +- Optional 0.7.1 MCP-readiness helpers only if narrowly justified - Additional tooling built on auditability surfaces -- Broader heuristic responsibility remains default-avoid unless tightly justified ### 1.0 Target diff --git a/docs/multi-engine.md b/docs/multi-engine.md index 92d9dc9..e7c3a0c 100644 --- a/docs/multi-engine.md +++ b/docs/multi-engine.md @@ -3,7 +3,7 @@ Most applications should start with a **single Context Compiler engine**. A single engine is not a single rule. -It maintains a complete authoritative state consisting of: +It maintains a complete saved state consisting of: - one premise (a single explicit conversational stance) - a set of per-item policy states (`use` or `prohibit`) @@ -39,7 +39,7 @@ Policies do not interact with each other. - There is no grouping - There is no domain model -Each policy entry is an independent authoritative key. +Each policy entry is an independent key in state. ## When to Use Multiple Engines @@ -52,20 +52,34 @@ Typical cases: - isolation between workflows - independent persistence or reset behavior -## Composition Is a Host Concern +## Composition Is an App Concern The compiler does not coordinate multiple engines. -The host is responsible for: +The app is responsible for: - selecting which engine(s) apply - combining state into model context - managing lifecycle (reset, persistence, replay) per engine -The compiler only maintains a single authoritative state per instance. +The compiler only maintains a single state instance per engine. ## Guideline Start with one engine. Introduce multiple engines only when you need **independent lifecycle or isolation**, not because a single engine is insufficient. + +## Combining Policies from Multiple Sources + +If you need to combine constraints from separate sources, do it explicitly in +host code by replaying directives through `step(...)` into a target engine. + +Pattern: + +1. Select ordered source directives +2. Replay each directive via `engine.step(...)` +3. Handle any returned `clarify` decisions explicitly + +This keeps conflict handling in normal engine semantics and avoids adding merge +semantics to core state APIs. diff --git a/examples/integrations/litellm/basic.py b/examples/integrations/litellm/basic.py index 9401ce3..2eba3ce 100644 --- a/examples/integrations/litellm/basic.py +++ b/examples/integrations/litellm/basic.py @@ -35,7 +35,17 @@ is_confirmation_text = _confirmation.is_confirmation_text -from host_support.confirmation import summarize_confirmation_update +try: + from host_support.confirmation import summarize_confirmation_update_from_checkpoint +except ImportError: + from host_support.confirmation import ( + summarize_confirmation_update as _summarize_confirmation_update_from_pending, + ) + + def summarize_confirmation_update_from_checkpoint(user_input: str, checkpoint: object) -> str: + pending = checkpoint.get("pending") if isinstance(checkpoint, dict) else None + return _summarize_confirmation_update_from_pending(user_input, pending) + try: from host_support import build_trace @@ -191,14 +201,6 @@ def _persist_session_checkpoint_if_needed( _CHECKPOINTS_BY_SESSION_KEY[session_key] = engine.export_checkpoint_json() -def _has_pending_clarification(engine: Engine) -> bool: - checker = getattr(engine, "has_pending_clarification", None) - if callable(checker): - return bool(checker()) - checkpoint = engine.export_checkpoint() - return checkpoint.get("pending") is not None - - def _normalize_confirmation_for_summary(value: str) -> str: normalized = value.strip().lower() normalized = re.sub(r"\s+", " ", normalized) @@ -223,9 +225,9 @@ def _near_miss_directive_clarify(value: str) -> str | None: return None -def _summarize_confirmation_update(user_input: str, pending: object) -> str: - summarize_fn: Callable[[str, object], str] = summarize_confirmation_update - return summarize_fn(user_input, pending) +def _summarize_confirmation_update(user_input: str, checkpoint: object) -> str: + summarize_fn: Callable[[str, object], str] = summarize_confirmation_update_from_checkpoint + return summarize_fn(user_input, checkpoint) def _summarize_update_from_input(user_input: str) -> str: @@ -294,9 +296,8 @@ def _append_trace( def handle_turn(user_input: str, engine: Engine, *, session_key: str | None = None) -> str: _restore_session_checkpoint_if_needed(engine, session_key) state_before = engine.state - pending_before = ( - engine.export_checkpoint().get("pending") if _has_pending_clarification(engine) else None - ) + has_pending_before = engine.has_pending_clarification() + checkpoint_before = engine.export_checkpoint() if has_pending_before else None logger.debug("litellm_basic: engine_input=%s", f"user_input len={len(user_input)}") decision = engine.step(user_input) kind = cast(str, decision["kind"]) @@ -326,8 +327,12 @@ def handle_turn(user_input: str, engine: Engine, *, session_key: str | None = No llm_called=False, ) _persist_session_checkpoint_if_needed(engine, kind, session_key) - if kind == DECISION_UPDATE and is_confirmation_text(user_input) and pending_before is not None: - response_text = _summarize_confirmation_update(user_input, pending_before) + if ( + kind == DECISION_UPDATE + and is_confirmation_text(user_input) + and checkpoint_before is not None + ): + response_text = _summarize_confirmation_update(user_input, checkpoint_before) return _append_trace( response_text, original_input=user_input, diff --git a/examples/integrations/litellm/with_preprocessor.py b/examples/integrations/litellm/with_preprocessor.py index 76fea17..9f49ada 100644 --- a/examples/integrations/litellm/with_preprocessor.py +++ b/examples/integrations/litellm/with_preprocessor.py @@ -47,7 +47,17 @@ is_confirmation_text = _confirmation.is_confirmation_text -from host_support.confirmation import summarize_confirmation_update +try: + from host_support.confirmation import summarize_confirmation_update_from_checkpoint +except ImportError: + from host_support.confirmation import ( + summarize_confirmation_update as _summarize_confirmation_update_from_pending, + ) + + def summarize_confirmation_update_from_checkpoint(user_input: str, checkpoint: object) -> str: + pending = checkpoint.get("pending") if isinstance(checkpoint, dict) else None + return _summarize_confirmation_update_from_pending(user_input, pending) + try: from host_support import build_trace @@ -302,14 +312,6 @@ def _persist_session_checkpoint_if_needed( _CHECKPOINTS_BY_SESSION_KEY[session_key] = engine.export_checkpoint_json() -def _has_pending_clarification(engine: Engine) -> bool: - checker = getattr(engine, "has_pending_clarification", None) - if callable(checker): - return bool(checker()) - checkpoint = engine.export_checkpoint() - return checkpoint.get("pending") is not None - - def _normalize_confirmation_for_summary(value: str) -> str: normalized = value.strip().lower() normalized = re.sub(r"\s+", " ", normalized) @@ -334,9 +336,9 @@ def _near_miss_directive_clarify(value: str) -> str | None: return None -def _summarize_confirmation_update(user_input: str, pending: object) -> str: - summarize_fn: Callable[[str, object], str] = summarize_confirmation_update - return summarize_fn(user_input, pending) +def _summarize_confirmation_update(user_input: str, checkpoint: object) -> str: + summarize_fn: Callable[[str, object], str] = summarize_confirmation_update_from_checkpoint + return summarize_fn(user_input, checkpoint) def _summarize_update_from_input(user_input: str) -> str: @@ -407,11 +409,10 @@ def _append_trace( def handle_turn(user_input: str, engine: Engine, *, session_key: str | None = None) -> str: _restore_session_checkpoint_if_needed(engine, session_key) state_before = engine.state - pending_before = ( - engine.export_checkpoint().get("pending") if _has_pending_clarification(engine) else None - ) + has_pending_before = engine.has_pending_clarification() + checkpoint_before = engine.export_checkpoint() if has_pending_before else None preprocessd: str | None = None - if _has_pending_clarification(engine): + if engine.has_pending_clarification(): compile_input = user_input else: preprocessd = _preprocess_user_input(user_input, engine.state) @@ -451,8 +452,12 @@ def handle_turn(user_input: str, engine: Engine, *, session_key: str | None = No llm_called=False, ) _persist_session_checkpoint_if_needed(engine, kind, session_key) - if kind == DECISION_UPDATE and is_confirmation_text(user_input) and pending_before is not None: - response_text = _summarize_confirmation_update(user_input, pending_before) + if ( + kind == DECISION_UPDATE + and is_confirmation_text(user_input) + and checkpoint_before is not None + ): + response_text = _summarize_confirmation_update(user_input, checkpoint_before) return _append_trace( response_text, original_input=user_input, diff --git a/examples/integrations/openwebui/open_webui_pipe.py b/examples/integrations/openwebui/open_webui_pipe.py index f65c2db..4c82f92 100644 --- a/examples/integrations/openwebui/open_webui_pipe.py +++ b/examples/integrations/openwebui/open_webui_pipe.py @@ -3,8 +3,8 @@ author: rlippmann author_url: https://github.com/rlippmann/context-compiler funding_url: https://github.com/rlippmann/context-compiler -version: 0.8.2 -requirements: context-compiler>=0.6.14 +version: 0.9.0 +requirements: context-compiler>=0.6.20 Minimal Open WebUI Pipe integration for Context Compiler. @@ -12,7 +12,7 @@ Open WebUI request flow. Scope is intentionally limited: -- Single Pipe Function for Open WebUI v0.7.2. +- Single Pipe Function for Open WebUI 0.8.x and 0.9.x. - In-memory per-process engine map keyed by chat key. - No persistence, no multi-worker coordination, no external storage. """ @@ -54,6 +54,7 @@ def Field(*, default: Any, description: str = "") -> Any: # type: ignore[no-red get_premise_value, ) from context_compiler.engine import Engine +from context_compiler.observability import build_compact_trace_text logger = logging.getLogger(__name__) @@ -173,50 +174,6 @@ def _normalize_state(value: object) -> State: return {"premise": None, "policies": {}, "version": 2} -def _active_state_summary(state: object) -> str: - normalized = _normalize_state(state) - premise = get_premise_value(normalized) - use_items = sorted(get_policy_items(normalized, "use")) - prohibit_items = sorted(get_policy_items(normalized, "prohibit")) - parts: list[str] = [] - if premise is not None: - parts.append(f'premise="{premise}"') - if use_items: - parts.append("use " + ", ".join(use_items)) - if prohibit_items: - parts.append("prohibit " + ", ".join(prohibit_items)) - return "; ".join(parts) if parts else "none" - - -def _compact_state_change(before: object, after: object) -> str: - before_state = _normalize_state(before) - after_state = _normalize_state(after) - before_premise = get_premise_value(before_state) - after_premise = get_premise_value(after_state) - before_use = set(get_policy_items(before_state, "use")) - after_use = set(get_policy_items(after_state, "use")) - before_prohibit = set(get_policy_items(before_state, "prohibit")) - after_prohibit = set(get_policy_items(after_state, "prohibit")) - - parts: list[str] = [] - if before_premise != after_premise: - if after_premise is None: - parts.append("-premise") - elif before_premise is None: - parts.append(f'+premise "{after_premise}"') - else: - parts.append(f'~premise "{after_premise}"') - for item in sorted(after_use - before_use): - parts.append(f"+use {item}") - for item in sorted(before_use - after_use): - parts.append(f"-use {item}") - for item in sorted(after_prohibit - before_prohibit): - parts.append(f"+prohibit {item}") - for item in sorted(before_prohibit - after_prohibit): - parts.append(f"-prohibit {item}") - return ", ".join(parts) if parts else "none" - - def _build_compact_trace_text( *, decision: object, @@ -225,31 +182,13 @@ def _build_compact_trace_text( llm_called: bool, state_injected: str, ) -> str: - kind_obj = decision.get("kind") if isinstance(decision, dict) else None - kind = kind_obj if isinstance(kind_obj, str) else "unknown" - lines = ["Context Compiler trace", "", f"decision kind: {kind}"] - - if kind == DECISION_UPDATE: - lines.append(f"state change: {_compact_state_change(state_before, state_after)}") - lines.append(f"active state: {_active_state_summary(state_after)}") - lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") - lines.append("") - lines.append(f"state injected: {state_injected}") - return "\n".join(lines) - - if kind == DECISION_CLARIFY: - prompt_obj = decision.get("prompt_to_user") if isinstance(decision, dict) else None - prompt = prompt_obj if isinstance(prompt_obj, str) else "" - lines.append(f"clarification prompt: {prompt}") - lines.append(f"active state: {_active_state_summary(state_after)}") - lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") - lines.append("state injected: no") - return "\n".join(lines) - - lines.append(f"active state: {_active_state_summary(state_after)}") - lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") - lines.append("state injected: no") - return "\n".join(lines) + return build_compact_trace_text( + decision=decision, + state_before=state_before, + state_after=state_after, + llm_called=llm_called, + state_injected=state_injected, + ) def _strip_trace_block_from_text(content: str) -> str: diff --git a/examples/integrations/openwebui/open_webui_pipe_with_preprocessor.py b/examples/integrations/openwebui/open_webui_pipe_with_preprocessor.py index 3be10f0..a17247e 100644 --- a/examples/integrations/openwebui/open_webui_pipe_with_preprocessor.py +++ b/examples/integrations/openwebui/open_webui_pipe_with_preprocessor.py @@ -3,8 +3,8 @@ author: rlippmann author_url: https://github.com/rlippmann/context-compiler funding_url: https://github.com/rlippmann/context-compiler -version: 0.8.2 -requirements: context-compiler[experimental]>=0.6.14 +version: 0.9.0 +requirements: context-compiler[experimental]>=0.6.20 Open WebUI integration with Context Compiler preprocessor. @@ -57,6 +57,7 @@ def Field(*, default: Any, description: str = "") -> Any: # type: ignore[no-red get_premise_value, ) from context_compiler.engine import Engine +from context_compiler.observability import build_compact_trace_text from experimental.preprocessor import ( PREPROCESS_OUTCOME_DIRECTIVE, parse_preprocessor_output, @@ -125,11 +126,7 @@ def _extract_latest_user_text(messages: list[dict[str, Any]]) -> str | None: def _has_pending_clarification(engine: Engine) -> bool: - checker = getattr(engine, "has_pending_clarification", None) - if callable(checker): - return bool(checker()) - checkpoint = engine.export_checkpoint() - return checkpoint.get("pending") is not None + return engine.has_pending_clarification() def _render_compiler_state_block(state: State) -> str: @@ -182,50 +179,6 @@ def _normalize_state(value: object) -> State: return {"premise": None, "policies": {}, "version": 2} -def _active_state_summary(state: object) -> str: - normalized = _normalize_state(state) - premise = get_premise_value(normalized) - use_items = sorted(get_policy_items(normalized, "use")) - prohibit_items = sorted(get_policy_items(normalized, "prohibit")) - parts: list[str] = [] - if premise is not None: - parts.append(f'premise="{premise}"') - if use_items: - parts.append("use " + ", ".join(use_items)) - if prohibit_items: - parts.append("prohibit " + ", ".join(prohibit_items)) - return "; ".join(parts) if parts else "none" - - -def _compact_state_change(before: object, after: object) -> str: - before_state = _normalize_state(before) - after_state = _normalize_state(after) - before_premise = get_premise_value(before_state) - after_premise = get_premise_value(after_state) - before_use = set(get_policy_items(before_state, "use")) - after_use = set(get_policy_items(after_state, "use")) - before_prohibit = set(get_policy_items(before_state, "prohibit")) - after_prohibit = set(get_policy_items(after_state, "prohibit")) - - parts: list[str] = [] - if before_premise != after_premise: - if after_premise is None: - parts.append("-premise") - elif before_premise is None: - parts.append(f'+premise "{after_premise}"') - else: - parts.append(f'~premise "{after_premise}"') - for item in sorted(after_use - before_use): - parts.append(f"+use {item}") - for item in sorted(before_use - after_use): - parts.append(f"-use {item}") - for item in sorted(after_prohibit - before_prohibit): - parts.append(f"+prohibit {item}") - for item in sorted(before_prohibit - after_prohibit): - parts.append(f"-prohibit {item}") - return ", ".join(parts) if parts else "none" - - def _build_compact_trace_text( *, decision: object, @@ -234,31 +187,13 @@ def _build_compact_trace_text( llm_called: bool, state_injected: str, ) -> str: - kind_obj = decision.get("kind") if isinstance(decision, dict) else None - kind = kind_obj if isinstance(kind_obj, str) else "unknown" - lines = ["Context Compiler trace", "", f"decision kind: {kind}"] - - if kind == DECISION_UPDATE: - lines.append(f"state change: {_compact_state_change(state_before, state_after)}") - lines.append(f"active state: {_active_state_summary(state_after)}") - lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") - lines.append("") - lines.append(f"state injected: {state_injected}") - return "\n".join(lines) - - if kind == DECISION_CLARIFY: - prompt_obj = decision.get("prompt_to_user") if isinstance(decision, dict) else None - prompt = prompt_obj if isinstance(prompt_obj, str) else "" - lines.append(f"clarification prompt: {prompt}") - lines.append(f"active state: {_active_state_summary(state_after)}") - lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") - lines.append("state injected: no") - return "\n".join(lines) - - lines.append(f"active state: {_active_state_summary(state_after)}") - lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") - lines.append("state injected: no") - return "\n".join(lines) + return build_compact_trace_text( + decision=decision, + state_before=state_before, + state_after=state_after, + llm_called=llm_called, + state_injected=state_injected, + ) def _strip_trace_block_from_text(content: str) -> str: diff --git a/host_support/confirmation.py b/host_support/confirmation.py index 63760fa..defc7d6 100644 --- a/host_support/confirmation.py +++ b/host_support/confirmation.py @@ -88,3 +88,22 @@ def summarize_confirmation_update(user_input: str, pending: object) -> str: if normalized not in _AFFIRMATIVE_CONFIRMATION_TOKENS: return "State updated." return _summarize_pending_confirmation_update(pending) + + +def summarize_confirmation_update_from_engine(user_input: str, engine: object) -> str: + """Return confirmation summary using pending details from an engine checkpoint.""" + checkpoint: object = None + export_checkpoint = getattr(engine, "export_checkpoint", None) + if callable(export_checkpoint): + try: + checkpoint = export_checkpoint() + except Exception: + checkpoint = None + pending = checkpoint.get("pending") if isinstance(checkpoint, dict) else None + return summarize_confirmation_update(user_input, pending) + + +def summarize_confirmation_update_from_checkpoint(user_input: str, checkpoint: object) -> str: + """Return confirmation summary using pending details from a checkpoint object.""" + pending = checkpoint.get("pending") if isinstance(checkpoint, dict) else None + return summarize_confirmation_update(user_input, pending) diff --git a/host_support/observability.py b/host_support/observability.py index 22b48b3..98efe91 100644 --- a/host_support/observability.py +++ b/host_support/observability.py @@ -1,6 +1,9 @@ """Shared human-readable integration trace helpers.""" from collections.abc import Mapping +from typing import cast + +from context_compiler import State, state_diff _MAX_INLINE_VALUE_LEN = 180 @@ -39,26 +42,129 @@ def _state_change_summary(before: object, after: object) -> str: return "none -> none" if before == after: return "unchanged" + if isinstance(before, dict) and isinstance(after, dict): + before_state = cast(State, before) + after_state = cast(State, after) + try: + diff = state_diff(before_state, after_state) + except Exception: + diff = None + if isinstance(diff, Mapping): + parts: list[str] = [] + premise = diff.get("premise") + if isinstance(premise, Mapping) and premise.get("changed") is True: + after_premise = premise.get("after") + if after_premise is None: + parts.append("-premise") + else: + parts.append(f'+premise "{after_premise}"') + policies = diff.get("policies") + if isinstance(policies, Mapping): + added = policies.get("added") + removed = policies.get("removed") + changed = policies.get("changed") + if isinstance(added, Mapping): + for item, value in sorted(added.items()): + if value == "use": + parts.append(f"+use {item}") + elif value == "prohibit": + parts.append(f"+prohibit {item}") + if isinstance(removed, Mapping): + for item, value in sorted(removed.items()): + if value == "use": + parts.append(f"-use {item}") + elif value == "prohibit": + parts.append(f"-prohibit {item}") + if isinstance(changed, Mapping): + for item, transition in sorted(changed.items()): + if not isinstance(transition, Mapping): + continue + after_value = transition.get("after") + if after_value == "use": + parts.append(f"~use {item}") + elif after_value == "prohibit": + parts.append(f"~prohibit {item}") + if parts: + return ", ".join(parts) + if isinstance(before, Mapping) and isinstance(after, Mapping): before_keys = set(before.keys()) after_keys = set(after.keys()) - added = sorted(str(key) for key in after_keys - before_keys) - removed = sorted(str(key) for key in before_keys - after_keys) + added_keys = sorted(str(key) for key in after_keys - before_keys) + removed_keys = sorted(str(key) for key in before_keys - after_keys) maybe_changed = sorted( str(key) for key in before_keys & after_keys if before[key] != after[key] ) - parts: list[str] = [] - if added: - parts.append(f"added={added}") - if removed: - parts.append(f"removed={removed}") + key_parts: list[str] = [] + if added_keys: + key_parts.append(f"added={added_keys}") + if removed_keys: + key_parts.append(f"removed={removed_keys}") if maybe_changed: - parts.append(f"changed={maybe_changed}") - if parts: - return "; ".join(parts) + key_parts.append(f"changed={maybe_changed}") + if key_parts: + return "; ".join(key_parts) return f"{_summarize_state(before)} -> {_summarize_state(after)}" +def _active_state_summary(state: object) -> str: + if not isinstance(state, dict): + return "none" + try: + normalized = cast(State, state) + premise = normalized.get("premise") + policies = normalized.get("policies") + except Exception: + return "none" + parts: list[str] = [] + if isinstance(premise, str): + parts.append(f'premise="{premise}"') + if isinstance(policies, Mapping): + use_items = sorted(str(item) for item, value in policies.items() if value == "use") + prohibit_items = sorted( + str(item) for item, value in policies.items() if value == "prohibit" + ) + if use_items: + parts.append("use " + ", ".join(use_items)) + if prohibit_items: + parts.append("prohibit " + ", ".join(prohibit_items)) + return "; ".join(parts) if parts else "none" + + +def build_compact_trace_text( + *, + decision: object, + state_before: object, + state_after: object, + llm_called: bool, + state_injected: str, +) -> str: + """Build OpenWebUI-style compact trace text with stable line formatting.""" + kind = _normalize_text(_decision_field(decision, "kind")) or "unknown" + lines = ["Context Compiler trace", "", f"decision kind: {kind}"] + + if kind == "update": + lines.append(f"state change: {_state_change_summary(state_before, state_after)}") + lines.append(f"active state: {_active_state_summary(state_after)}") + lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") + lines.append("") + lines.append(f"state injected: {state_injected}") + return "\n".join(lines) + + if kind == "clarify": + prompt = _normalize_text(_decision_field(decision, "prompt_to_user")) or "" + lines.append(f"clarification prompt: {prompt}") + lines.append(f"active state: {_active_state_summary(state_after)}") + lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") + lines.append("state injected: no") + return "\n".join(lines) + + lines.append(f"active state: {_active_state_summary(state_after)}") + lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") + lines.append("state injected: no") + return "\n".join(lines) + + def build_trace( *, original_input: str, diff --git a/src/context_compiler/observability.py b/src/context_compiler/observability.py new file mode 100644 index 0000000..e70a92d --- /dev/null +++ b/src/context_compiler/observability.py @@ -0,0 +1,169 @@ +"""Public observability helpers for host integrations.""" + +from collections.abc import Mapping +from typing import cast + +from .controller import state_diff +from .engine import State + +_MAX_INLINE_VALUE_LEN = 180 + + +def _decision_field(decision: object, key: str) -> object: + if isinstance(decision, Mapping): + return decision.get(key) + return getattr(decision, key, None) + + +def _normalize_text(value: object) -> str | None: + if not isinstance(value, str): + return None + normalized = value.strip() + return normalized or None + + +def _inline(value: object) -> str: + rendered = repr(value) + if len(rendered) <= _MAX_INLINE_VALUE_LEN: + return rendered + return f"{rendered[: _MAX_INLINE_VALUE_LEN - 3]}..." + + +def _summarize_state(state: object) -> str: + if state is None: + return "none" + if isinstance(state, Mapping): + keys = sorted(str(key) for key in state) + return f"dict keys={keys}" + return _inline(state) + + +def _state_change_summary(before: object, after: object) -> str: + if before is None and after is None: + return "none -> none" + if before == after: + return "unchanged" + if isinstance(before, dict) and isinstance(after, dict): + before_state = cast(State, before) + after_state = cast(State, after) + try: + diff = state_diff(before_state, after_state) + except Exception: + diff = None + if isinstance(diff, Mapping): + parts: list[str] = [] + premise = diff.get("premise") + if isinstance(premise, Mapping) and premise.get("changed") is True: + after_premise = premise.get("after") + if after_premise is None: + parts.append("-premise") + else: + parts.append(f'+premise "{after_premise}"') + policies = diff.get("policies") + if isinstance(policies, Mapping): + added = policies.get("added") + removed = policies.get("removed") + changed = policies.get("changed") + if isinstance(added, Mapping): + for item, value in sorted(added.items()): + if value == "use": + parts.append(f"+use {item}") + elif value == "prohibit": + parts.append(f"+prohibit {item}") + if isinstance(removed, Mapping): + for item, value in sorted(removed.items()): + if value == "use": + parts.append(f"-use {item}") + elif value == "prohibit": + parts.append(f"-prohibit {item}") + if isinstance(changed, Mapping): + for item, transition in sorted(changed.items()): + if not isinstance(transition, Mapping): + continue + after_value = transition.get("after") + if after_value == "use": + parts.append(f"~use {item}") + elif after_value == "prohibit": + parts.append(f"~prohibit {item}") + if parts: + return ", ".join(parts) + + if isinstance(before, Mapping) and isinstance(after, Mapping): + before_keys = set(before.keys()) + after_keys = set(after.keys()) + added_keys = sorted(str(key) for key in after_keys - before_keys) + removed_keys = sorted(str(key) for key in before_keys - after_keys) + maybe_changed = sorted( + str(key) for key in before_keys & after_keys if before[key] != after[key] + ) + key_parts: list[str] = [] + if added_keys: + key_parts.append(f"added={added_keys}") + if removed_keys: + key_parts.append(f"removed={removed_keys}") + if maybe_changed: + key_parts.append(f"changed={maybe_changed}") + if key_parts: + return "; ".join(key_parts) + return f"{_summarize_state(before)} -> {_summarize_state(after)}" + + +def _active_state_summary(state: object) -> str: + if not isinstance(state, dict): + return "none" + try: + normalized = cast(State, state) + premise = normalized.get("premise") + policies = normalized.get("policies") + except Exception: + return "none" + parts: list[str] = [] + if isinstance(premise, str): + parts.append(f'premise="{premise}"') + if isinstance(policies, Mapping): + use_items = sorted(str(item) for item, value in policies.items() if value == "use") + prohibit_items = sorted( + str(item) for item, value in policies.items() if value == "prohibit" + ) + if use_items: + parts.append("use " + ", ".join(use_items)) + if prohibit_items: + parts.append("prohibit " + ", ".join(prohibit_items)) + return "; ".join(parts) if parts else "none" + + +def build_compact_trace_text( + *, + decision: object, + state_before: object, + state_after: object, + llm_called: bool, + state_injected: str, +) -> str: + """Build OpenWebUI-style compact trace text with stable line formatting.""" + kind = _normalize_text(_decision_field(decision, "kind")) or "unknown" + lines = ["Context Compiler trace", "", f"decision kind: {kind}"] + + if kind == "update": + lines.append(f"state change: {_state_change_summary(state_before, state_after)}") + lines.append(f"active state: {_active_state_summary(state_after)}") + lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") + lines.append("") + lines.append(f"state injected: {state_injected}") + return "\n".join(lines) + + if kind == "clarify": + prompt = _normalize_text(_decision_field(decision, "prompt_to_user")) or "" + lines.append(f"clarification prompt: {prompt}") + lines.append(f"active state: {_active_state_summary(state_after)}") + lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") + lines.append("state injected: no") + return "\n".join(lines) + + lines.append(f"active state: {_active_state_summary(state_after)}") + lines.append(f"downstream LLM call: {'yes' if llm_called else 'no'}") + lines.append("state injected: no") + return "\n".join(lines) + + +__all__ = ["build_compact_trace_text"] diff --git a/tests/test_host_observability.py b/tests/test_host_observability.py index cd51708..c74397a 100644 --- a/tests/test_host_observability.py +++ b/tests/test_host_observability.py @@ -246,3 +246,41 @@ def test_build_trace_state_summary_truncates_very_long_repr() -> None: assert "state change:" in output assert "..." in output + + +def test_build_compact_trace_text_update_shape() -> None: + module = _load_module() + output = module.build_compact_trace_text( + decision={"kind": "update"}, + state_before={"premise": None, "policies": {}, "version": 2}, + state_after={ + "premise": "concise replies", + "policies": {"docker": "use"}, + "version": 2, + }, + llm_called=False, + state_injected="yes", + ) + + assert output.startswith("Context Compiler trace\n\ndecision kind: update\n") + assert 'state change: +premise "concise replies", +use docker' in output + assert 'active state: premise="concise replies"; use docker' in output + assert "downstream LLM call: no" in output + assert output.endswith("state injected: yes") + + +def test_build_compact_trace_text_clarify_shape() -> None: + module = _load_module() + output = module.build_compact_trace_text( + decision={"kind": "clarify", "prompt_to_user": "Use what item?"}, + state_before={"premise": None, "policies": {}, "version": 2}, + state_after={"premise": None, "policies": {}, "version": 2}, + llm_called=False, + state_injected="no", + ) + + assert output.startswith("Context Compiler trace\n\ndecision kind: clarify\n") + assert "clarification prompt: Use what item?" in output + assert "active state: none" in output + assert "downstream LLM call: no" in output + assert output.endswith("state injected: no") diff --git a/tests/test_litellm_checkpoint_integration.py b/tests/test_litellm_checkpoint_integration.py index c054a19..82f760c 100644 --- a/tests/test_litellm_checkpoint_integration.py +++ b/tests/test_litellm_checkpoint_integration.py @@ -43,6 +43,9 @@ def export_checkpoint(self) -> dict[str, object]: "pending": pending, } + def has_pending_clarification(self) -> bool: + return self._has_pending + def step(self, _text: str) -> dict[str, object]: self.step_calls += 1 if self.kind == "clarify": @@ -165,6 +168,9 @@ def export_checkpoint(self) -> dict[str, object]: "pending": pending, } + def has_pending_clarification(self) -> bool: + return self.pending + def step(self, text: str) -> dict[str, object]: self.step_inputs.append(text) if self.pending and text in {"yes", "no"}: @@ -428,6 +434,9 @@ def export_checkpoint(self) -> dict[str, object]: "pending": self._pending, } + def has_pending_clarification(self) -> bool: + return self._pending is not None + def export_checkpoint_json(self) -> str: return "ckpt-fallback" diff --git a/tests/test_litellm_integration_error_paths.py b/tests/test_litellm_integration_error_paths.py index 9382250..2790df9 100644 --- a/tests/test_litellm_integration_error_paths.py +++ b/tests/test_litellm_integration_error_paths.py @@ -264,6 +264,9 @@ def state(self) -> dict[str, object]: def export_checkpoint(self) -> dict[str, object]: return self._engine.export_checkpoint() + def has_pending_clarification(self) -> bool: + return self._engine.has_pending_clarification() + def step(self, text: str) -> dict[str, object]: seen_engine_inputs.append(text) return self._engine.step(text) diff --git a/tests/test_openwebui_preprocessor_pipe.py b/tests/test_openwebui_preprocessor_pipe.py index d8884f9..f87d753 100644 --- a/tests/test_openwebui_preprocessor_pipe.py +++ b/tests/test_openwebui_preprocessor_pipe.py @@ -517,6 +517,9 @@ def export_checkpoint(self) -> dict[str, object]: "pending": pending, } + def has_pending_clarification(self) -> bool: + return self.has_pending + def step(self, _text: str) -> dict[str, object]: if self.kind == "clarify": return {"kind": "clarify", "prompt_to_user": "confirm?", "state": None} @@ -798,6 +801,9 @@ def export_checkpoint(self) -> dict[str, object]: "pending": pending, } + def has_pending_clarification(self) -> bool: + return self.pending + def export_checkpoint_json(self) -> str: return "ckpt-out"