refactor(core): make spaCy opt-in + demand-gate the Python runtime (#5056)

[graycyrus]

Summary

• Default memory_tree.spacy_enabled to false (was true) — spaCy becomes opt-in.
• Demand-gate the python_runtime harness-init step so managed CPython is not downloaded speculatively at boot.
• Result: a fresh install no longer spawns the runtime Python server on every launch, nor provisions a spaCy venv + en_core_web_sm model on first run, when no local NLP is configured.

Problem

On a fresh install, spacy_enabled defaulted true, so enabled_backends() returned [Spacy] and the runtime Python server launched on every boot, with the first run building a spaCy venv + downloading en_core_web_sm. Separately, the python_runtime step resolved/downloaded managed CPython whenever runtime_python.enabled (default true), even when nothing needed Python. This is the core of #5056's "unnecessary runtimes" item (and relates to #4814). spaCy only improves person/org recall in the E2GraphRAG query-entity extractor, which already has a full in-Rust regex+LLM fallback.

Solution

• default_memory_tree_spacy_enabled() → false. With spaCy off (Kompress already off by default), enabled_backends() is empty → the runtime Python server never launches at boot and the spaCy venv/model is never provisioned. Query-entity extraction degrades to score::extract. Opt in via config or OPENHUMAN_MEMORY_TREE_SPACY_ENABLED=1.
• New python_needed_eagerly(config) = runtime_python.enabled && !enabled_backends(config).is_empty(). The python_runtime step's is_done/run gate on it, so managed CPython is provisioned eagerly only when a Python backend actually needs it. Lazy consumers (Python tools / skills / MCP) still resolve the interpreter on first use.
• Node, Kompress, and the server-launch steps are unchanged — they already derive correctly.
• CREATE_NO_WINDOW coverage is untouched (already complete); this PR changes whether work happens at boot, not how processes spawn.

Coordinates with #5055 (spaCy provider reliability): this makes spaCy opt-in, it does not remove it.

Submission Checklist

• Tests added or updated — added: spacy_enabled defaults false; enabled_backends(&Config::default()) is empty; python_runtime step is auto-done / no-op by default and provisions when a backend is enabled. Updated existing harness_init / runtime_python_server / config-default tests for the new default.
• Diff coverage ≥ 80% — changed lines are covered by the added/updated unit tests.
• Coverage matrix updated — N/A: behaviour-default change (no feature row add/remove; spaCy path unchanged, only its default).
• All affected feature IDs listed under ## Related
• No new external network dependencies introduced (this removes speculative downloads)
• Manual smoke checklist — N/A: retrieval works identically via the regex fallback; opt-in spaCy path unchanged.
• Linked issue — partial; see ## Related (does not close Remove unused/redundant features: screen intelligence, order complete, desktop automation, autocomplete, Whisper, unnecessary runtimes #5056).

Impact

• Runtime: fresh installs no longer spawn the Python server every launch or download spaCy/CPython on first run. Retrieval quality: query-entity extraction uses the regex+LLM fallback until a user opts into spaCy (person/org recall slightly lower, no correctness regression).
• Migration: existing installs that persisted spacy_enabled: true keep spaCy on (serde default applies only to absent fields) — no behaviour change for current users; only fresh installs get the new default.
• Build: cargo check passes (media ON + disabled gate); harness_init 12/12, runtime_python_server 17/17, memory_tree 210/210, config:: 472/472.

Related

• Part of Remove unused/redundant features: screen intelligence, order complete, desktop automation, autocomplete, Whisper, unnecessary runtimes #5056 (does not close — one slice of a multi-part cleanup)
• Relates to CMD windows flash on every launch during startup — should only appear on first install #4814 (CMD windows / unnecessary boot spawns) and Fix local model reliability: validate local model behavior in Tiny Agents and correct provider configuration #5055 (spaCy provider reliability — coordinate)
• Follow-up PR(s): remaining Remove unused/redundant features: screen intelligence, order complete, desktop automation, autocomplete, Whisper, unnecessary runtimes #5056 slices

---

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

• Key: N/A
• URL: N/A

Commit & Branch

• Branch: fixes/spacy-off-lazy-runtimes
• Commit SHA: 832c0a2

Validation Run

• pnpm --filter openhuman-app format:check — N/A (no app changes)
• pnpm typecheck — N/A (no TS changes)
• Focused tests: harness_init 12/12, runtime_python_server 17/17, memory_tree 210/210, config:: 472/472
• Rust fmt/check (if changed): cargo fmt --check clean; cargo check media ON + disabled gate pass
• Tauri fmt/check (if changed): N/A

Behavior Changes

• Intended behavior change: spaCy is opt-in; the Python runtime + server are not provisioned/launched at boot unless a backend needs them
• User-visible effect: fewer background processes / no first-run spaCy download on fresh installs; retrieval uses the regex fallback until spaCy is enabled

Parity Contract

• Legacy behavior preserved: existing installs with spacy_enabled persisted keep current behavior; enabling spaCy restores the full path
• Guard/fallback/dispatch parity checks: enabled_backends derivation unchanged; server/kompress/node steps unchanged

Duplicate / Superseded PR Handling

• Duplicate PR(s): none
• Canonical PR: this
• Resolution: N/A

Summary by CodeRabbit

• New Features

  • Python runtime components are now provisioned only when an enabled feature requires them.
  • spaCy support is now opt-in and disabled by default on fresh installations.

• Bug Fixes

  • Prevented unnecessary Python downloads and setup when no Python-backed features are enabled.
  • Improved initialization behavior while preserving automatic setup when required.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: d24045c1-7f93-491e-ad7c-4117d40c6eb9

📥 Commits

Reviewing files that changed from the base of the PR and between 40852cd and 832c0a2.

📒 Files selected for processing (3)

• src/openhuman/config/schema/storage_memory.rs
• src/openhuman/harness_init/registry.rs
• src/openhuman/runtime_python_server/registry.rs

---

📝 Walkthrough

Walkthrough

Changes

Runtime opt-in gating

Layer / File(s)	Summary
Opt-in configuration defaults src/openhuman/config/schema/storage_memory.rs, src/openhuman/runtime_python_server/registry.rs	spaCy and Python runtime backends are disabled by default, with tests covering the default configuration.
Conditional eager provisioning src/openhuman/harness_init/registry.rs	Python provisioning and probing now require Python to be enabled and at least one backend to be active; tests cover both disabled and spaCy-enabled cases.

Estimated code review effort: 2 (Simple) | ~10 minutes

Sequence Diagram(s)

sequenceDiagram
  participant HarnessInit
  participant Config
  participant enabled_backends
  participant python_runtime
  HarnessInit->>Config: Read runtime_python.enabled
  HarnessInit->>enabled_backends: Resolve enabled Python backends
  alt Python enabled and a backend is enabled
    HarnessInit->>python_runtime: Probe or provision eagerly
  else No enabled backend
    HarnessInit->>python_runtime: Report done and skip run
  end

Loading

Suggested labels: feature, memory

Poem

I’m a rabbit guarding the start-up gate,
No Python sprout shall wake too late.
spaCy sleeps till called by name,
Fresh installs stay quiet all the same.
Opt in, and runtimes hop to play!

---

_{Comment @coderabbitai help to get the list of available commands.}

[senamakel]

Automated technical review: approved for the inspected head commit.

Summary of analysis

This PR changes default_memory_tree_spacy_enabled() from true to false, making spaCy opt-in, and introduces python_needed_eagerly() to demand-gate the managed-CPython provision step so that the interpreter is not speculatively downloaded at boot when no Python backend (spaCy/Kompress) is enabled.

What is correct

• The python_needed_eagerly predicate correctly composes runtime_python.enabled with enabled_backends being non-empty, so the eager provision only fires when a backend actually needs it.
• The existing spacy_step already independently gates on spacy_enabled, and runtime_python_server_step already gates on enabled_backends being non-empty, so they correctly reflect the new default without additional wiring.
• Serde's #[serde(default)] means persisted configs retain their stored value; only fresh installs see false. Migration is seamless.
• Test coverage: spacy_enabled_defaults_to_false (config), enabled_backends_is_empty_by_default (runtime_python_server), python_runtime_step_is_done_by_default_with_no_backend_enabled and python_needed_eagerly_true_when_spacy_backend_enabled (harness_init). All hermetic, no disk/network.
• All CI hard gates pass: Rust Quality, Feature-Gate Smoke, Rust Core Coverage, PR CI Gate. The only red check (PR Submission Checklist) is a soft informational check.

One minor documentation finding (not blocking)

In src/openhuman/config/schema/storage_memory.rs at line 358, the doc comment on spacy_enabled still reads "When true (default)". Since the default is now false, the (default) parenthetical is stale and could mislead human readers of the API docs. Recommend changing to "When true (optional, default false)" or similar. The function-level comment above default_memory_tree_spacy_enabled() already correctly describes the new opt-in behavior — this is just the field-level docstring that needs alignment.

[greptile-apps]

Greptile Summary

This PR makes spaCy opt-in by defaulting spacy_enabled to false, and adds a python_needed_eagerly gate so the managed CPython runtime is not provisioned at boot unless a Python backend (spaCy or Kompress) is actually enabled. Fresh installs will no longer spawn the runtime Python server or download en_core_web_sm; query-entity extraction falls back to the existing in-Rust regex+LLM extractor.

• default_memory_tree_spacy_enabled() changed from true → false; existing installs with a persisted spacy_enabled: true are unaffected (serde only applies the default for absent fields).
• python_needed_eagerly(config) introduced in harness_init/registry.rs to gate python_is_done/python_run; the runtime Python server and spaCy steps already derived from enabled_backends and are unchanged.
• Tests added for all three gating invariants: default config has empty backends, python_runtime step is a no-op by default, and the gate enables provisioning once a backend is turned on.

Confidence Score: 4/5

Safe to merge — the logic change is small and well-tested; the only issue is a stale doc comment on spacy_enabled.

The gating predicate is straightforward and all three changed invariants are covered by the new tests. The stale doc comment is the only issue and does not affect runtime behaviour.

The spacy_enabled doc comment in src/openhuman/config/schema/storage_memory.rs needs its wording updated to reflect the new default of false.

Important Files Changed

Filename	Overview
src/openhuman/config/schema/storage_memory.rs	Changed default_memory_tree_spacy_enabled() from true to false; field-level doc comment still says "(default)" next to true — stale, but only a doc issue.
src/openhuman/harness_init/registry.rs	Introduced python_needed_eagerly(config) to gate managed CPython provisioning; python_is_done/python_run now early-return when no backend is enabled. Tests cover both the default no-op case and the enabled-backend case correctly.
src/openhuman/runtime_python_server/registry.rs	No functional code changes; adds enabled_backends_is_empty_by_default test that validates the new default-off spaCy behaviour end-to-end through the backend registry.

Comments Outside Diff (1)

1. src/openhuman/config/schema/storage_memory.rs, line 357-363 (link)

   The field-level doc comment still describes true as the default, but the default is now false. Any developer reading the rustdoc or IDE hover will get the wrong information about whether spaCy activates on a fresh install.

_{Reviews (1): Last reviewed commit: "refactor(core): make spaCy opt-in + dema..." | Re-trigger Greptile}