fix(update): startup readiness and reliability cleanups

[elkaix]

Related Issue

No linked issue.

Description

• Preserve terminal background-agent task state during recovery by re-reading runtime under the store update lock and centralizing guarded status transitions.
• Simplify project memory recall by removing the unused SQLite FTS retriever seam, dropping a dead store_path recall argument, and exposing a public memory-store root accessor.
• Improve shell transcript rendering for resolved pasted-content input and align background todo continuation rows.
• Make the pre-start update prompt revalidate stale/current caches with a bounded conditional refresh, and harden release promotion so /releases/latest waits for exact assets, PyPI, and Homebrew readiness.
• Add review reports and changelog notes for the reliability fixes.

Testing

• uv run pytest tests/background/test_manager.py tests/core/test_memory_phase_bcd.py tests/core/test_recall_provider.py tests/ui_and_conv/test_shell_prompt_echo.py tests/ui_and_conv/test_shell_run_placeholders.py tests/ui_and_conv/test_shell_update.py tests/ui_and_conv/test_visualize_running_prompt.py -q — 190 passed
• uv run ruff check ... on touched Python files — passed
• uv run ruff format --check ... on touched Python files — passed
• uv run pyright ... on touched Python files — 0 errors
• make check-pythinker-code — ruff and pyright passed; ty reported existing diagnostics in tests/ui_and_conv/test_native_update_parity.py and remains non-blocking in this Make target

Checklist

• Added/updated focused tests for behavior changes.
• Updated the changelog.
• Linked issue, if any.
• Generated docs; not needed for these code/runtime changes.

Summary by CodeRabbit

• New Features

  • Release promotion now validates all required native installers, checksums, PyPI JSON, and Homebrew formula readiness before finalizing.
  • Startup update check revalidates cached version status with a short timeout.

• Bug Fixes

  • Background task recovery no longer overwrites already-completed runtimes.
  • Shell transcripts and prompts echo resolved/expanded commands.
  • Background TODO rows now align correctly.

• Refactor

  • Memory retrieval simplified; legacy SQLite retriever seam removed.
  • Background task status transitions consolidated via a unified transition path.

• Documentation

  • Added architecture and scan reports and updated changelog and agent guidance.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c362d792-f628-443c-b877-3a0ad37f51eb

📥 Commits

Reviewing files that changed from the base of the PR and between 1c285dd and 1faa86d.

📒 Files selected for processing (3)

• tests/core/test_default_agent.py
• tests/telemetry/test_instrumentation.py
• tests/test_release_update_pipeline.py

---

📝 Walkthrough

Walkthrough

This PR applies five coordinated improvements to fix a background task race condition, simplify the memory system by removing unused abstractions, polish the shell UI with correct command echoing and alignment, guard the startup prompt's cache refresh with a timeout, and enhance the release workflow with comprehensive readiness polling before promoting.

Changes

Background Task Recovery and Status Transitions

Layer / File(s)	Summary
Locked runtime transition mechanism src/pythinker_code/background/manager.py	New _transition_status helper consolidates runtime mutations to use store.update_runtime() with terminal-guarded callbacks. _recover_agent_view refactored to apply recovery via the same locked mechanism, preventing stale snapshots from overwriting completed tasks. All terminal outcome helpers (_mark_task_*) rewired to route through _transition_status, eliminating direct private lock/write access.
Recovery regression test tests/background/test_manager.py	Verifies that recovery with a stale view does not overwrite an already-terminal runtime and correctly reconciles subagent status to idle.

Memory System Simplification

Layer / File(s)	Summary
Remove Retriever ABC and SQLite seam src/pythinker_code/memory/retriever.py, src/pythinker_code/memory/retriever_sqlite.py	Removes the unused Retriever abstract base class entirely, along with SqliteFts5Retriever. LexicalRetriever drops ABC inheritance and simplifies term-frequency computation with Counter(doc). FTS5 fallback tests removed.
Public root accessor and store_path removal src/pythinker_code/project_memory.py, src/pythinker_code/memory/consolidation.py, src/pythinker_code/memory/recall.py, tests/core/test_recall_provider.py	ProjectMemoryStore gains public ensure_root() async method. consolidation.py switches from private _ensure_dir() to public API. build_recall_block removes unused store_path parameter; call sites and tests updated accordingly.

Shell UI Improvements

Layer / File(s)	Summary
Echo resolved command text src/pythinker_code/ui/shell/__init__.py, src/pythinker_code/ui/shell/visualize/_interactive.py, tests/ui_and_conv/test_shell_prompt_echo.py, tests/ui_and_conv/test_shell_run_placeholders.py	Agent input echoing, queued message draining, and Ctrl+S steer now print resolved/expanded command text instead of raw placeholders. Transcripts display the actual executed form (e.g., multiline text instead of [Pasted text #N +lines]).
Timeout-guarded cache refresh with staleness detection src/pythinker_code/ui/shell/update.py, tests/ui_and_conv/test_shell_update.py	New _refresh_update_cache_for_prompt() wraps refresh with asyncio.wait_for timeout. _resolve_latest_version_for_prompt() detects staleness via semver comparison; forces refresh only when cache is missing or stale, not on every check. Tests verify stale-cache bypass and timeout-fallback behavior.
Visual alignment for background TODO rows src/pythinker_code/ui/shell/prompt.py, src/pythinker_code/ui/shell/echo.py, tests/ui_and_conv/test_visualize_running_prompt.py	Background TODO rows compute continuation prefix from first-item gutter marker display width, replacing hardcoded spacing. Hidden TODO summary uses same prefix. Docstring for render_user_echo_text clarified. New test verifies consistent icon/title alignment across states.

Release Workflow and Documentation

Layer / File(s)	Summary
Multi-condition release readiness polling .github/workflows/promote-release.yml	Release promotion now polls for PyPI availability, Homebrew formula version match, and all expanded required assets (including .sha256 files, architecture variants) before clearing prerelease and setting make_latest=true. Polling budget enforced; workflow fails if readiness not achieved in time.
Architecture review and code scan reports .pythinker/reports/arch-review-2026-05-30.md, .pythinker/reports/deep-code-scan-findings.md	Detailed findings document the race condition fixed in this PR, the unused Retriever abstraction removed, duplicated task state mutations consolidated, and five medium-severity elegance improvements. Coordinated fix sequence provided; verification against current codebase recorded.
CHANGELOG, agent guidance, and system prompt CHANGELOG.md, AGENTS.md, src/pythinker_code/agents/default/system.md	CHANGELOG entries summarize the four main user-facing changes. Agent guidance adds rules for minimal scope and avoiding over-abstraction. System prompt broadens self-check to "overcomplicated or over-engineered" and adds anti-fragmentation guidance.

---

🎯 4 (Complex) | ⏱️ ~75 minutes

---

Possibly related PRs

• TechMatrix-labs/pythinker-code#25: Also modifies .github/workflows/promote-release.yml polling/budgeting for release readiness.
• TechMatrix-labs/pythinker-code#13: Overlaps in BackgroundTaskManager recovery and terminal-state transition changes using locked store updates.
• TechMatrix-labs/pythinker-code#29: Related BackgroundTaskStore.update_runtime(...) evolution that this PR builds on.

---

bug

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 27.87% which is insufficient. The required threshold is 70.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	Title follows conventional commits format with type 'fix', scope 'update', and descriptive message aligned to the changeset.
Description check	✅ Passed	Description covers objectives, testing results, and checklist completion; changelog update confirmed via CHANGELOG.md changes.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/reliability-cleanups-update-readiness

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}