Skip to content

feat(session): async commit, session metadata, and archive continuity threading#900

Merged
chenjw merged 10 commits intomainfrom
feat/async-session-commit
Mar 24, 2026
Merged

feat(session): async commit, session metadata, and archive continuity threading#900
chenjw merged 10 commits intomainfrom
feat/async-session-commit

Conversation

@qin-ctx
Copy link
Collaborator

@qin-ctx qin-ctx commented Mar 23, 2026

Description

This PR reshapes the session commit flow around a strict two-phase model:

  • Phase 1 persists the archived messages immediately and returns an accepted response with task_id
  • Phase 2 runs in the background to generate archive summaries, extract memories, write relations, update active counts, and finalize task status

On top of the async commit split, this PR also makes session history usable as continuity context instead of isolated archive snapshots. The latest completed archive overview is threaded into both archive summary generation and memory extraction, and get_context_for_search() now exposes latest_archive_overview plus current in-session messages. The structured summary prompt was updated accordingly so later archives can explicitly carry forward stable context, separate newly introduced information, and preserve continuity-relevant user quotes.

This PR also adds persistent session metadata via .meta.json, simplifies the session API surface, and updates client/example integrations and docs to the new task-based commit semantics.

Related Issue

N/A

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • Reworked session commit into a two-phase async flow in Session.commit_async():
    • Phase 1 writes messages.jsonl, clears the active message buffer, persists updated metadata, and returns { status: "accepted", task_id, archive_uri, archived }
    • Phase 2 runs under task tracking and handles archive summary generation, memory extraction, relation writes, active count updates, redo-log protection, .done finalization, and final metadata updates
  • Added persistent session metadata in .meta.json via SessionMeta, including message_count, commit_count, memories_extracted, last_commit_at, and accumulated LLM token usage
  • Updated session retrieval to return metadata-backed session info and added GET /sessions/{id}?auto_create=true support for lazy session creation
  • Added client-side task polling support with get_task() and aligned sync/async/local clients with the new commit response shape
  • Simplified session server/router flow so /sessions/{id}/commit always archives inline and returns a background task_id, while conflict protection is handled via task tracker checks
  • Threaded the latest completed archive overview into:
    • archive summary generation
    • v2 memory extraction
    • session search context (latest_archive_overview + current_messages)
  • Added _get_latest_completed_archive_overview() to skip incomplete archives and only expose the newest completed overview
  • Reworked compression/structured_summary prompt so later archive overviews are continuity-aware and explicitly separate:
    • carried-forward historical context
    • new information introduced in the current archive
    • current archive analysis
    • continuity-relevant key user quotes
    • open loops / next steps
  • Updated OpenClaw example integration to use the new commit/task semantics, per-agent identity caching, and session-based capture/cleanup flow
  • Refreshed English and Chinese session docs, API docs, and related examples to match the new behavior

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows

Test coverage added/updated includes:

  • task-based async commit behavior and race/conflict handling
  • metadata-backed session retrieval and session context shape changes
  • propagation of latest_archive_overview into summary generation and memory extraction
  • skipping incomplete archives when resolving continuity context
  • compressor v2 and session task tracking updates

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

Screenshots (if applicable)

N/A

Additional Notes

Suggested reviewer focus:

  • correctness of the Phase 1 / Phase 2 boundary and task-tracked lifecycle
  • backward-compatibility of client/API behavior around session commit and get_session
  • whether continuity threading via latest_archive_overview is the right abstraction for summary generation, extraction, and search context

qin-ctx added 4 commits March 23, 2026 15:40
Session commit now returns immediately after archiving messages (Phase 1).
Summary generation and memory extraction (Phase 2) run in the background
via asyncio.create_task(), returning a task_id for polling progress.

- Add get_task() API across all client layers for querying background task status
- get_session() auto-creates session if it does not exist
- Remove wait parameter and telemetry from commit endpoint
- Add .done completion marker to archive directories
- Update docs (EN/ZH) and tests for new two-phase flow
… get_session

SessionService.get() now defaults to auto_create=False, raising NotFoundError
for missing sessions. A new SessionMeta dataclass tracks created_at, updated_at,
message_count, commit_count, memories_extracted (by category), last_commit_at,
and cumulative llm_token_usage. Meta is persisted to .meta.json and updated on
add_message, commit Phase 1 (message clear), and commit Phase 2 completion
(token usage, memory counts via bind_telemetry). All client layers
(local/async/sync/HTTP) and API docs updated accordingly.
@github-actions
Copy link

Failed to generate code suggestions for PR

wlff123 and others added 5 commits March 24, 2026 11:35
Thread the latest completed archive overview into archive summary generation and memory extraction, and simplify search context assembly to current messages plus the latest archive overview.

Co-Authored-By: Claude Opus 4.6
@qin-ctx qin-ctx changed the title WIP: feat(session): two-phase async commit with meta persistence feat(session): two-phase async commit with meta persistence Mar 24, 2026
@qin-ctx qin-ctx changed the title feat(session): two-phase async commit with meta persistence feat(session): async commit, session metadata, and archive continuity threading Mar 24, 2026
@chenjw chenjw merged commit 2c598e8 into main Mar 24, 2026
5 of 6 checks passed
@chenjw chenjw deleted the feat/async-session-commit branch March 24, 2026 14:30
@github-project-automation github-project-automation bot moved this from Backlog to Done in OpenViking project Mar 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants