Skip to content

perf(session): flush buffered turn events in a single transaction - #678

Merged
pancacake merged 1 commit into
HKUDS:devfrom
rncchen:perf/batch-turn-event-flush
Jul 24, 2026
Merged

perf(session): flush buffered turn events in a single transaction#678
pancacake merged 1 commit into
HKUDS:devfrom
rncchen:perf/batch-turn-event-flush

Conversation

@rncchen

@rncchen rncchen commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Problem

After a turn's answer has fully streamed, _flush_buffered_events persists the
turn's buffered events one at a time via store.append_turn_event. Each call
opens a connection, looks up the turn, computes MAX(seq), inserts one row,
updates the turn, and commits — i.e. one fsync per event.

On fast SSDs this is invisible. On slower storage it dominates turn
finalisation: running DeepTutor on a Synology NAS (spinning disks, btrfs),
each commit took ~0.65 s, so a mastery-path turn with 550 buffered events
kept flushing for ~6 minutes after the visible answer had finished. Because
the terminal done event is only published after the flush, the client
spinner keeps running the whole time — the app looks hung even though the
answer is already on screen.

Evidence from the affected turn (turn_events rows, generated vs stored):

first event last event
timestamp (generated) 02:05:32 02:07:01
created_at (stored) 02:07:01 02:12:58

550 rows stored over ~5 min 57 s ≈ 1.5 rows/s.

Change

  • Add SQLiteSessionStore.append_turn_events(turn_id, events): assigns seqs
    and inserts the whole batch inside one transaction. Measured locally:
    501 events in ~6 ms (vs minutes with per-event commits on the NAS).
  • _flush_buffered_events prefers the batch method when the store provides
    one (via getattr), keeping the existing per-event loop as a fallback for
    stores without batch support (e.g. the PocketBase store, unchanged).
  • When the turn has vanished mid-drain (session deleted while flushing),
    log one summary line instead of one warning per buffered event — a long
    turn previously emitted hundreds of identical
    "Skip persisting event for missing turn" warnings.

Notes

  • Seq assignment semantics match the single-event path: an explicit
    seq on an event is honored; otherwise seqs continue from MAX(seq).
  • No API or schema changes.

Post-stream finalisation appended buffered events one at a time, each in
its own fsync'd commit (connection open + turn lookup + MAX(seq) + INSERT
+ UPDATE + COMMIT per event). On slow storage this dominates turn
finalisation: on a NAS with spinning disks each commit took ~0.65 s, so a
550-event turn kept the client spinner running for ~6 minutes after the
answer had fully streamed, because the terminal done event is only
published after the flush.

Add SQLiteSessionStore.append_turn_events, which assigns seqs and inserts
the whole batch inside one transaction (measured: 501 events in ~6 ms),
and make _flush_buffered_events prefer it, keeping the per-event path as
a fallback for stores without batch support. When the turn has vanished
(session deleted mid-drain) the flush now logs one summary line instead
of one warning per buffered event.
@pancacake
pancacake changed the base branch from main to dev July 24, 2026 08:03
@pancacake
pancacake merged commit dfbe287 into HKUDS:dev Jul 24, 2026
pancacake added a commit that referenced this pull request Jul 24, 2026
Same-day maintenance follow-up to v1.5.3, focused on how chat feels.

- Chat responsiveness: the post-answer "generating" stall is gone (DONE
  carries the persisted message ids so the frontend reconciles in place
  instead of refetching the session), turn events flush in one
  transaction (#678), the streaming autoscroll stops forcing per-frame
  layouts, and Enter during streaming no longer fires an interleaved
  message (#674).
- Partners: markdown-table row splitting unified across channels — empty
  cells survive Slack (#679) and Feishu (#683) tables, and empty rows are
  no longer misread as header separators (#682).
- LLM-output parsing: <think> reasoning tags stripped before parsing
  (#675), adjacent JSON values no longer break Deep Research extraction
  (#680), and the parser returns the longest decodable value instead of
  the first prefix (#692).
- Assorted: streaming quiz cards stay scoped to their own turn (#677),
  the create-KB form survives the background indexing poll (#691), and
  Math Animator reads ms as milliseconds, not minutes (#681).
- Typing: SQLite session store add_message accepts str parents to match
  SessionStoreProtocol (PocketBase record ids).

Release notes: assets/releases/ver1-5-4.md
vaskoyudha added a commit to vaskoyudha/deeptutor-for-programmer-fork that referenced this pull request Jul 25, 2026
…UDS#678)

Post-stream finalisation appended buffered events one at a time, each in
its own fsync'd commit (connection open + turn lookup + MAX(seq) + INSERT
+ UPDATE + COMMIT per event). On slow storage this dominates turn
finalisation: on a NAS with spinning disks each commit took ~0.65 s, so a
550-event turn kept the client spinner running for ~6 minutes after the
answer had fully streamed, because the terminal done event is only
published after the flush.

Add SQLiteSessionStore.append_turn_events, which assigns seqs and inserts
the whole batch inside one transaction (measured: 501 events in ~6 ms),
and make _flush_buffered_events prefer it, keeping the per-event path as
a fallback for stores without batch support. When the turn has vanished
(session deleted mid-drain) the flush now logs one summary line instead
of one warning per buffered event.
vaskoyudha added a commit to vaskoyudha/deeptutor-for-programmer-fork that referenced this pull request Jul 25, 2026
Same-day maintenance follow-up to v1.5.3, focused on how chat feels.

- Chat responsiveness: the post-answer "generating" stall is gone (DONE
  carries the persisted message ids so the frontend reconciles in place
  instead of refetching the session), turn events flush in one
  transaction (HKUDS#678), the streaming autoscroll stops forcing per-frame
  layouts, and Enter during streaming no longer fires an interleaved
  message (HKUDS#674).
- Partners: markdown-table row splitting unified across channels — empty
  cells survive Slack (HKUDS#679) and Feishu (HKUDS#683) tables, and empty rows are
  no longer misread as header separators (HKUDS#682).
- LLM-output parsing: <think> reasoning tags stripped before parsing
  (HKUDS#675), adjacent JSON values no longer break Deep Research extraction
  (HKUDS#680), and the parser returns the longest decodable value instead of
  the first prefix (HKUDS#692).
- Assorted: streaming quiz cards stay scoped to their own turn (HKUDS#677),
  the create-KB form survives the background indexing poll (HKUDS#691), and
  Math Animator reads ms as milliseconds, not minutes (HKUDS#681).
- Typing: SQLite session store add_message accepts str parents to match
  SessionStoreProtocol (PocketBase record ids).

Release notes: assets/releases/ver1-5-4.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants