Skip to content

Add streaming autocomplete mode#79

Merged
FuJacob merged 1 commit into
mainfrom
codex/streaming-autocomplete
Apr 29, 2026
Merged

Add streaming autocomplete mode#79
FuJacob merged 1 commit into
mainfrom
codex/streaming-autocomplete

Conversation

@FuJacob

@FuJacob FuJacob commented Apr 29, 2026

Copy link
Copy Markdown
Owner

Summary

  • Rebases streaming autocomplete onto current main, after KV-cache reuse and AXObserver focus tracking landed.
  • Resolves conflicts by preserving per-app enable/disable settings alongside the streaming autocomplete toggle.
  • Updates the streaming llama path to await async KV-cache reset calls from main so stream failures do not race the next generation.

Validation

xcodebuild test -scheme tabby -project tabby.xcodeproj -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO
# ** TEST SUCCEEDED **
# Executed 69 tests, with 0 failures (0 unexpected)

Note: the test process still printed the existing llama.cpp/ggml Metal teardown GGML_ASSERT([rsets->data count] == 0) after XCTest reported success; xcodebuild exited 0.

Linked issues

Refs #11
Refs #13

Risk / rollout notes

  • Streaming is behind the menu-bar/settings toggle and defaults off.
  • Partial output is only surfaced at stable chunk boundaries through SuggestionStreamChunker so token fragments are not shown directly.
  • Runtime reset semantics now follow the async reset contract from main for both non-streaming and streaming llama paths.

@FuJacob FuJacob merged commit 5b68437 into main Apr 29, 2026
0 of 3 checks passed
FuJacob added a commit that referenced this pull request Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant