Improve context-aware autocomplete behavior

[Joaov41]

What changed

This branch ports the local changes from the modified Tabby working tree into a real git branch and documents the differences from the original upstream code.

The main behavior changes are:

• richer Accessibility-derived field context, including focused-field metadata and nearby AX text
• suffix-aware request building so completions can fit naturally before existing text after the caret
• stronger normalization and filtering to reduce generic filler, copied UI/OCR fragments, assistant-style replies, and whole-word echo artifacts
• local partial-word completion through the macOS completion APIs before falling back to model generation
• snapshot-driven rescheduling when text changes arrive through Accessibility without a normal key event
• visual-context handling that treats Screen Recording as optional for screenshot context rather than a hard requirement for plain autocomplete
• README updates describing the behavioral differences from the original upstream code and clarifying the current permission model

Why

The goal is to make suggestions more specific and less chatty, especially in real app text fields where upstream context is sparse and model outputs can otherwise collapse into generic continuations.

Validation

• xcodebuild -project tabby.xcodeproj -scheme tabby -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO -derivedDataPath .codex-derived build

Notes

I did not include local build artifacts, app bundles, or derived data in this PR.

Greptile Summary

This PR ports a working-tree delta into a clean branch, making Tabby's autocomplete more context-aware and less prone to generating generic or model-leaked suggestions. The changes are broadly well-structured and address several issues from the previous review round (pre-buffered suppression registration in SuggestionInserter, scoped question detection via recentSentenceFragment, anchored time-unit patterns, removal of debug AX dump scaffolding).

• Local-first completion path: Two new providers (LocalSpellCorrectionProvider, LocalWordCompletionProvider) intercept common cases before hitting the generative model, using NSSpellChecker with Levenshtein scoring and safe-pattern guards; the new SuggestionAcceptanceEdit enum and replacePreviousCharacters on the inserter give spell correction a clean, atomic commit path.
• Richer AX context: FocusSnapshotResolver now collects bounded nearby AX text (maxNodes = 140, maxChars = 1 200) and field-level metadata (placeholder, title, description) and threads these through the request; SuggestionTextNormalizer gains an isLikelyAuxiliaryContextCopy guard specifically tuned to reject verbatim copies of that context.
• Snapshot-driven rescheduling: schedulePredictionIfFocusedTextChangedWithoutKeyEvent detects AX-only text mutations (automation, some IMEs) and schedules prediction without a CGEvent tap; the isRefreshingFocusForInputEvent flag prevents duplicate scheduling when refreshNow() fires its synchronous callback during normal key-event handling.

Confidence Score: 5/5

Safe to merge; the new local-completion path and normalizer guards are well-bounded and the inserter's atomic suppression registration prevents the partial-failure issue from the previous review.

All core paths are covered by unit tests and the logic is deterministic. The two findings are narrow: a redundant prompt instruction wastes a token budget line, and the suffixText inclusion in the generic-continuation gate can occasionally allow a context-poor generic phrase through when the user has trailing text. Neither affects correctness of insertion, suppression, or session lifecycle.

tabby/Support/SuggestionTextNormalizer.swift (lacksConcreteAuxiliaryContext) and tabby/Support/LlamaPromptRenderer.swift (duplicate instruction)

Important Files Changed

Filename	Overview
tabby/Support/SuggestionTextNormalizer.swift	Adds ~500 lines of normalization guards; OCR detection now requires ≥2 suspicious words; digit-to-letter substitution correctly scoped to OCR matching only; lacksConcreteAuxiliaryContext mixes suffixText into the auxiliary context signal
tabby/Services/Suggestion/LocalSpellCorrectionProvider.swift	New: Levenshtein spell correction via NSSpellChecker with candidate scoring, transposition detection, and atomic one-shot acceptance via replacePreviousCharacters
tabby/Services/Suggestion/LocalWordCompletionProvider.swift	New: Fast word-tail completion via NSSpellChecker.completions; returns only the suffix tail to avoid double-strip with normalizer
tabby/Services/Suggestion/SuggestionInserter.swift	Adds replacePreviousCharacters; events pre-buffered before registerSuppression, fixing partial-failure suppression race; DI closures enable unit tests
tabby/Support/LlamaPromptRenderer.swift	Extended with fieldContextText, suffixText, anti-hallucination instructions; contains two consecutive identical task bullets
tabby/Services/Focus/FocusSnapshotResolver.swift	Replaced debug AX dump with focusedFieldContextText and bounded nearbyAccessibilityTextContext (maxNodes=140, maxChars=1200)
tabby/App/Coordinators/SuggestionCoordinator+Input.swift	Adds snapshot-driven prediction scheduling; isRefreshingFocusForInputEvent prevents duplicate scheduling during synchronous AX callbacks
tabby/App/Coordinators/SuggestionCoordinator+Prediction.swift	Local providers tried before model generation; acceptanceEdit threaded through startSession; screenRecordingGranted removed from all checks
tabby/Models/SuggestionModels.swift	SuggestionAcceptanceEdit enum added; suffixText and fieldContextText propagated through request/context types
tabby/Models/FocusModels.swift	fieldContextText added to contentSignature — AX subtree changes can still trigger snapshot-driven predictions when user is idle

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["Key event / AX snapshot change"] --> B{"isRefreshingFocusForInputEvent?"}
    B -- yes --> C["materializeContext only (no prediction)"]
    B -- no --> D{"contentSignature changed & no active session?"}
    D -- no --> E["skip"]
    D -- yes --> F["schedulePrediction()"]
    F --> G["FocusedInputContext materialized"]
    G --> H{"LocalSpellCorrectionProvider.suggestion?"}
    H -- yes --> I["SuggestionResult .replacePreviousCharacters"]
    H -- no --> J{"LocalWordCompletionProvider.suggestion?"}
    J -- yes --> K["SuggestionResult .insert (tail only)"]
    J -- no --> L["Build SuggestionRequest (prefix+suffix+field+visual)"]
    L --> M["Prompt renderer"]
    M --> N["Model output"]
    N --> O["SuggestionTextNormalizer.normalize"]
    O --> P["SuggestionResult.text"]
    I --> Q["startSession(acceptanceEdit:)"]
    K --> Q
    P --> Q
    Q --> R["Ghost text overlay"]
    R --> S{"Tab pressed?"}
    S -- insert --> T["SuggestionInserter.insert"]
    S -- replace --> U["SuggestionInserter.replacePreviousCharacters"]

Loading

Comments Outside Diff (3)

1. tabby/Support/SuggestionAvailabilityEvaluator.swift, line 10-56 (link)

   screenRecordingGranted parameter is now unused

   Both disabledReason and shouldSchedulePrediction still accept a screenRecordingGranted: Bool parameter, but the guard that used it was removed as part of this PR. Every call-site in the coordinator still passes the value through, so the parameter is silently ignored.

   This is harmless in isolation, but it is misleading for callers who may assume the value continues to influence the gating decision. If Screen Recording is now intentionally an optional-only signal (for visual context), the parameter should be removed from these two functions and the call sites updated accordingly, or a comment should document that the parameter is retained only for a future use.

   

2. tabby/Services/Focus/FocusSnapshotResolver.swift, line 13-15 (link)

   Temporary debug utility flag still present

   dumpAXTree is false in production so it causes no harm, but the comment explicitly marks it as "temporary — remove after caret placement is fixed." If the caret issue has been resolved, this block (along with lastDumpedElementID and the two print-based helpers at the bottom of the file) should be removed before the branch is merged to avoid accumulating debug scaffolding in shared code.

   

3. tabby/Models/FocusModels.swift, line 218-228 (link)

   fieldContextText in contentSignature triggers spurious snapshot-driven predictions on external AX changes

   schedulePredictionIfFocusedTextChangedWithoutKeyEvent fires whenever focusedContext.contentSignature differs from the previous snapshot. By adding fieldContextText ?? "" to contentSignature, any change in the nearby AX tree — a new chat message arriving, a sidebar label refreshing, a status indicator updating — changes the signature and causes prediction to be scheduled even though the user typed nothing.

   In a chat app such as Slack or Messages, every incoming message updates the AX subtree around the compose field. Under the new code this would continuously trigger prediction while the user is idle in the compose box, wasting generation cycles and potentially flashing irrelevant suggestions.

   The contentSignature should reflect only the editable content (precedingText, trailingText, selection). fieldContextText is context metadata, not a user-edit signal, and should be excluded from this comparison key.

   

_{Reviews (7): Last reviewed commit: "Narrow relative-time metadata filtering" | Re-trigger Greptile}

[FuJacob]

Thanks for the work! I tested this locally and it seems to have degraded the completions and I'm getting more issues like "```" etc than before. Could you please take a look, thanks!!

[Joaov41]

Thanks for the work! I tested this locally and it seems to have degraded the completions and I'm getting more issues like "```" etc than before. Could you please take a look, thanks!!

You are right. Funny enough using Apple Intelligence seems to be a little better but with other models is a regression without a doubt. Back to experimentation.