Cotabby 2: cut the Open Source path to base models

[FuJacob]

Summary

Cotabby 2: hard cut of the Open Source (llama) path to base-model continuation. The instruct GGUF catalog is replaced with base models (Qwen3.5 0.8B/2B/4B + Gemma E2B/E4B base, from mradermacher's i1 GGUF repos), product tiers are renamed tabby-2-*, onboarding and preferredModelNames point at base (default Qwen3.5-2B-Base), and BaseCompletionPromptRenderer becomes the unconditional local prompt. The useBaseCompletionPipeline flag is removed. Old instruct GGUFs are no longer listed or supported. The Foundation Models (Apple Intelligence) path is unchanged.

Validation

xcodebuild build-for-testing -project Cotabby.xcodeproj -scheme Cotabby \
  -destination 'platform=macOS' -derivedDataPath build/DerivedData
# ** BUILD SUCCEEDED ** (full app + CotabbyTests target compile)

swiftlint lint --config .swiftlint.yml --quiet   # changed files, exit 0

Local app-hosted test execution is blocked by a Team ID / code-signing mismatch loading the xctest bundle into the signed host; CI's xcodebuild test (macOS) job runs the suite with proper signing. Updated the two tests that pinned the old filenames/display-names (OnboardingTemplateRecommenderTests, ModelAndPresentationValueTests).

Linked issues

None.

Risk / rollout notes

• Hard behavior change. The OSS path now renders the base prompt unconditionally and prefers base models. Generation quality on base models is the accepted bet (the reference base-model app runs these same models on the same llama.cpp b9310 this repo pins); it is not dogfood-verified in CI.
• Existing users. Old instruct GGUFs are unsupported going forward. preferredModelNames lists base only, so the locator falls back to any other discovered GGUF until the user installs a base model. No forced-migration UX in this PR (a deliberate follow-up).
• Model metadata. Base GGUF expectedSizeBytes/sha256 are nil (matching the reference app); the download manager skips size/hash validation when nil. Backfill from CDN headers later.
• Known follow-ups: M2 engine primitives (logits / batched decode / vocab introspection) then M3 constrained generation + token profiles (the deeper quality core); OnboardingTemplateRecommender memory thresholds and warning copy are still sized for the old ~5 GB models and are now over-conservative; existing-user base-model provisioning UX.

🤖 Generated with Claude Code

Greptile Summary

This PR hard-cuts the Open Source (llama) path from instruction-tuned GGUFs to base-model continuation, replacing the tabby-1-* instruct GGUF catalog with Qwen3.5 (0.8B/2B/4B) and Gemma E2B/E4B base checkpoints from mradermacher's i1 repos, renaming product tiers to tabby-2-*, and making BaseCompletionPromptRenderer the unconditional local prompt.

• Removes LlamaPromptRenderer.swift and its instruct-blob format entirely; BaseCompletionPromptRenderer (no task preamble, prefix-last, base-model conditioning) is now the sole local render path.
• Removes the useBaseCompletionPipeline feature flag from SuggestionSettingsSnapshot, SuggestionSettingsModel, and the Combine publisher stack; the flag is no longer read or written.
• Updates RuntimeModelCatalog.downloadableModels, preferredModelNames, OnboardingTemplate.openSourceModelFilename, and all tests that pinned old filenames or label strings.

Confidence Score: 5/5

Safe to merge; this is a focused catalog and prompt-path replacement with no new runtime logic introduced.

The change is a clean hard-cut: one renderer deleted, one promoted to unconditional use, a feature flag removed end-to-end, and the model catalog swapped in lockstep with onboarding templates and tests. The Foundation Models path is structurally unchanged. The only open item — test coverage for three of the five new display names — was flagged in a prior review and is tracked as a follow-up.

CotabbyTests/ModelAndPresentationValueTests.swift — only 2 of 5 new catalog display names are asserted; a future rename in the switch for tabby-2-pro, tabby-2-gemma-mini, or tabby-2-gemma-pro would pass silently.

Important Files Changed

Filename	Overview
Cotabby/Support/SuggestionRequestFactory.swift	Removes the useBaseCompletionPipeline branch; BaseCompletionPromptRenderer is now called unconditionally. Foundation Models path is unaffected as it builds its own messages from request fields rather than consuming request.prompt.
Cotabby/Models/LlamaRuntimeModels.swift	Catalog replaced with five base-model GGUFs; introduces the hfURL helper (force-unwrap is safe on compile-time literal inputs). expectedSizeBytes/sha256 are nil pending CDN-header backfill — intentionally documented. preferredModelNames now leads with Qwen3.5-2B-Base.
Cotabby/Support/BaseCompletionPromptRenderer.swift	Doc comment updated to reflect it is now the unconditional local renderer; logic unchanged. authorFraming correctly handles all combinations of name/rules/language.
Cotabby/Models/SuggestionSettingsModel.swift	Removes @published var useBaseCompletionPipeline, its UserDefaults key, and its slot in the CombineLatest4 → CombineLatest3 publisher chain. Clean removal with no dangling references.
CotabbyTests/ModelAndPresentationValueTests.swift	Pins 2 of the 5 new tabby-2-* display names; tabby-2-pro, tabby-2-gemma-mini, and tabby-2-gemma-pro are not directly asserted — a typo in the catalog switch for those names would pass silently.
CotabbyTests/LlamaPromptCacheHintTrackerTests.swift	New file; cache-hint tests migrated from the now-deleted LlamaPromptRendererTests.swift with identical helper logic. Four deterministic test cases covering initial nil, common-prefix, field-change invalidation, and sampling-fingerprint invalidation.
CotabbyTests/SuggestionRequestFactoryTests.swift	String constants updated from LlamaPromptRenderer labels to BaseCompletionPromptRenderer equivalents (e.g. 'On the clipboard:', 'Notes the writer keeps in mind:').
CotabbyTests/OnboardingTemplateRecommenderTests.swift	Filenames updated to the three new Qwen3.5 base GGUFs; test logic unchanged.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[SuggestionRequestFactory.buildRequest] --> B{selectedEngine}
    B -->|llamaOpenSource| C[BaseCompletionPromptRenderer.prompt]
    B -->|appleIntelligence| C
    C --> D[SuggestionRequest with prompt field]
    D --> E{Engine router}
    E -->|llamaOpenSource| F[llama runtime consumes request.prompt]
    E -->|appleIntelligence| G[FoundationModelPromptRenderer.promptPreview builds own messages]

    subgraph Removed
        X[LlamaPromptRenderer instruct-blob format]
        Y[useBaseCompletionPipeline flag + UserDefaults key]
    end

    subgraph Catalog tabby-2
        M1[Qwen3.5-0.8B-Base Q6_K → tabby-2-mini]
        M2[Qwen3.5-2B-Base Q4_K_M → tabby-2-base]
        M3[Qwen3.5-4B-Base Q4_K_M → tabby-2-pro]
        M4[gemma-4-E2B Q6_K → tabby-2-gemma-mini]
        M5[gemma-4-E4B Q4_K_M → tabby-2-gemma-pro]
    end

Loading

Comments Outside Diff (1)

1. CotabbyTests/ModelAndPresentationValueTests.swift, line 148-161 (link)

   Only 2 of the 5 new tabby-2-* display names are asserted. A typo in tabby-2-pro, tabby-2-gemma-mini, or tabby-2-gemma-pro inside the displayName switch would pass this test undetected.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

   

_{Reviews (2): Last reviewed commit: "Remove dead code from the base-model mig..." | Re-trigger Greptile}