Skip to content

Commit abfc248

Browse files
committed
AI settings: cloud presets, conn check, RAM gauge
- Replace "OpenAI-compatible" with "Cloud / API" toggle label and add preset dropdown with 15 providers (OpenAI, Anthropic, Gemini, Groq, Together AI, Fireworks, Mistral, OpenRouter, DeepSeek, xAI, Perplexity, Azure, Ollama, LM Studio, Custom) - Store per-provider API keys and model configs in `ai.cloudProviderConfigs` JSON blob; migrate old flat settings automatically - Add two-step connection check: auto-validates endpoint on key/URL change (1s debounce), shows inline status (connected/auth error/unreachable) - Add model combobox: fetches available models via `GET /v1/models`, falls back to text input when unavailable - Add RAM gauge for local LLM context size: stacked bar showing system/Cmdr AI/free memory via `sysinfo` crate, with legend - Replace auto-debounce context size restart with explicit "Apply" button + warning icons at >70% and >90% projected RAM - Show multi-step download progress ("Step N of 4: ...") with ETA instead of getting stuck at 100% - Make model deletion async (`spawn_blocking`) so dialog spinner renders without UI freeze - Log download cancellation as info, not error
1 parent 47a2e8e commit abfc248

22 files changed

Lines changed: 1836 additions & 204 deletions

AGENTS.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,9 @@ There are two MCP servers available to you:
126126
disabilities.
127127
- All actions longer than, say, 1 second should be immediately cancelable, canceling not just the UI but any background
128128
processes as well, to avoid wasting the user's resources.
129+
- Write _elegant_ code. Not quick code, not overengineered code, but elegant code. If you need to choose between a small
130+
refactor that leads to a slightly better architecture or a larger refactor that leads to a near-perfect architecture,
131+
choose the larger refactor.
129132
- When shortcuts are available for a feature, always display the shortcut in a tooltip or somewhere, less prominent than
130133
the main UI.
131134
- **Platform-native, not generic.** The app should look and feel as if it was specifically made for the user's OS. Never

Cargo.lock

Lines changed: 34 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

apps/desktop/knip.json

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,11 @@
11
{
22
"$schema": "https://unpkg.com/knip@5/schema.json",
33
"ignoreBinaries": ["only-allow"],
4-
"ignore": [
5-
"src/routes/+layout.ts",
6-
"src/lib/tauri-commands/**",
7-
"src/lib/indexing/index-events.ts",
8-
"src/lib/indexing/index-priority.ts"
9-
],
4+
"ignore": ["src/lib/tauri-commands/**"],
105
"ignoreDependencies": [
6+
"oxlint",
117
"@tauri-apps/cli",
128
"@testing-library/svelte",
13-
"@sveltejs/adapter-static",
149
"prettier-plugin-svelte",
1510
"@wdio/globals",
1611
"@wdio/local-runner",

apps/desktop/src-tauri/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ lru = "0.16"
7676
xattr = "1"
7777
filetime = "0.2"
7878
exacl = "0.12"
79+
sysinfo = { version = "0.38.4", default-features = false, features = ["system"] }
7980

8081
[target.'cfg(unix)'.dependencies]
8182
libc = "0.2"

apps/desktop/src-tauri/src/ai/CLAUDE.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ Three provider modes:
2121

2222
### Tauri commands
2323

24-
Core: `get_ai_status`, `get_ai_model_info`, `get_ai_runtime_status`, `configure_ai`, `start_ai_server`, `stop_ai_server`, `start_ai_download`, `cancel_ai_download`, `get_folder_suggestions`.
24+
Core: `get_ai_status`, `get_ai_model_info`, `get_ai_runtime_status`, `configure_ai`, `start_ai_server`, `stop_ai_server`, `check_ai_connection`, `start_ai_download`, `cancel_ai_download`, `get_folder_suggestions`.
2525
Legacy (still wired, used by toast): `uninstall_ai`, `dismiss_ai_offer`, `opt_out_ai`, `opt_in_ai`, `is_ai_opted_out`.
2626

2727
## Startup flow
@@ -48,6 +48,17 @@ Frontend loads
4848
- `local` -> uses local llama-server (if running)
4949
- `openai-compatible` -> builds `AiBackend::OpenAi` from stored config, calls `chat_completion`
5050

51+
## Download/install event sequence
52+
53+
`do_download()` emits events for each install step so the frontend can show progress:
54+
1. `ai-extracting` -- binary extraction from bundled archive (usually instant)
55+
2. `ai-download-progress` (repeated) -- model download with bytes/total/speed/eta
56+
3. `ai-verifying` -- file size verification after download completes
57+
4. `ai-installing` -- server startup begins (health check polling)
58+
5. `ai-install-complete` -- server is healthy and ready
59+
60+
The frontend (`AiSection.svelte`) tracks `installStep` state and displays "Step N of 4" labels.
61+
5162
## Key patterns
5263

5364
- Two install flags: `AiState.installed` AND `AiState.model_download_complete` -- both must be true.

apps/desktop/src-tauri/src/ai/manager.rs

Lines changed: 143 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,8 +199,13 @@ pub fn cancel_ai_download() {
199199
}
200200

201201
/// Uninstalls the AI model and binary, resets state.
202+
/// Async because `stop_process` can block up to 5 seconds (SIGTERM + wait + SIGKILL).
202203
#[tauri::command]
203-
pub fn uninstall_ai() {
204+
pub async fn uninstall_ai() {
205+
tauri::async_runtime::spawn_blocking(uninstall_ai_sync).await.ok();
206+
}
207+
208+
fn uninstall_ai_sync() {
204209
let mut manager = MANAGER.lock_ignore_poison();
205210
if let Some(ref mut m) = *manager {
206211
// Stop server if running
@@ -359,6 +364,30 @@ pub fn get_ai_runtime_status() -> AiRuntimeStatus {
359364
}
360365
}
361366

367+
/// System memory info returned to frontend for the RAM gauge.
368+
#[derive(Debug, Clone, serde::Serialize)]
369+
#[serde(rename_all = "camelCase")]
370+
pub struct SystemMemoryInfo {
371+
pub total_bytes: u64,
372+
/// Memory actively used by processes (app + wired + compressed on macOS).
373+
pub used_bytes: u64,
374+
/// Memory available for new allocations (free + inactive + purgeable on macOS).
375+
pub available_bytes: u64,
376+
}
377+
378+
/// Returns system memory info (total, used by processes, and available).
379+
/// Uses the `sysinfo` crate for cross-platform accuracy.
380+
#[tauri::command]
381+
pub fn get_system_memory_info() -> SystemMemoryInfo {
382+
let mut sys = sysinfo::System::new();
383+
sys.refresh_memory();
384+
SystemMemoryInfo {
385+
total_bytes: sys.total_memory(),
386+
used_bytes: sys.used_memory(),
387+
available_bytes: sys.available_memory(),
388+
}
389+
}
390+
362391
/// Stores provider + context size + OpenAI config in manager state.
363392
/// If provider is `local` and model is installed and hardware is supported, starts the server
364393
/// in a background task. If provider is NOT `local` and a server is running, stops it.
@@ -497,6 +526,116 @@ pub fn start_ai_server<R: Runtime>(app: AppHandle<R>, ctx_size: u32) -> Result<(
497526
Ok(())
498527
}
499528

529+
/// Result of checking connectivity to an AI API endpoint.
530+
#[derive(Debug, Clone, serde::Serialize)]
531+
#[serde(rename_all = "camelCase")]
532+
pub struct AiConnectionCheckResult {
533+
pub connected: bool,
534+
pub auth_error: bool,
535+
pub models: Vec<String>,
536+
pub error: Option<String>,
537+
}
538+
539+
/// Checks connectivity to an AI API endpoint by calling GET {base_url}/models.
540+
/// Returns connection status, auth status, and available model list.
541+
#[tauri::command]
542+
pub async fn check_ai_connection(base_url: String, api_key: String) -> AiConnectionCheckResult {
543+
let url = format!("{}/models", base_url.trim_end_matches('/'));
544+
545+
let client = match reqwest::Client::builder()
546+
.timeout(std::time::Duration::from_secs(10))
547+
.build()
548+
{
549+
Ok(c) => c,
550+
Err(e) => {
551+
return AiConnectionCheckResult {
552+
connected: false,
553+
auth_error: false,
554+
models: vec![],
555+
error: Some(format!("Can't create HTTP client: {e}")),
556+
};
557+
}
558+
};
559+
560+
let mut request = client.get(&url);
561+
if !api_key.is_empty() {
562+
request = request.header("Authorization", format!("Bearer {api_key}"));
563+
}
564+
565+
let response = match request.send().await {
566+
Ok(r) => r,
567+
Err(e) => {
568+
let msg = if e.is_timeout() {
569+
String::from("Can't reach server (timed out)")
570+
} else if e.is_connect() {
571+
String::from("Can't reach server")
572+
} else {
573+
format!("Can't reach server: {e}")
574+
};
575+
return AiConnectionCheckResult {
576+
connected: false,
577+
auth_error: false,
578+
models: vec![],
579+
error: Some(msg),
580+
};
581+
}
582+
};
583+
584+
let status = response.status();
585+
586+
if status == reqwest::StatusCode::UNAUTHORIZED || status == reqwest::StatusCode::FORBIDDEN {
587+
return AiConnectionCheckResult {
588+
connected: true,
589+
auth_error: true,
590+
models: vec![],
591+
error: Some(String::from("API key is invalid")),
592+
};
593+
}
594+
595+
if status == reqwest::StatusCode::OK {
596+
let body = response.text().await.unwrap_or_default();
597+
// Try parsing OpenAI-style response: { "data": [{ "id": "model-name" }, ...] }
598+
let models = parse_model_ids(&body);
599+
return AiConnectionCheckResult {
600+
connected: true,
601+
auth_error: false,
602+
models,
603+
error: None,
604+
};
605+
}
606+
607+
// Other HTTP error
608+
let body = response.text().await.unwrap_or_default();
609+
let body_preview = if body.len() > 200 {
610+
format!("{}...", &body[..200])
611+
} else {
612+
body
613+
};
614+
AiConnectionCheckResult {
615+
connected: true,
616+
auth_error: false,
617+
models: vec![],
618+
error: Some(format!("HTTP {status}: {body_preview}")),
619+
}
620+
}
621+
622+
/// Parses model IDs from an OpenAI-compatible /models response.
623+
/// Returns empty vec on parse failure (connected but can't list models).
624+
fn parse_model_ids(body: &str) -> Vec<String> {
625+
#[derive(serde::Deserialize)]
626+
struct ModelsResponse {
627+
data: Vec<ModelEntry>,
628+
}
629+
#[derive(serde::Deserialize)]
630+
struct ModelEntry {
631+
id: String,
632+
}
633+
634+
serde_json::from_str::<ModelsResponse>(body)
635+
.map(|r| r.data.into_iter().map(|m| m.id).collect())
636+
.unwrap_or_default()
637+
}
638+
500639
/// Formats bytes as GB with one decimal place (like "4.3 GB").
501640
fn format_bytes_gb(bytes: u64) -> String {
502641
let gb = bytes as f64 / 1_000_000_000.0;
@@ -633,6 +772,7 @@ async fn do_download<R: Runtime>(app: &AppHandle<R>) -> Result<(), String> {
633772
// Step 1: Extract llama-server from bundled archive (instant, no download needed)
634773
let binary_path = ai_dir.join(LLAMA_SERVER_BINARY);
635774
if !binary_path.exists() {
775+
let _ = app.emit("ai-extracting", ());
636776
extract_bundled_llama_server(app, &ai_dir)?;
637777
}
638778

@@ -660,7 +800,8 @@ async fn do_download<R: Runtime>(app: &AppHandle<R>) -> Result<(), String> {
660800

661801
download_file(app, model.url, &model_path, is_cancel_requested).await?;
662802

663-
// Verify download integrity by checking file size
803+
// Step 3: Verify download integrity by checking file size
804+
let _ = app.emit("ai-verifying", ());
664805
let actual_size = fs::metadata(&model_path)
665806
.map(|m| m.len())
666807
.map_err(|e| format!("Failed to read downloaded model file: {e}"))?;

apps/desktop/src-tauri/src/lib.rs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -823,6 +823,8 @@ pub fn run() {
823823
ai::manager::configure_ai,
824824
ai::manager::start_ai_server,
825825
ai::manager::stop_ai_server,
826+
ai::manager::check_ai_connection,
827+
ai::manager::get_system_memory_info,
826828
ai::manager::start_ai_download,
827829
ai::manager::cancel_ai_download,
828830
ai::manager::dismiss_ai_offer,

apps/desktop/src/lib/ai/AiNotification.test.ts

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,19 @@ vi.mock('./ai-state.svelte', () => ({
1313
import { getAiState, handleDownload, handleCancel, handleDismiss, handleOptOut, handleGotIt } from './ai-state.svelte'
1414
import AiToastContent from './AiToastContent.svelte'
1515

16+
type AiNotificationState = 'hidden' | 'offer' | 'downloading' | 'installing' | 'ready' | 'starting'
17+
1618
let mockState = {
17-
notificationState: 'hidden' as string,
19+
notificationState: 'hidden' as AiNotificationState,
1820
downloadProgress: null as { bytesDownloaded: number; totalBytes: number; speed: number; etaSeconds: number } | null,
1921
progressText: '',
2022
modelInfo: {
2123
id: 'ministral-3b-instruct-q4km',
2224
displayName: 'Ministral 3B',
2325
sizeBytes: 2147023008,
2426
sizeFormatted: '2.1 GB',
27+
kvBytesPerToken: 106496,
28+
baseOverheadBytes: 3500000000,
2529
},
2630
}
2731

@@ -43,6 +47,8 @@ describe('AiToastContent', () => {
4347
displayName: 'Ministral 3B',
4448
sizeBytes: 2147023008,
4549
sizeFormatted: '2.1 GB',
50+
kvBytesPerToken: 106496,
51+
baseOverheadBytes: 3500000000,
4652
},
4753
}
4854
vi.mocked(getAiState).mockReturnValue(mockState)
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
import AiToastContent from './AiToastContent.svelte'
2+
import { getAiState } from './ai-state.svelte'
3+
import { addToast, dismissToast } from '$lib/ui/toast'
4+
5+
export function initAiToastSync(): void {
6+
$effect(() => {
7+
if (getAiState().notificationState === 'hidden') {
8+
dismissToast('ai')
9+
} else {
10+
addToast(AiToastContent, { id: 'ai', dismissal: 'persistent' })
11+
}
12+
})
13+
}

apps/desktop/src/lib/settings/CLAUDE.md

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -38,19 +38,31 @@ Single source of truth for all settings. Each `SettingDefinition` contains:
3838
`UpdatesSection`, `ThemesSection`, `AdvancedSection`, `DriveIndexingSection`, `AiSection`, `LicenseSection`.
3939

4040
`AiSection` is a hybrid special section (like `LicenseSection` above): it combines dynamic runtime state from the
41-
backend (via `getAiRuntimeStatus()` and Tauri events) with registry settings (`ai.provider`, `ai.openaiApiKey`, etc.).
42-
It conditionally renders provider-specific content, handles auto-stop/start of the local server on provider switch, and
43-
debounces context size changes with a 2-second restart delay.
41+
backend (via `getAiRuntimeStatus()` and Tauri events) with registry settings (`ai.provider`, `ai.cloudProvider`,
42+
`ai.cloudProviderConfigs`, etc.). It conditionally renders provider-specific content, handles auto-stop/start of the
43+
local server on provider switch. Context size changes are not auto-applied; the user must click an explicit "Apply"
44+
button, which triggers a server restart. A RAM gauge (stacked bar) shows memory usage relative to system total, with
45+
warning icons at >70% and >90% projected usage. System memory info is polled every 5 seconds via
46+
`get_system_memory_info`. The "Cloud / API" provider mode uses a preset dropdown (`cloud-providers.ts`) with
47+
per-provider API key storage in a JSON blob (`ai.cloudProviderConfigs`). Old flat settings (`ai.openaiApiKey`,
48+
`ai.openaiBaseUrl`, `ai.openaiModel`) are migrated on first load. The Cloud/API section includes a two-step connection
49+
check (`check_ai_connection` Tauri command) that auto-triggers on API key or base URL changes (1s debounce), fetches
50+
available models from the `/models` endpoint, and shows connection status (connected, auth error, unreachable). When
51+
models are available, the Model field becomes a combobox with filtered dropdown; otherwise it's a plain text input.
4452

4553
### Components (`components/`)
4654

4755
11 reusable setting UI primitives used by section components: `SettingsSection` (wrapper providing shared section title
4856
and action button styles), `SettingRow`, `SettingSwitch`, `SettingSelect`, `SettingSlider`, `SettingNumberInput`,
49-
`SettingPasswordInput`, `SettingRadioGroup`, `SettingToggleGroup`, `SettingsSidebar`, `SettingsContent`. Also
50-
`SectionSummary` for collapsed-section previews.
57+
`SettingPasswordInput` (supports both settings-store-driven and controlled/external value+onchange modes),
58+
`SettingRadioGroup`, `SettingToggleGroup`, `SettingsSidebar`, `SettingsContent`. Also `SectionSummary` for
59+
collapsed-section previews.
5160

5261
### Other files
5362

63+
- **cloud-providers.ts** — Cloud provider preset definitions (OpenAI, Anthropic, Groq, etc.) and per-provider config
64+
helpers (`getProviderConfigs`, `setProviderConfig`, `resolveCloudConfig`). Used by `AiSection` and the startup flow in
65+
`+layout.svelte` to resolve the effective API key, base URL, and model before calling `configureAi`.
5466
- **settings-search.ts** — Fuzzy search over setting definitions; returns ranked matches with highlight ranges
5567
- **settings-applier.ts** — Listens for setting changes and applies side effects (CSS vars, backend config sync)
5668
- **network-settings.ts** — Network-specific setting helpers (proxy config, SMB auth defaults)

0 commit comments

Comments
 (0)