feat: live bottleneck indicator on the Activity dashboard by pmaxhogan · Pull Request #311 · pmaxhogan/driven

pmaxhogan · 2026-08-17T18:59:10Z

Summary

Closes #308.

Backend classifier (new src-tauri/src/bottleneck_hub.rs, 1s cadence, idle
suppression) that names the current limiting pipeline stage:
not_backing_up / disk / network / api / cpu / mixed. Streams as a
new sync:bottleneck event with a bottleneck_status hydration command.

Heuristic (documented in classify()'s doc comment, unit-tested over all
six states + boundary cases):

No account mid-cycle => not_backing_up.
A mid-cycle account's rate pacer is backing off (or the orchestrator
itself is in the Backoff circuit-breaker state) => api, naming the
destination + remaining ms. Checked before rates because a paced account
can otherwise show misleadingly healthy numbers between gated requests.
Otherwise compare disk-read / net-wire-accepted / blake3-hashed bytes/sec.
Stages at or below a 32 KB/s idle floor are dropped from consideration.
Among the remaining active stages, the slowest is the bottleneck,
provided the fastest active stage clears it by >= 1.5x; short of that
there's no clear winner => mixed.

Signals wired in:

Disk/net rates: existing IoCounters (crates/driven-core/src/iostat.rs),
diffed by the sampler - untouched by the AdaptiveController's separate
drain-based ThroughputProbe.
api: new public AimdPacer::backoff_remaining_ms() getter (non-blocking
read of the private backoff_until_ms deadline against the injected
clock), exposed through the Pacer trait (default None) and two new
default Orchestrator trait methods (pacer_backoff_remaining_ms,
backend_label) so the sampler can poll every account's orchestrator
through its existing Arc<dyn Orchestrator> handle with no structural
changes to orchestrator.rs/executor.rs.
cpu: net-new cumulative hash-byte counter added alongside IoCounters'
existing disk/net fields (IoCounters::add_hashed), credited from the
upload pipeline's cpu_stage (streamed path) and inline_upload
(buffered small-file path), plus the scanner's deep-verify re-hash
(threaded through a new optional io_counters param on
scan_with_priority). A relaxed atomic add on the existing counter -
no new hot-path allocation.

Frontend

BottleneckStatTile.vue: a plain-value sibling of ThroughputStatTile
(same STAT_TILE chrome, dt/dd typography); no sparkline since the
backend classifies one current state, not a series.
stores/bottleneck.ts: subscribe()/hydrate() pattern like stores/iostat.ts,
plus a debounce/hysteresis gate - a new state must hold for 5s (measured
off the snapshots' own tsMs, not Date.now(), so it's deterministic
under test) before the tile adopts it, so brief flapping never reaches the
UI.
Activity stat grid: 7 tiles now (Bottleneck joined), so both the real grid
and its loading skeleton moved to lg:grid-cols-6 (previously mismatched
at 5 vs 6).
README: mentions the new tile where the Activity dashboard's features are
listed.

Test plan

cargo test -p driven-core -p driven-app --lib - 557 + 455 passed
cargo clippy -p driven-core -p driven-app --all-targets -- -D warnings - clean
cargo fmt --check - clean
pnpm -C ui run test:unit - 805 passed (61 files), including new
bottleneck-store.test.ts (7 tests) and bottleneck-stat-tile.test.ts
(9 tests)
npx vue-tsc --noEmit / eslint / prettier --check - clean
Visual baselines regenerated via the just visual-update Docker image,
scoped to activity.spec.ts (light/dark x populated/empty/loading;
error.png unchanged) - screenshots reviewed, tile renders correctly
in both themes at the new 6-column grid.

🤖 Generated with Claude Code

https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v

github-actions · 2026-08-17T19:05:52Z

Coverage

Area	main	this PR	delta
Rust (lib crates)	84.51%	84.52%	+0.01 (OK)
UI (vue/ts)	93.37%	93.40%	+0.03 (OK)

Gate: passed - no coverage regression (epsilon 0.1 pp).

PR #311's coverage gate flagged the new backend surface as under-covered (84.34% vs main's 84.56%). Adds targeted unit tests for every area the gate report highlighted: - bottleneck_hub.rs: extracted the AppHandle-independent decision logic out of tick_once into pure, directly-testable helpers (account_signal, aggregate_accounts, build_snapshot, should_emit), each covering a branch the monolithic async function previously hid from a unit test. Also covers BottleneckHub::push/latest directly. - SyncOrchestrator::pacer_backoff_remaining_ms / backend_label: default (unwired), wired-but-clear (FakePacer), and wired-and-throttled (a real AimdPacer that has actually rate-limited) cases; plus a driven-app test proving FakeOrchestrator inherits the Orchestrator trait's default bodies. - AimdPacer::backoff_remaining_ms: already covered by the previous commit's test, still verified against a FakeClock in and after the backoff window. - IoCounters::add_hashed/snapshot: already covered by the previous commit's test. - Executor credit paths: new inline_upload (buffered, <4MiB) and stream_upload (streaming cpu_stage, >=4MiB) tests asserting the hash counter is credited the full plaintext size in both pipelines. - Scanner credit path: a direct scan_with_priority call (bypassing the scan()/scan_with_progress() wrappers, which always pass None) proving a deep-verify re-hash credits the hash counter. - assembly.rs: bottleneck_backend_label covers all four BackendKind variants with distinct labels. - app_state.rs: bottleneck_hub()/set_bottleneck_task()/shutdown_bottleneck_task() round-trip (register, signal, join, idempotent no-op). Residual uncovered lines are structural: tick_once's AppHandle/AppState plumbing, the bottleneck_status Tauri command (needs a real tauri::State, which this codebase has no harness to construct - matches the pre-existing untested pattern for io_throughput_series/get_sync_status), the emit_sync_bottleneck event (needs a real AppHandle, matches emit_sync_io_throughput's own untested baseline), and the lib.rs sampler spawn/drain wiring (needs full app boot). None of these are new patterns - each mirrors an already-uncovered sibling that predates this PR. Local cargo llvm-cov (same command as .github/workflows/coverage.yml) moved 83.73% -> 83.96% on this diff; cargo test / clippy -D warnings / fmt all clean (1032 driven-core + driven-app tests passing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v

… rollups (#315) ## Summary Closes #305, #306, #307 (v2.12.0 wave, PR3). - **#306 destination folder picker**: client-side sort (Name asc/desc, Modified asc/desc, default Name asc), type-to-filter (substring, current folder only, auto-focus on open, resets on navigation), backend pagination verified complete on all three backends, near-fullscreen sizing (92vw x 85vh capped 1400px) in the add-source wizard modal specifically on the destination-folder and exclusions steps. - **#307 create/rename**: "New folder" button on every browsable backend (Drive, S3, SFTP - reuses `ensure_folder`'s idempotent search-then-create), inline hover rename on Drive + SFTP (`RemoteStore::rename_folder`: Drive is a `files.update` name-only PATCH; SFTP does a real `RENAME` + moves the sidecar, and can mint a new id since SFTP ids are path-derived - the UI replaces the row by its old id with the backend's returned entry). S3 stays on the trait's unsupported default (`BackendKind::supports_rename` = false) and shows a disabled control with an explanatory tooltip instead of hiding the affordance outright. All actions surface errors inline, next to the control that failed - never an alert dialog. - **#305 exclusions rollups**: the backend preview walker now streams a per-directory file-count + byte-size rollup with each batch - a descended directory's rollup starts at 0 and settles upward as its subtree streams in (re-emitting the same row without duplicating it), a pruned excluded directory gets its final answer immediately via one lightweight recursive disk count (no classification). The summary line gained an excluded-bytes "would be freed" stat to match the existing included-bytes one. ## Visual baselines Not touched in this PR. Per the wave lead's sequencing note: `#311` (bottleneck tile) and `#313` (queue nav button, which changes all 104 full-page screenshots) land first; I'll rebase and run `just visual-update` in a follow-up commit on this branch afterward. The one baseline my changes affect is `setup-wizard.spec.ts`'s `step3-source.png` (light + dark) - the `SetupWizard` step 3 embeds the picker's new toolbar (sort/filter/new-folder button). ## Test plan - [x] `cargo test` across touched crates (`driven-remote`, `driven-drive`, `driven-sftp`, `driven-backend`, `driven-core`) and `src-tauri` - 447 src-tauri tests + all crate tests green, including new coverage for rollup settling/pruning, SFTP rename (real move + sidecar + id change + clobber refusal + same-name no-op), and the fake Drive store's rename. - [x] `cargo clippy --all-targets` clean (`driven-remote`/`-drive`/`-sftp`/ `-backend`/`-core` and `src-tauri`). - [x] `cargo fmt --check` clean. - [x] `pnpm run test:unit` - 806 vitest tests green (added ~30: picker sort/filter/create/rename, tree rollup rendering + fill sizing, store rollup settling). - [x] `vue-tsc --noEmit`, `eslint .`, `prettier --check` all clean. - [x] `just e2e-run wizard-first-run local-folder-round-trip` - both PASS against the real containerized app (boots into the setup wizard, renders the picker, completes a real backup/restore round trip). Did not run the full `just e2e` suite (expensive); these two scenarios are the ones that exercise the wizard/picker paths this PR touches. - [ ] Visual regression (`just visual-update`) - deferred per the baseline sequencing above. ## README Updated: the live-exclusion-preview bullet now mentions the per-folder size rollup, and a new bullet describes the picker's sort/filter/new-folder/rename behavior including the S3 rename caveat. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Adds a backend classifier (new src-tauri/src/bottleneck_hub.rs sampler, 1s cadence, idle suppression) that names the current limiting pipeline stage - not_backing_up / disk / network / api / cpu / mixed - by comparing disk-read, net-wire, and a net-new blake3 hash-throughput counter, plus a per-account AimdPacer backoff signal exposed via a new public getter and two default Orchestrator trait methods (pacer_backoff_remaining_ms, backend_label). Hash bytes are credited from the upload pipeline's cpu stage (streamed and buffered) and the scanner's deep-verify re-hash, alongside the existing disk/net IoCounters. Classification streams as a new sync:bottleneck event with a bottleneck_status hydration command. Adds a BottleneckStatTile.vue tile (sibling of ThroughputStatTile: same STAT_TILE chrome, dt/dd typography) to the Activity stat grid, fed by a new Pinia store that debounces + hysteresis-gates the raw ~1s classifications so the tile does not flap. The stat grid moves from 5/6 real/skeleton columns to a consistent lg:grid-cols-6 for both, now that Bottleneck is a 7th tile. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v

PR #311's coverage gate flagged the new backend surface as under-covered (84.34% vs main's 84.56%). Adds targeted unit tests for every area the gate report highlighted: - bottleneck_hub.rs: extracted the AppHandle-independent decision logic out of tick_once into pure, directly-testable helpers (account_signal, aggregate_accounts, build_snapshot, should_emit), each covering a branch the monolithic async function previously hid from a unit test. Also covers BottleneckHub::push/latest directly. - SyncOrchestrator::pacer_backoff_remaining_ms / backend_label: default (unwired), wired-but-clear (FakePacer), and wired-and-throttled (a real AimdPacer that has actually rate-limited) cases; plus a driven-app test proving FakeOrchestrator inherits the Orchestrator trait's default bodies. - AimdPacer::backoff_remaining_ms: already covered by the previous commit's test, still verified against a FakeClock in and after the backoff window. - IoCounters::add_hashed/snapshot: already covered by the previous commit's test. - Executor credit paths: new inline_upload (buffered, <4MiB) and stream_upload (streaming cpu_stage, >=4MiB) tests asserting the hash counter is credited the full plaintext size in both pipelines. - Scanner credit path: a direct scan_with_priority call (bypassing the scan()/scan_with_progress() wrappers, which always pass None) proving a deep-verify re-hash credits the hash counter. - assembly.rs: bottleneck_backend_label covers all four BackendKind variants with distinct labels. - app_state.rs: bottleneck_hub()/set_bottleneck_task()/shutdown_bottleneck_task() round-trip (register, signal, join, idempotent no-op). Residual uncovered lines are structural: tick_once's AppHandle/AppState plumbing, the bottleneck_status Tauri command (needs a real tauri::State, which this codebase has no harness to construct - matches the pre-existing untested pattern for io_throughput_series/get_sync_status), the emit_sync_bottleneck event (needs a real AppHandle, matches emit_sync_io_throughput's own untested baseline), and the lib.rs sampler spawn/drain wiring (needs full app boot). None of these are new patterns - each mirrors an already-uncovered sibling that predates this PR. Local cargo llvm-cov (same command as .github/workflows/coverage.yml) moved 83.73% -> 83.96% on this diff; cargo test / clippy -D warnings / fmt all clean (1032 driven-core + driven-app tests passing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v

#313 (the pending-work queue) merged to main first and added a nav-bar button that appears in the sticky header on every page, including Activity - so rebasing wave-1-bottleneck onto the new main made this branch's own activity.spec.ts baselines doubly stale (missing the nav button on top of already missing the Bottleneck tile). Took main's PNGs wholesale during the rebase conflict, then regenerated via `just visual-update` scoped to activity.spec.ts so light/dark now show the queue nav icon and the Bottleneck tile together. query failure unchanged (no visible delta there). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v

…quit drain Rebases wave-1-debug-diag onto origin/main now that #310-#316 merged (#312's quit-path restructuring in particular). Structural follow-up: - The debug-logging-mode watchdog (debug_mode.rs) previously used the detached memlog.rs pattern (no shutdown tracking). #312 replaced the old shutdown_orchestrators() with a proper ShutdownHandles/ drain_shutdown_handles structure that every other periodic background task (updater, telemetry, iostat, and now #311's bottleneck sampler) registers into for a no-orphan quit drain. Re-homed the watchdog into that same structure: a new DebugModeRuntime (task + shutdown watch, no shared hub - the watchdog only reads/writes settings directly) on AppState, set_debug_mode_task/shutdown_debug_mode_task mirroring set_bottleneck_task/shutdown_bottleneck_task exactly, a debug_mode field on ShutdownHandles, and spawn_watchdog now runs the same select!-on-shutdown-or-tick loop bottleneck_hub/iostat_hub use instead of a bare loop. - Added app_state::tests::debug_mode_runtime_task_and_shutdown_round_trip, mirroring bottleneck's round-trip test. - privacy.png (light+dark) and 9 shell.spec.ts baselines (light+dark) regenerated via `just visual-update` (Docker) - the shell baselines drifted independently of this PR's own diff (same delta across every scenario in both themes), consistent with normal headless-Chromium rendering drift between visual-update runs; all 106 visual specs pass against the regenerated set. No other conflicts: README.md, dtos.rs, settings.rs's redaction code, en-US.json, Activity.vue, and fixtures.ts all auto-merged cleanly with #311's bottleneck-tile additions coexisting alongside this PR's debug logging toggle and diagnostic-bundle changes. Verified after rebase: cargo test -p driven-app --lib (494 passed), cargo clippy --workspace --all-targets -- -D warnings (clean), cargo fmt --all --check (clean), pnpm lint (0 errors), pnpm format:check (clean), pnpm test:unit (861 passed, 64 files), pnpm build / vue-tsc (clean). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v

…dles (#314) ## Summary Closes #309, closes #204. Part of the v2.12.0 wave (PR5). **#204 (diagnostic bundle PII/secret leaks)** - `redact_settings()` used to clone `GlobalSettings` verbatim and patch only `proxy_url`. It now builds a field-for-field `RedactedGlobalSettings` struct, so a future secret-bearing field added to `GlobalSettings` fails to *compile* here until someone decides how to redact it, instead of leaking silently. Fixes the three concrete leaks the issue named: - `pre_backup_hook` / `post_backup_hook` (command lines - a classic home for embedded secrets) are now redacted wholesale (`<hook-redacted: N chars>`), not shipped raw. - `custom_root_ca_path` is now hashed through the same `<path:hash>` scheme the rest of the bundle already uses. - `proxy_url` in PAC mode (a local file path, not a URL) is now hashed instead of only having userinfo-stripping applied (which never matched a bare path). - The issue's "also worth fixing" item (`ProxyError`'s `Display` embedding raw userinfo) was already fixed by #208 - verified via `git blame`, not touched again. **#309 (debug logging mode)** - a new Settings > Privacy & Data toggle ("Debug logging") with an always-visible amber warning panel (shown before the toggle is ever switched on, per the approved mockup), backed by: - A real runtime-reloadable tracing filter (`logging.rs`, `tracing_subscriber::reload`) - flipping the toggle now actually changes the live process's verbosity, no restart needed. This also closes a long-documented gap where `global.log_level` only ever exported `RUST_LOG` for the *next* launch; it now reloads the live filter too (deferred while debug mode is active, so it doesn't undo the debug-mode filter). - A persisted epoch-ms expiry + a boot-time reconcile and periodic watchdog (`debug_mode.rs`) that auto-turns the toggle off 24h after enabling - honoured across a restart, not just while the app keeps running. The watchdog is registered on `AppState` and joined by #312's no-orphan quit drain (`ShutdownHandles`/`drain_shutdown_handles`), the same pattern #311's bottleneck sampler uses. - A rolling log cap that widens from 25 MB to 250 MB while debug mode is on. - The diagnostic bundle gains `DEBUG_MODE.txt` and an unredacted `debug/engine_state.txt` while debug mode is on - the one deliberate exception to the #204 redaction rules, gated on the user's explicit opt-in (every other bundle file stays redacted regardless). - Every bundle now also ships `manifest.txt` (entry name + size), a small bundle-usefulness improvement. - Activity's "Export diagnostic bundle" button shows an amber "Debug data included" chip while debug mode is on. ## Also in this PR - **Rebased onto `main`** after #310-#316 merged. Re-homed the debug-mode watchdog from a detached `memlog.rs`-style task into #312's `ShutdownHandles`/`drain_shutdown_handles` no-orphan quit drain (new `DebugModeRuntime` on `AppState`, `set_debug_mode_task`/`shutdown_debug_mode_task` mirroring `set_bottleneck_task`/`shutdown_bottleneck_task`). - **CodeQL `rust/path-injection` fix** (not a dismissal): two test helpers (`settings.rs`'s pre-existing `seeded_repo()` and this PR's new `debug_mode.rs` one) hand-rolled a temp dir via `std::env::temp_dir().join(format!(...))` before feeding it to `SqliteStateRepo::open` - exactly the pattern this repo's CodeQL rule flags (see the `tempfile` dependency comment in `src-tauri/Cargo.toml`, and PR 151 precedent). Switched both to `tempfile::tempdir().keep()`, an opaque external call CodeQL's dataflow can't see into, so the taint chain never forms. - **Also carries the h2 advisory fix** (RUSTSEC-2026-0258, low severity, unbounded empty DATA frames) - `cargo update -p h2` (0.4.15 -> 0.4.16), lockfile-only, no `Cargo.toml` changes. This advisory is unrelated to this PR's own diff (`git diff` against the pre-PR base shows zero `Cargo.lock` changes before this commit) and would fail `cargo deny` repo-wide on `main` too; landing it here unblocks this PR's `cargo deny` check and delivers the fix to `main` in the same step. ## Test plan - [x] `cargo test -p driven-app --lib` - 494 passed (18 #204 redaction tests with leak-shaped fixtures, incl. one asserting the full serialized bundle JSON end-to-end; 5 debug-mode watchdog/expiry tests; 6 settings-persistence round-trip tests; 1 `AppState` debug-mode task/shutdown round-trip test) - [x] `cargo clippy --workspace --all-targets -- -D warnings` - clean - [x] `cargo fmt --all -- --check` - clean - [x] `cargo build --workspace --tests` - clean - [x] `cargo deny check` - clean (advisories ok, bans ok, licenses ok, sources ok) - [x] `pnpm lint` / `pnpm format:check` / `pnpm test:unit` (861 passed, 64 files) / `pnpm build` (vue-tsc + vite) - all clean, run in the CI job's exact order - [x] Linux visual baselines regenerated via `just visual-update` (Docker) - `privacy.png` (light+dark) plus 9 `shell.spec.ts` baselines (light+dark) that had drifted independently of this PR; all 106 visual specs pass - [x] README updated (Features list + comparison-table footnote ³⁴) 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_019xKUm9vH4ifb5LHR5szy1v --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

🤖 I have created a release *beep* *boop* --- ## [2.12.0](v2.11.1...v2.12.0) (2026-08-18) ### Features * **app:** opt-in debug logging mode and safer, richer diagnostic bundles ([#314](#314)) ([33c281c](33c281c)) * **core:** allow nested backup sources when the parent excludes the child ([#294](#294)) ([0b62df9](0b62df9)) * **core:** live exclusion pickup and a visible pending-work queue ([#313](#313)) ([e6427c7](e6427c7)) * live bottleneck indicator on the Activity dashboard ([#311](#311)) ([2d9d763](2d9d763)) * **ui:** folder picker sort/filter/create/rename and exclusions size rollups ([#315](#315)) ([7e87341](7e87341)) ### Bug Fixes * **app:** never freeze on tray quit during a backup; quitting tray state; honest recovery status ([#312](#312)) ([f951cde](f951cde)) * clear the attention banner after a passing run and make source removal backend-aware ([#310](#310)) ([6d8e1ab](6d8e1ab)) * **s3:** honest per-source versioning across backends and multipart-upload leak sweep ([#316](#316)) ([d462592](d462592)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

github-project-automation Bot added this to Driven Aug 17, 2026

github-project-automation Bot moved this to Todo in Driven Aug 17, 2026

pmaxhogan force-pushed the wave-1-bottleneck branch from 8d22e8e to 5b157d8 Compare August 17, 2026 19:47

pmaxhogan mentioned this pull request Aug 17, 2026

flaky: driven-sftp a_session_can_read_and_write_the_served_directory under coverage #268

Open

pmaxhogan and others added 3 commits August 17, 2026 15:22

pmaxhogan force-pushed the wave-1-bottleneck branch from 5b157d8 to 7905a97 Compare August 17, 2026 20:24

pmaxhogan enabled auto-merge (squash) August 17, 2026 20:25

pmaxhogan merged commit 2d9d763 into main Aug 17, 2026
18 checks passed

pmaxhogan deleted the wave-1-bottleneck branch August 17, 2026 20:48

github-project-automation Bot moved this from Todo to Done in Driven Aug 17, 2026

pmaxhogan mentioned this pull request Aug 17, 2026

chore(main): release 2.12.0 #295

Merged

pmaxhogan mentioned this pull request Aug 18, 2026

feat(app): opt-in debug logging mode and safer, richer diagnostic bundles #314

Merged

8 tasks

Navigation Menu

feat: live bottleneck indicator on the Activity dashboard - #311

feat: live bottleneck indicator on the Activity dashboard#311
pmaxhogan merged 3 commits into
mainfrom
wave-1-bottleneck

pmaxhogan commented Aug 17, 2026

Uh oh!

github-actions Bot commented Aug 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

pmaxhogan commented Aug 17, 2026

Summary

Frontend

Test plan

Uh oh!

github-actions Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented Aug 17, 2026 •

edited

Loading