Skip to content

feat: local on-device LLMs + task extraction from transcripts#1

Merged
Aman-Verma-28 merged 1 commit into
mainfrom
feat/local-llm-task-extraction
Jun 24, 2026
Merged

feat: local on-device LLMs + task extraction from transcripts#1
Aman-Verma-28 merged 1 commit into
mainfrom
feat/local-llm-task-extraction

Conversation

@Aman-Verma-28

Copy link
Copy Markdown
Contributor

Before Submitting This PR

⚠️ Heads up (feature freeze): This is a new feature, not a bug fix. Per the template and CONTRIBUTING, new features need community support gathered in Discussions before merge. No discussion has been opened yet — please start one and link it under Community Feedback, or treat this as a proposal/draft.

Please confirm you have done the following:

  • I have searched existing issues and pull requests (including closed ones) to ensure this isn't a duplicate
  • I have read CONTRIBUTING.md

If this is a feature or change that was previously closed/rejected:

  • I have explained in the description below why this should be reconsidered
  • I have gathered community feedback (link to discussion below)

Human Written Description

TODO: author to fill in.

Related Issues/Discussions

Fixes #
Discussion:

Community Feedback

Not yet gathered.

What this PR does

Adds four open-weight, on-device LLMs — DeepSeek (R1-Distill), Qwen 2.5, Google Gemma 2, and Mistral — as downloadable GGUF models that run fully offline via llama.cpp, alongside the existing voice models. On top of that, a new "Tasks" feature extracts a structured task list (title / description / priority) from any transcript in the History view, with no API keys and no network at inference time.

Backend

  • New ModelKind { Transcription, Llm } so LLM models share the download/verify/storage machinery but never appear in the voice model selector.
  • LlmManager mirrors TranscriptionManager (lazy load, idle-unload, panic-safe). Uses the GGUF's embedded chat template, strips DeepSeek-R1 <think> blocks, and constrains output to valid JSON with a GBNF grammar derived from the task schema.
  • Models download directly from Hugging Face with pinned SHA256 (small + standard size per family).
  • tasks_* settings, task-prompt CRUD commands, and a tasks_json history-DB column (append-only migration) so generated tasks persist.

Frontend

  • "Tasks" settings panel (model picker reusing ModelCard + prompt editor), an enable toggle, a useLlmModels hook, and a TaskListPanel with generate/regenerate, priority badges, and copy.

Testing

  • cargo check clean (0 errors/warnings from new code); cargo test tasks:: → 4/4 pass (JSON/<think>/bare-array/empty parsing)
  • tsc --noEmit clean; eslint src/... clean; Prettier applied
  • ⚠️ Build note: native deps (whisper.cpp + llama.cpp) must compile with Apple Clang, not Homebrew GCC. Verified locally with CXX=/usr/bin/clang++ CC=/usr/bin/clang (and without CMAKE_POLICY_VERSION_MINIMUM). CI/tauri-action uses the standard toolchain, so this should be fine there.
  • ⏳ Not yet done: end-to-end manual run (download a model → generate tasks → restart to confirm persistence). Help with additional testing welcome.

Translations

Only en/translation.json has the ~37 new keys; the other 19 locales fall back to English at runtime. check:translations will report them missing until translators add them (per CONTRIBUTING_TRANSLATIONS.md).

Screenshots/Videos (if applicable)

AI Assistance

  • No AI was used in this PR
  • AI was used (please describe below)

If AI was used:

  • Tools used: Claude Code (Opus 4.8)
  • How extensively: Substantial. AI explored the codebase, designed the approach, and wrote the implementation (Rust backend, React/TS frontend, bindings, i18n) and tests. A human should review and complete the human-written description, gather community feedback, and run end-to-end manual testing before merge.

🤖 Generated with Claude Code

Adds DeepSeek, Qwen, Google Gemma, and Mistral as downloadable on-device
GGUF models (via llama.cpp) alongside the existing voice models, plus a
"Tasks" feature that extracts a structured task list from a transcript in
the History view. This turns voice notes into actionable to-do items fully
offline, with no API keys.

- New ModelKind discriminator so LLM models coexist with voice models
  without affecting the transcription model selector
- LlmManager mirrors TranscriptionManager (lazy load, idle-unload); output
  is constrained to JSON via a GBNF grammar derived from the task schema
- Models download directly from Hugging Face with pinned SHA256
- Tasks settings panel (model + prompt), generation from History, and
  persistence in the history DB

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Aman-Verma-28
Aman-Verma-28 merged commit 63dee1d into main Jun 24, 2026
1 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant