Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@
_(nothing active — pick the next batch from below)_

## Deferred / future refinements
- [ ] **Auto-summarize — queue-decoupled background run.** P13a-2 generates the auto-summary **inline** in
`_persistCompleted` before the next download pumps (gated on "model present" so it can't stall on a
fetch), exactly like `autoTranscribe`. Generation is heavier than whisper-tiny, so a fuller design
would run it **off the critical path** after the queue drains. Shares the existing "queue-decoupled
background transcription" deferral below; the **LLM + Cozo HNSW RAM co-residency** check (P13d) also
applies once auto-summary and the graph index can be resident together. *(From P13a-2.)*
- [ ] **AI summary — staleness on later transcript.** The cached `aiSummary` (P13a) is generated from
`transcript ?? description` at the moment the user runs it; if a transcript is added *after* a
description-based summary, the cache isn't auto-invalidated (the user can hit **Regenerate**). A
Expand Down
9 changes: 9 additions & 0 deletions docs/VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -927,6 +927,15 @@ entries, or verify after P11c lands.)*
- [ ] On a capable device with generation **not yet enabled**: tapping **Summarize with AI** shows a
"set up text generation" hint and opens **AI settings** (the on-ramp).

### P13a-2 — Auto-summarize on download *(install `app-arm64-v8a-debug.apk`; needs a capable device)*
- [ ] AI settings → with text generation enabled, the **Auto-summarize new downloads** toggle is visible;
enable it. Finish a download (an item with a description/transcript) → an **AI summary** appears on
the item **and** an Activity Inbox entry ("Summary ready"), **fully offline**.
- [ ] With auto-summarize on but **no generation model downloaded**: a download produces a one-time
"Finish setting up summaries" inbox nudge that opens **AI settings** (no summary written).
- [ ] **Default off** (and when generation is disabled): downloads produce **no** auto-summary and no AI
nudge; the queue still drains normally; the on-demand "Summarize with AI" still works.

### P13 (later subphases)
- [ ] **Transcription / summarization / translation / OCR** each work (capability-gated) and write
results back to the item.
Expand Down
21 changes: 21 additions & 0 deletions docs/design/P13-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,27 @@ The first **real** generation feature — an LLM TL;DR layered on the existing e
end-to-end widget flow is APK-verified (the item-detail screen's player/related shimmer makes a full
`pumpAndSettle` widget test unreliable — same boundary as the P10f-2 transcript flow).

#### `[~]` P13a-2 — Opt-in auto-summarize on download *(generation; APK)*
A maintainer-requested follow-up: auto-generate the abstractive summary for newly downloaded items in the
background, **opt-in (default off)**, mirroring the `autoTranscribe` precedent (P12e-3).
- `SettingsModel.autoSummarizeOnDownload` (+ setter); a toggle in the AI-settings generation card, shown only
when generation is enabled.
- In `queue_controller._persistCompleted` (after the auto-transcribe block, so a just-built transcript is the
source): for each completed item, gated by `autoSummarizeOnDownload && generationEnabled` and a pure
`autoSummaryDecision` (skip / needsModel / summarize) — **runs only when the model is already downloaded
(`ensureReady`, never fetches)**; collects `generate(buildSummaryPrompt(text))` and persists via
`updateAiSummary`.
- Activity Inbox: a `category: ai` success entry (`summary_$id`), or a one-time "finish setting up summaries"
nudge (`summary_needs_model`) when opted in but no model is downloaded.
- **Exit / review:** with auto-summarize on + a downloaded model, a finished download gets a summary + an
inbox entry **offline**; no model → one nudge; default-off / generation-off → nothing auto-runs; the queue
still drains. APK spot-check.
- **Status:** implemented (CI-green) — settings field + setter, `autoSummaryDecision` (unit-tested), queue
integration + two inbox posts, the AI-settings toggle, and three queue tests (ready → summary + ai entry;
no-model → nudge; generation-off → no-op). No schema change (reuses P13a's columns). **Pending APK
spot-check.** Shares the queue-decoupled-background-AI deferral (inline-before-next-pump like
`autoTranscribe`) and the LLM+HNSW RAM co-residency check (P13d) — both in `BACKLOG.md`.

### `[ ]` P13b — Translation & OCR (ML Kit) *(native; new deps; APK; split into 2 PRs)*
On-device text intelligence that is **device-universal-ish** — gated on ML Kit + opt-in, not the RAM tier.
Adds `google_mlkit_translation` / `google_mlkit_text_recognition`; measure APK-size impact in the first build.
Expand Down
31 changes: 31 additions & 0 deletions lib/features/library/presentation/ai_summary.dart
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,34 @@ AiSummaryAction aiSummaryAction({
if (!modelReady) return AiSummaryAction.offerDownload;
return AiSummaryAction.summarizeNow;
}

/// What auto-summarize-on-download (P13a-2) should do for a freshly downloaded
/// item, assuming the feature is opted in. A pure decision so the queue path is
/// testable (mirrors `transcribeFallbackAction`). The caller only enters this
/// when `autoSummarizeOnDownload` **and** generation are enabled — so this
/// decides per item, given its content + model readiness.
enum AutoSummaryDecision {
/// Nothing to do — no text to summarize, or a summary already exists.
skip,

/// Would summarize, but the generation model isn't downloaded — nudge once.
needsModel,

/// Summarize this item now (model ready).
summarize,
}

/// [hasText] is whether the item has a `transcript ?? description` to condense;
/// [alreadySummarized] is whether an `aiSummary` is already stored; [modelReady]
/// is whether the generation model is downloaded (`engine.ensureReady()` — no
/// fetch).
AutoSummaryDecision autoSummaryDecision({
required bool hasText,
required bool alreadySummarized,
required bool modelReady,
}) {
if (!hasText || alreadySummarized) return AutoSummaryDecision.skip;
return modelReady
? AutoSummaryDecision.summarize
: AutoSummaryDecision.needsModel;
}
90 changes: 90 additions & 0 deletions lib/features/queue/presentation/queue_controller.dart
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import 'dart:io';
import 'package:drift/drift.dart' show Value;
import 'package:flutter/widgets.dart' show AppLifecycleState;
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:grabbit/core/ai/generation_provider.dart';
import 'package:grabbit/core/ai/transcription_provider.dart';
import 'package:grabbit/core/db/database.dart';
import 'package:grabbit/core/db/database_provider.dart';
Expand All @@ -24,6 +25,7 @@ import 'package:grabbit/features/downloader/presentation/error_messages.dart';
import 'package:grabbit/features/library/data/library_repository.dart';
import 'package:grabbit/features/library/data/metadata_repository.dart';
import 'package:grabbit/features/library/data/transcript_service.dart';
import 'package:grabbit/features/library/presentation/ai_summary.dart';
import 'package:grabbit/features/notifications/data/notification_enums.dart';
import 'package:grabbit/features/notifications/data/notifications_repository.dart';
import 'package:grabbit/features/notifications/data/system_notification_service.dart';
Expand All @@ -45,6 +47,10 @@ typedef _PersistResult = ({
// P12e-3: a caption-less item was skipped because transcription is opted in
// but no whisper model is downloaded → prompt the user to finish setup.
bool transcriptionNeedsModel,
// P13a-2: count of items auto-summarized, and whether auto-summarize is opted
// in but the generation model isn't downloaded → prompt to finish setup.
int summaryCount,
bool summaryNeedsModel,
});

class QueueConfig {
Expand Down Expand Up @@ -385,6 +391,33 @@ class QueueController extends _$QueueController {
dedupeKey: 'transcribe_needs_model',
);
}
// P13a-2: an auto-summary was generated for the new download.
if (result.summaryCount > 0) {
await center.post(
category: NotificationCategory.ai,
severity: NotificationSeverity.success,
title: queued.title,
body: result.summaryCount > 1
? '${result.summaryCount} summaries ready'
: 'Summary ready',
targetRoute: route,
itemId: single ? result.primaryId : null,
dedupeKey: 'summary_$id',
);
}
// P13a-2: opted into auto-summarize but the generation model isn't
// downloaded. Nudge once (deduped) to finish setup.
if (result.summaryNeedsModel) {
await center.post(
category: NotificationCategory.ai,
severity: NotificationSeverity.info,
title: 'Finish setting up summaries',
body:
'Download a text-generation model to auto-summarize new downloads.',
targetRoute: '/settings/ai',
dedupeKey: 'summary_needs_model',
);
}
await _maybeNotifyOs(
taskId: id,
title: queued.title,
Expand Down Expand Up @@ -543,6 +576,8 @@ class QueueController extends _$QueueController {
itemCount: 0,
transcriptCount: 0,
transcriptionNeedsModel: false,
summaryCount: 0,
summaryNeedsModel: false,
);
// Files land in a per-task subfolder (see YtDlpHost `-o`): the task id names
// the folder, the user's template names the file inside it.
Expand Down Expand Up @@ -685,11 +720,66 @@ class QueueController extends _$QueueController {
}
}

// P13a-2: auto-generate an abstractive summary for the freshly downloaded
// items when the user opted in and a generation model is already downloaded
// (never fetches mid-queue — mirrors auto-transcribe). Runs after the
// transcript block so a just-built transcript is the preferred source.
var summaryCount = 0;
var summaryNeedsModel = false;
if (settings.autoSummarizeOnDownload && settings.generationEnabled) {
final generation = ref.read(generationEngineProvider);
final genReady = await generation.ensureReady();
final modelId = ref.read(activeGenerationModelProvider)?.id;
final metadata = ref.read(metadataRepositoryProvider);
for (final (i, _) in outputs.media.indexed) {
final itemId = single ? id : '${id}__$i';
final meta = await (db.select(
db.mediaMetadata,
)..where((m) => m.itemId.equals(itemId))).getSingleOrNull();
final text = meta?.transcript ?? meta?.description;
switch (autoSummaryDecision(
hasText: text != null && text.trim().isNotEmpty,
alreadySummarized: meta?.aiSummary?.trim().isNotEmpty ?? false,
modelReady: genReady,
)) {
case AutoSummaryDecision.skip:
continue;
case AutoSummaryDecision.needsModel:
summaryNeedsModel = true;
continue;
case AutoSummaryDecision.summarize:
try {
final p = buildSummaryPrompt(text!);
final buffer = StringBuffer();
await for (final token in generation.generate(
p.prompt,
systemPrompt: p.systemPrompt,
)) {
buffer.write(token);
}
final summary = buffer.toString().trim();
if (summary.isNotEmpty) {
await metadata.updateAiSummary(
itemId,
summary,
modelId: modelId,
);
summaryCount++;
}
} catch (_) {
// A per-item summary failure must not fail the download.
}
}
}
}

return (
primaryId: single ? id : '${id}__0',
itemCount: outputs.media.length,
transcriptCount: transcriptCount,
transcriptionNeedsModel: transcriptionNeedsModel,
summaryCount: summaryCount,
summaryNeedsModel: summaryNeedsModel,
);
}

Expand Down
6 changes: 6 additions & 0 deletions lib/features/settings/data/settings_model.dart
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,12 @@ abstract class SettingsModel with _$SettingsModel {
// eligibility-guarded by activeGenerationModelProvider.
@Default(false) bool generationEnabled,
@Default('') String selectedGenerationModelId,
// P13a-2: auto-generate the abstractive summary for a newly downloaded item
// in the background. Opt-in (defaults off); only runs when text generation
// is enabled and its model is already downloaded (no surprise mid-queue
// fetch — mirrors `autoTranscribe`). The on-demand summary on item detail
// (P13a) works regardless.
@Default(false) bool autoSummarizeOnDownload,
// On-device speech transcription (P12e). Opt-in (defaults off); the whisper
// model is downloaded only when the user enables it + picks a model.
// `selectedTranscriptionModelId` empty = the device-tier recommendation;
Expand Down
42 changes: 42 additions & 0 deletions lib/features/settings/presentation/ai_settings_screen.dart
Original file line number Diff line number Diff line change
Expand Up @@ -587,13 +587,55 @@ class _GenerationCard extends ConsumerWidget {
child: SettingsCard(
children: [
for (final m in eligible) _GenerationModelTile(model: m),
const _AutoSummarizeTile(),
const _GenerationSelfTestTile(),
],
),
);
}
}

/// Opt-in (P13a-2): auto-generate the abstractive summary for newly downloaded
/// items in the background. Shown only when text generation is enabled (a model
/// is active); it runs only when that model is already downloaded — the
/// on-demand "Summarize with AI" on item detail works regardless.
class _AutoSummarizeTile extends ConsumerWidget {
const _AutoSummarizeTile();

@override
Widget build(BuildContext context, WidgetRef ref) {
final enabled = ref.watch(
settingsControllerProvider.select(
(s) => s.value?.generationEnabled ?? false,
),
);
if (!enabled) return const SizedBox.shrink();
final auto = ref.watch(
settingsControllerProvider.select(
(s) => s.value?.autoSummarizeOnDownload ?? false,
),
);
return SwitchListTile(
secondary: const InfoHintButton(
InfoHint(
title: 'Auto-summarize new downloads',
body:
'Generate an AI summary for each new download automatically, in '
'the background — all on-device. Runs only when a text-generation '
'model is downloaded; you can always summarize an item by hand '
'from its detail screen.',
),
),
title: const Text('Auto-summarize new downloads'),
subtitle: const Text('Summarize each download in the background'),
value: auto,
onChanged: (v) => ref
.read(settingsControllerProvider.notifier)
.setAutoSummarizeOnDownload(v),
);
}
}

/// One selectable generation model row: badge (Recommended / size band) + size,
/// a radio-like check for the active selection. Selecting it opts in + downloads
/// (storage-guarded); selecting the active one again turns generation off.
Expand Down
4 changes: 4 additions & 0 deletions lib/features/settings/presentation/settings_controller.dart
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,10 @@ class SettingsController extends _$SettingsController {
Future<void> setSelectedGenerationModelId(String id) async =>
_update((await future).copyWith(selectedGenerationModelId: id));

/// Auto-summarize newly downloaded items in the background (P13a-2).
Future<void> setAutoSummarizeOnDownload(bool value) async =>
_update((await future).copyWith(autoSummarizeOnDownload: value));

/// On-device transcription opt-in (P12e).
Future<void> setTranscriptionEnabled(bool value) async =>
_update((await future).copyWith(transcriptionEnabled: value));
Expand Down
46 changes: 46 additions & 0 deletions test/features/library/ai_summary_test.dart
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,50 @@ void main() {
);
});
});

group('autoSummaryDecision (P13a-2)', () {
test('no text → skip', () {
expect(
autoSummaryDecision(
hasText: false,
alreadySummarized: false,
modelReady: true,
),
AutoSummaryDecision.skip,
);
});

test('already summarized → skip (no rework)', () {
expect(
autoSummaryDecision(
hasText: true,
alreadySummarized: true,
modelReady: true,
),
AutoSummaryDecision.skip,
);
});

test('text + no summary + model not ready → needs model (nudge)', () {
expect(
autoSummaryDecision(
hasText: true,
alreadySummarized: false,
modelReady: false,
),
AutoSummaryDecision.needsModel,
);
});

test('text + no summary + model ready → summarize', () {
expect(
autoSummaryDecision(
hasText: true,
alreadySummarized: false,
modelReady: true,
),
AutoSummaryDecision.summarize,
);
});
});
}
Loading
Loading