Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@
_(nothing active — pick the next batch from below)_

## Deferred / future refinements
- [ ] **GraphRAG — store only the *cited* sources.** P13d-2a persists **all** retrieved sources as a turn's
citations (and renders every `[n]` the model emits); it doesn't prune to the subset the answer actually
cites. If answers reference few sources, post-parse the `[n]` markers and persist only those (smaller
`citationsJson`, tidier Sources row). *(From P13d-2a.)*
- [ ] **GraphRAG — surface retrieval/generation errors per turn.** An `InferenceException` mid-answer sets a
screen-level error banner but persists no assistant row, so the failed question lingers without a reply
bubble. Consider a retry affordance or an inline "couldn't answer" turn. *(From P13d-2a.)*
- [ ] **GraphRAG — retrieval-only answer persistence.** On low / ineligible tiers (`ragAvailability ==
retrievalOnly`) P13d falls back to an ephemeral "most relevant items" answer that **isn't** saved to the
chat history (nothing is generated to revisit). Decide at d-3 whether these should be persisted as a
Expand Down
12 changes: 11 additions & 1 deletion docs/SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,9 @@ IntColumn orderIndex; // v3 (P9d): queue reor
// settings (key/value JSON, single row)
// notifications (v9, P11): id, createdAt, category, severity, title, body,
// targetRoute?, itemId?, taskId?, readAt?, dedupeKey?, expiresAt? — Activity Inbox
// chats (v14, P13d-2a): id, title, createdAt, updatedAt, archivedAt? — "Ask your library"
// chat_messages (v14, P13d-2a): id(autoinc), chatId→chats(cascade), role, content,
// citationsJson?, createdAt
```

> **Forward seam (v2 Things Engine).** A generic **`things`** table (schema.org Things as JSON-LD +
Expand All @@ -131,7 +134,7 @@ IntColumn orderIndex; // v3 (P9d): queue reor
> the derived index, and `things.id` is kept alignable to `media_items.id` (no FK). See ADR-0001
> (schema-as-data) and ADR-0003 (MediaObject bridge); overview in `docs/things-engine.md`.

Migration strategy: Drift `schemaVersion` (currently **10**); write `MigrationStrategy`
Migration strategy: Drift `schemaVersion` (currently **14**); write `MigrationStrategy`
steps; never drop user data without migration. Add a schema test on bump (upgrade tests
live in `test/core/db/database_test.dart`). **v3 (P9a)** adds
`media_items.{isFavorite,contentHash,lastAccessedAt}` + `download_tasks.orderIndex` and
Expand Down Expand Up @@ -161,6 +164,13 @@ decode via the `image` package) and best-effort backfilled for legacy items by `
**v10 (P12f)** adds the empty **`things`** table (schema.org Things as JSON-LD + promoted
`name`/`url`/`created_at`/`updated_at`; `id` alignable to `media_items.id`, **no FK**) — the v2
Things-Engine forward seam, created empty and **unused in v1** (Drift canonical).
**v11 (P13a)** adds `media_metadata.{aiSummary,aiSummaryModelId}` (cached on-device LLM summary + its
model). **v12 (P13b-1)** adds `media_metadata.ocrText` and extends `media_fts` with an `ocr` column
(the FTS5 table is dropped + rebuilt by the migration since FTS5 can't `ALTER ADD COLUMN`). **v13 (P13c-2)**
adds `media_tags.source` (`'user'` default; `'ai'` for auto-applied tags). **v14 (P13d-2a)** adds the
**`chats`** + **`chat_messages`** tables backing the "Ask your library" GraphRAG chat — `chats`
(id/title/createdAt/updatedAt/`archivedAt?`) and `chat_messages` (autoinc id, `chatId`→`chats` **FK
cascade**, role, content, `citationsJson?`, createdAt); created by `m.createTable` (no data migration).

---

Expand Down
13 changes: 13 additions & 0 deletions docs/VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -990,6 +990,19 @@ entries, or verify after P11c lands.)*
native path). It's exercised by unit tests (fake embedder + graph + seeded in-memory metadata). The
end-to-end **"Ask your library"** flow is verified at P13d-2 (chat screen + generation).

### P13d-2a — "Ask your library" chat *(install `app-arm64-v8a-debug.apk`; needs a capable device + a downloaded generation model)*
- [ ] On a capable device with a generation model set up, the Dashboard shows an **"Ask"** entry; tapping it
opens the chat. (On a low-end device, or with the graph index unavailable, the entry is absent.)
- [ ] Ask a natural-language question about your library → the answer **streams in**, is **grounded** in your
items, and shows inline **`[n]` citations** plus a **Sources** row — **fully offline** (airplane mode).
- [ ] Tapping a citation (inline `[n]` or a Sources chip) **opens the cited item**.
- [ ] Ask a follow-up → it still answers (fresh retrieval + prior turns as context); the turns **persist**
(the transcript stays on screen for the session).
- [ ] Ask something your library can't answer → a graceful **"couldn't find anything"** reply (no invented
answer).
- [ ] With generation **not** set up (eligible device, no model), sending shows the **on-ramp** snackbar and
routes to AI settings.

### P13 (later subphases)
- [ ] **Transcription / summarization / translation / OCR** each work (capability-gated) and write
results back to the item.
Expand Down
15 changes: 13 additions & 2 deletions docs/design/P13-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,13 +239,24 @@ Incapable / low tiers fall back to an ephemeral **retrieval-only** answer (d-3).
well-formed, size-bounded, history-aware prompt; degrades to empty-sources when retrieval is unavailable;
covered by unit tests. ✓

#### `[ ]` P13d-2a — Chat schema + Ask screen (single conversation) *(native; APK)*
#### `[~]` P13d-2a — Chat schema + Ask screen (single conversation) *(native; APK)*
- Drift **`chats` + `chat_messages`** schema; a dedicated **"Ask your library"** screen from the Dashboard
that runs P13d-1's per-turn fresh retrieval + bounded history through `GenerationEngine.generate()` and
**streams** a grounded answer with **tappable citations** deep-linking to the cited items. Generation-gated
via `aiSummaryAction` (on-ramp when no model).
- **Status:** implemented (CI-green). **Schema v13→v14** (`Chats`: id/title/createdAt/updatedAt/`archivedAt?`;
`ChatMessages`: autoinc id, `chatId`→`Chats` FK cascade, role, content, `citationsJson?`, createdAt —
forward-included so d-2b needs no further migration). New `lib/features/ai/data/chat_repository.dart`
(`ChatRepository` + `chatRepositoryProvider`/`chatMessagesProvider`); `lib/features/ai/presentation/`
`ask_chat.dart` (pure — `messagesToHistory`, `encode`/`decodeCitations`, `parseCitationSpans`),
`ask_controller.dart` (`AskController`: create-on-first-send → append user → re-retrieve with bounded history
→ stream grounded answer → persist with citations; no-sources → graceful reply, no LLM call),
`ask_screen.dart` (transcript + streaming bubble + inline `[n]`/Sources citations + gated input);
`dashboard/.../widgets/ask_entry_tile.dart` (auto-hides off the full-generation path); `/ask` route.
Tests: migration v13→v14 (+ cascade), repo, the pure helpers, the controller (full/no-sources/error), and
the entry-tile gating. **No new deps.**
- **Exit / review:** ask a natural-language question on a capable device → a streamed, grounded, cited answer
**offline**; the turn persists; citations navigate. APK spot-check.
**offline**; the turn persists; citations navigate. APK spot-check. ✓ (CI parts) · APK owed

#### `[ ]` P13d-2b — Conversation list + manage *(native)*
- A conversation **list** with **continue / rename / archive / delete**; resuming a chat re-feeds the bounded
Expand Down
40 changes: 39 additions & 1 deletion lib/core/db/database.dart
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,36 @@ class AppSettings extends Table {
Set<Column<Object>> get primaryKey => {id};
}

/// P13d-2a: a persisted "Ask your library" GraphRAG conversation. Each chat is a
/// multi-turn thread of [ChatMessages]; every turn re-retrieves fresh sources
/// and feeds back a bounded slice of this history. `title` is derived from the
/// first question (renamable in d-2b); `archivedAt` (null = active) backs the
/// d-2b archive action. Forward-compatible so d-2b needs no further migration.
class Chats extends Table {
TextColumn get id => text()(); // 'chat_<micros>'
TextColumn get title => text()();
DateTimeColumn get createdAt => dateTime()();
DateTimeColumn get updatedAt => dateTime()(); // bumped each turn → list sort
DateTimeColumn get archivedAt => dateTime().nullable()(); // null = active

@override
Set<Column<Object>> get primaryKey => {id};
}

/// P13d-2a: one message in a [Chats] thread. `id` autoincrements for natural
/// per-chat ordering; deleting a chat cascades its messages. `citationsJson`
/// (assistant turns only) stores the cited sources as compact JSON so the inline
/// `[n]` citations stay tappable after the chat is reloaded.
class ChatMessages extends Table {
IntColumn get id => integer().autoIncrement()();
TextColumn get chatId =>
text().references(Chats, #id, onDelete: KeyAction.cascade)();
TextColumn get role => text()(); // user | assistant
TextColumn get content => text()();
TextColumn get citationsJson => text().nullable()();
DateTimeColumn get createdAt => dateTime()();
}

@DriftDatabase(
tables: [
MediaItems,
Expand All @@ -213,14 +243,16 @@ class AppSettings extends Table {
AppSettings,
Notifications,
Things,
Chats,
ChatMessages,
],
)
class AppDatabase extends _$AppDatabase {
AppDatabase([QueryExecutor? executor])
: super(executor ?? driftDatabase(name: 'grabbit'));

@override
int get schemaVersion => 13;
int get schemaVersion => 14;

@override
MigrationStrategy get migration => MigrationStrategy(
Expand Down Expand Up @@ -299,6 +331,12 @@ class AppDatabase extends _$AppDatabase {
).get()).isNotEmpty;
if (hasMediaTags) await m.addColumn(mediaTags, mediaTags.source);
}
if (from < 14) {
// P13d-2a: persisted "Ask your library" chats. New tables only — no
// data migration (mirrors the v8→v9 notifications / v9→v10 things steps).
await m.createTable(chats);
await m.createTable(chatMessages);
}
await _createIndices();
await _createFtsObjects();
},
Expand Down
7 changes: 7 additions & 0 deletions lib/core/routing/app_router.dart
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import 'package:grabbit/features/library/presentation/collections_screen.dart';
import 'package:grabbit/features/library/presentation/entity_hub_screen.dart';
import 'package:grabbit/features/library/presentation/home_screen.dart';
import 'package:grabbit/features/library/presentation/item_detail_screen.dart';
import 'package:grabbit/features/ai/presentation/ask_screen.dart';
import 'package:grabbit/features/ai/presentation/graph_view_screen.dart';
import 'package:grabbit/features/library/presentation/media_studio_screen.dart';
import 'package:grabbit/features/library/presentation/metadata_edit_screen.dart';
Expand Down Expand Up @@ -254,6 +255,12 @@ GoRouter appRouter(Ref ref) {
parentNavigatorKey: _rootNavigatorKey,
builder: (context, state) => const InboxScreen(),
),
GoRoute(
path: '/ask',
name: 'ask',
parentNavigatorKey: _rootNavigatorKey,
builder: (context, state) => const AskScreen(),
),
],
);
}
Expand Down
84 changes: 84 additions & 0 deletions lib/features/ai/data/chat_repository.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import 'package:drift/drift.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:grabbit/core/db/database.dart';
import 'package:grabbit/core/db/database_provider.dart';

/// Persistence for the "Ask your library" GraphRAG chat (P13d-2a): a thread of
/// [Chat]s, each a list of ordered [ChatMessage]s. Drift stays canonical; this
/// repo only reads/writes the `chats` + `chat_messages` tables. The conversation
/// list + rename/archive/delete land in d-2b on the same schema.
class ChatRepository {
ChatRepository(this._db);

final AppDatabase _db;

/// Creates a new chat with [title] (derived from the first question) and
/// returns its id. `createdAt`/`updatedAt` start equal.
Future<String> createChat(String title) async {
final now = DateTime.now();
final id = 'chat_${now.microsecondsSinceEpoch}';
await _db
.into(_db.chats)
.insert(
ChatsCompanion.insert(
id: id,
title: title,
createdAt: now,
updatedAt: now,
),
);
return id;
}

/// Appends a message to [chatId] and bumps the chat's `updatedAt` (so the
/// d-2b list sorts most-recent-first). [citationsJson] is set on assistant
/// turns only.
Future<void> appendMessage(
String chatId, {
required String role,
required String content,
String? citationsJson,
}) async {
await _db.transaction(() async {
await _db
.into(_db.chatMessages)
.insert(
ChatMessagesCompanion.insert(
chatId: chatId,
role: role,
content: content,
citationsJson: Value(citationsJson),
createdAt: DateTime.now(),
),
);
await (_db.update(_db.chats)..where((t) => t.id.equals(chatId))).write(
ChatsCompanion(updatedAt: Value(DateTime.now())),
);
});
}

/// Live transcript for [chatId], ordered oldest-first.
Stream<List<ChatMessage>> watchMessages(String chatId) =>
(_db.select(_db.chatMessages)
..where((t) => t.chatId.equals(chatId))
..orderBy([(t) => OrderingTerm.asc(t.id)]))
.watch();

/// One-shot read of [chatId]'s messages (oldest-first) — used to build the
/// bounded history window for the next turn's prompt.
Future<List<ChatMessage>> messagesForChat(String chatId) =>
(_db.select(_db.chatMessages)
..where((t) => t.chatId.equals(chatId))
..orderBy([(t) => OrderingTerm.asc(t.id)]))
.get();
}

final chatRepositoryProvider = Provider<ChatRepository>(
(ref) => ChatRepository(ref.watch(appDatabaseProvider)),
);

/// Live transcript provider. Hand-written (returns the Drift row type
/// [ChatMessage]) per CLAUDE.md §8 — `riverpod_generator` can't return rows.
final chatMessagesProvider = StreamProvider.family<List<ChatMessage>, String>(
(ref, chatId) => ref.watch(chatRepositoryProvider).watchMessages(chatId),
);
111 changes: 111 additions & 0 deletions lib/features/ai/presentation/ask_chat.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
/// Pure, engine-free helpers for the "Ask your library" chat (P13d-2a): turning
/// persisted messages into bounded RAG history, the citation persistence codec,
/// and splitting an answer into tappable `[n]` citation spans. Kept out of the
/// controller/UI so they're unit-testable in isolation.
library;

import 'dart:convert';

import 'package:grabbit/core/db/database.dart';
import 'package:grabbit/features/ai/data/rag_context.dart';

const String kRoleUser = 'user';
const String kRoleAssistant = 'assistant';

/// A decoded citation persisted on an assistant message — enough to render and
/// deep-link the inline `[n]` markers without re-running retrieval.
class Citation {
const Citation({
required this.index,
required this.itemId,
required this.title,
});

final int index;
final String itemId;
final String title;
}

/// One piece of a rendered answer: literal [text], plus a non-null [citation]
/// when this piece is a tappable `[n]` marker.
class CitationSpan {
const CitationSpan.text(this.text) : citation = null;
const CitationSpan.cite(this.text, Citation this.citation);

final String text;
final Citation? citation;

bool get isCitation => citation != null;
}

/// Pairs consecutive user→assistant messages into RAG history turns. An
/// assistant message with no preceding question is ignored; a trailing,
/// not-yet-answered user message (e.g. the in-flight turn) is dropped.
List<RagChatTurn> messagesToHistory(List<ChatMessage> msgs) {
final turns = <RagChatTurn>[];
String? pendingQuestion;
for (final m in msgs) {
if (m.role == kRoleUser) {
pendingQuestion = m.content;
} else if (m.role == kRoleAssistant && pendingQuestion != null) {
turns.add(RagChatTurn(question: pendingQuestion, answer: m.content));
pendingQuestion = null;
}
}
return turns;
}

/// Serializes the retrieved [sources] to the compact JSON stored on an assistant
/// message's `citationsJson` (only `index`/`itemId`/`title` — the snippet the
/// model saw isn't needed to render citations).
String encodeCitations(List<RagSource> sources) => jsonEncode([
for (final s in sources) {'i': s.index, 'id': s.itemId, 'title': s.title},
]);

/// Inverse of [encodeCitations]; tolerant of null/blank/malformed input
/// (returns an empty list rather than throwing at a UI boundary).
List<Citation> decodeCitations(String? json) {
if (json == null || json.trim().isEmpty) return const [];
try {
final decoded = jsonDecode(json);
if (decoded is! List) return const [];
return [
for (final e in decoded)
if (e is Map &&
e['i'] is int &&
e['id'] is String &&
e['title'] is String)
Citation(
index: e['i'] as int,
itemId: e['id'] as String,
title: e['title'] as String,
),
];
} on FormatException {
return const [];
}
}

final _citationMarker = RegExp(r'\[(\d+)\]');

/// Splits [answer] into text + citation spans. A `[n]` whose number matches a
/// known citation becomes a tappable span; an out-of-range `[n]` stays plain
/// text so nothing is dropped.
List<CitationSpan> parseCitationSpans(String answer, List<Citation> citations) {
final byIndex = {for (final c in citations) c.index: c};
final spans = <CitationSpan>[];
var cursor = 0;
for (final match in _citationMarker.allMatches(answer)) {
final citation = byIndex[int.parse(match.group(1)!)];
if (citation == null) continue; // leave unknown [n] in the surrounding text
if (match.start > cursor) {
spans.add(CitationSpan.text(answer.substring(cursor, match.start)));
}
spans.add(CitationSpan.cite(match.group(0)!, citation));
cursor = match.end;
}
if (cursor < answer.length) {
spans.add(CitationSpan.text(answer.substring(cursor)));
}
return spans;
}
Loading
Loading