Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@
_(nothing active — pick the next batch from below)_

## Deferred / future refinements
- [ ] **Rediscover — betweenness centrality.** P13e-2 ranks by **PageRank** (cheap, deterministic). Betweenness
(bridge-ness) is O(V·E) — too costly for a query-time strip; explore it as an alternative/secondary signal,
likely alongside e-3's path/bridge work. *(From P13e-2.)*
- [ ] **Shared entity item-graph builder.** `community_clustering.dart` (e-1, unweighted adjacency + label
propagation) and `centrality.dart` (e-2, weighted adjacency + PageRank) both rebuild an item graph from the
same membership + co-download pulls. Extract a shared builder once a third consumer appears. *(From P13e-2.)*
- [ ] **Rediscover — tuning + "See all".** `freshWindow`/`stalenessCapDays`/`limit` are fixed defaults and the
scores recompute query-time (like Suggested/Discovered); consider per-tier tuning, a full "See all"
Rediscover screen, and caching. *(From P13e-2.)*
- [ ] **Community albums — semantic-similarity + device-tier enhancements.** P13e-1 detects communities over
the **entity** graph only (every-device). Fold **semantic-similarity edges** (looser-threshold vector kNN)
into the same label-propagation graph on capable tiers for richer thematic grouping, gated like the
Expand Down
2 changes: 1 addition & 1 deletion docs/GRAPH-SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ gracefully when `GraphStore.isAvailable` is false.
| **Tag suggestions** (P10c-c-2, **live**) | `GraphQueryService.coOccurringTags`: tags on items sharing a deterministic signal with this one (`postedBy`/`inPlaylist`/`taggedWith`/`coDownloadedWith`), minus the item's own tags, ranked in Dart (`cooccurrence_ranking.dart`) by distinct supporting items. Surfaced as tappable chips in the metadata editor. Pure Datalog — every device. |
| **Interactive graph viz** (P10c-e/f) | **e (render, live):** `GraphQueryService.neighborhood(id)` (an item's direct entity + duplicate/co-download edges) → a force-directed `graphview` render with pan/zoom + a type legend, reached via item-detail "View in graph". Deterministic edges — no embedder. **f (live):** tap a media node → its item, tap an entity node → expand its media (and long-press → open hub), edge-type **legend filters**, expansion capped (`:limit`). |
| **Graph-clustered auto-albums** (P13) | **community detection / label propagation** over the similarity + entity graph. **Realized P13e-1** over the **entity** graph (shared uploader/playlist/tag + co-download) via pure deterministic label propagation (`community_clustering.dart`) — every-device, no embedder; surfaced as the **"Discovered"** album section. Folding in semantic-similarity edges is deferred (BACKLOG). |
| **Centrality "Rediscover"** (P13) | `PageRank` / betweenness × `lastAccessedAt` to resurface central-but-stale items. |
| **Centrality "Rediscover"** (P13) | `PageRank` / betweenness × `lastAccessedAt` to resurface central-but-stale items. **Realized P13e-2** via pure-Dart **PageRank** over the entity item-graph (`centrality.dart`) × staleness (`rankRediscover`) — every-device, no embedder; surfaced as the **"Rediscover"** Dashboard/Library strip. Betweenness deferred (BACKLOG / e-3). |
| **Path / bridge discovery** (P13) | shortest-path / connectivity between two items or entities. |
| **Local GraphRAG "Ask your library"** (P13) | hybrid retrieval (vector + graph re-rank) feeds the on-device LLM (see `AI-SPEC.md`). |

Expand Down
10 changes: 10 additions & 0 deletions docs/VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1033,6 +1033,16 @@ entries, or verify after P11c lands.)*
- [ ] Tapping a discovered album opens its items; **Save** creates a normal collection containing them.
- [ ] The Discovered section is **absent** when the graph index is unavailable (e.g. unsupported ABI).

### P13e-2 — Centrality "Rediscover" strip *(install `app-arm64-v8a-debug.apk`; needs an established library)*
- [ ] On a library with cross-links (shared channels/playlists/tags or co-downloaded batches), a **"Rediscover"**
strip appears on the **Dashboard** (below "Recently opened") **and** atop the **Library**, surfacing items
that are well-connected but that you **haven't opened in a while**.
- [ ] Items opened in the **last ~2 weeks do not** appear (no overlap with "Recently opened"); tapping a tile
opens the item.
- [ ] The strip is **absent** on a small/new library, while searching/filtering/selecting in the Library, and
when the graph index is unavailable.
- [ ] Works on a **low-end device** (entity graph only — no embedder/LLM).

### P13 (later subphases)
- [ ] **Transcription / summarization / translation / OCR** each work (capability-gated) and write
results back to the item.
Expand Down
21 changes: 16 additions & 5 deletions docs/design/P13-PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -311,11 +311,22 @@ The richer graph payoff beyond P10's Duplicates + Suggested-similarity albums (G
- **Exit / review:** clusters are coherent on a real library; degrades to nothing when the graph is
unavailable; saving a cluster creates a normal collection. ✓ (CI parts) · APK owed

#### `[ ]` P13e-2 — Centrality "Rediscover" *(graph; APK)*
- **PageRank / betweenness × `lastAccessedAt`** to resurface central-but-stale items; surfaced as a
"Rediscover" strip (Dashboard/Library).
- **Exit / review:** Rediscover surfaces genuinely central, not-recently-opened items; empty/ graceful when
the graph is unavailable.
#### `[~]` P13e-2 — Centrality "Rediscover" *(graph; APK)*
- **PageRank × staleness** to resurface central-but-stale items; surfaced as a "Rediscover" strip on the
**Dashboard and Library**.
- **Status:** implemented (CI-green; APK spot-check owed). **PageRank** chosen (deterministic power iteration;
betweenness is O(V·E) — too costly for a strip, deferred to BACKLOG/e-3). Runs over the **entity item-graph**
(shared uploader/playlist/tag + co-download), reusing e-1's `entityMembershipScript()`/`coDownloadPairsScript()`
pulls (decode shared via a private `_entityGraph()` helper) — **every-device, no embedder**. New
`lib/core/graph/centrality.dart` (pure `buildItemGraph` weighted adjacency + `pageRank`);
`GraphQueryService.itemCentrality()`; pure `rediscover.dart` `rankRediscover` (`score = centrality ×
staleness`; staleness = days since `lastAccessedAt ?? createdAt`, capped at 30d; **excludes** items touched
within 14d); `rediscoverProvider` + a `RediscoverRow` strip (mirrors `RecentMediaRow`, auto-hides when empty)
on the Dashboard and atop the Library (only when not searching/filtering/selecting). **No schema, no deps.**
Tests: centrality (weight accumulation, bucket pruning, hub > leaf, determinism, dangling), `itemCentrality`
decode, `rankRediscover` (fresh-exclusion, staleness weighting, cap, empty), the provider, and the row.
- **Exit / review:** Rediscover surfaces genuinely central, not-recently-opened items; empty/graceful when
the graph is unavailable. ✓ (CI parts) · APK owed

#### `[ ]` P13e-3 — Path/bridge discovery + graph-view polish *(graph; APK)*
- **Shortest-path / connectivity** between two items or entities ("how are these related?"), plus graph-view
Expand Down
90 changes: 90 additions & 0 deletions lib/core/graph/centrality.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/// Pure centrality scoring for the "Rediscover" strip (P13e-2). No Flutter,
/// engine, or AI imports — `GraphQueryService` feeds it the decoded entity
/// memberships + co-download pairs and it returns per-item PageRank, so the
/// whole thing is unit-testable. Sits beside `community_clustering.dart` (the
/// same entity item-graph, scored for importance rather than grouped).
library;

/// A **weighted, undirected** item↔item adjacency built from the entity graph:
/// items sharing an entity bucket ([memberships]; `group` is a type-prefixed key
/// like `u:<id>` / `p:<id>` / `t:<tag>`) gain +1 edge weight per shared bucket,
/// and each direct [pairs] edge (co-download) adds +1 more. Buckets larger than
/// [maxGroupSize] are dropped as **too generic** (the "discard blobs" rule from
/// the clusterer). Returns `id → {neighbor → weight}` (both directions).
Map<String, Map<String, num>> buildItemGraph({
required List<({String item, String group})> memberships,
required List<({String a, String b})> pairs,
int maxGroupSize = 50,
}) {
final byGroup = <String, List<String>>{};
for (final m in memberships) {
(byGroup[m.group] ??= <String>[]).add(m.item);
}

final adj = <String, Map<String, num>>{};
void link(String a, String b) {
if (a == b) return;
final ma = adj[a] ??= <String, num>{};
ma[b] = (ma[b] ?? 0) + 1;
final mb = adj[b] ??= <String, num>{};
mb[a] = (mb[a] ?? 0) + 1;
}

for (final members in byGroup.values) {
if (members.length < 2 || members.length > maxGroupSize) continue;
for (var i = 0; i < members.length; i++) {
for (var j = i + 1; j < members.length; j++) {
link(members[i], members[j]);
}
}
}
for (final p in pairs) {
link(p.a, p.b);
}
return adj;
}

/// Weighted **PageRank** over [adjacency] (from [buildItemGraph]) — each item's
/// score is its standing in the library's web, so hub items (shared with many
/// others through many signals) rank highest. Deterministic power iteration with
/// dangling-mass redistribution; returns `id → score` (scores sum to ~1).
Map<String, double> pageRank(
Map<String, Map<String, num>> adjacency, {
int iterations = 40,
double damping = 0.85,
}) {
final nodes = adjacency.keys.toList();
final n = nodes.length;
if (n == 0) return const {};

final outWeight = {
for (final node in nodes)
node: adjacency[node]!.values.fold<double>(0, (s, w) => s + w),
};
final base = (1 - damping) / n;
var rank = {for (final node in nodes) node: 1.0 / n};

for (var iter = 0; iter < iterations; iter++) {
final next = {for (final node in nodes) node: base};
var danglingMass = 0.0;
for (final node in nodes) {
final ow = outWeight[node]!;
if (ow == 0) {
danglingMass += rank[node]!;
continue;
}
final share = damping * rank[node]! / ow;
adjacency[node]!.forEach((nb, w) {
next[nb] = next[nb]! + share * w;
});
}
if (danglingMass > 0) {
final spread = damping * danglingMass / n;
for (final node in nodes) {
next[node] = next[node]! + spread;
}
}
rank = next;
}
return rank;
}
49 changes: 41 additions & 8 deletions lib/core/graph/graph_query_service.dart
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import 'package:grabbit/core/graph/centrality.dart';
import 'package:grabbit/core/graph/community_clustering.dart';
import 'package:grabbit/core/graph/cooccurrence_ranking.dart';
import 'package:grabbit/core/graph/cozo_query.dart';
Expand Down Expand Up @@ -173,6 +174,45 @@ class GraphQueryService {
int maxGroupSize = 50,
}) async {
if (!_store.isAvailable) return const [];
final (:memberships, :pairs) = await _entityGraph();
if (memberships.isEmpty) return const [];
return detectCommunities(
memberships: memberships,
pairs: pairs,
minSize: minSize,
maxSize: maxSize,
maxGroupSize: maxGroupSize,
);
}

/// PageRank **centrality** of every connected item over the entity graph
/// (P13e-2) — `id → score`, ranking items by how woven they are into the
/// library's web (shared uploader/playlist/tag + co-download). Every-device
/// (pure Datalog + Dart; no embedder). `{}` when the store is unavailable.
/// Feeds the "Rediscover" strip via `rankRediscover`.
Future<Map<String, double>> itemCentrality({int maxGroupSize = 50}) async {
if (!_store.isAvailable) return const {};
final (:memberships, :pairs) = await _entityGraph();
if (memberships.isEmpty && pairs.isEmpty) return const {};
return pageRank(
buildItemGraph(
memberships: memberships,
pairs: pairs,
maxGroupSize: maxGroupSize,
),
);
}

/// Decodes the entity-membership (`item`, type-prefixed `group`) + co-download
/// (`a`/`b`) pulls shared by the community (P13e-1) and centrality (P13e-2)
/// builders over the deterministic entity graph.
Future<
({
List<({String item, String group})> memberships,
List<({String a, String b})> pairs,
})
>
_entityGraph() async {
final memberships = [
for (final r in decodeRows(
await _store.runScript(entityMembershipScript()),
Expand All @@ -182,21 +222,14 @@ class GraphQueryService {
if (r['key'] case final Object key)
(item: id.toString(), group: '$kind:$key'),
];
if (memberships.isEmpty) return const [];
final pairs = [
for (final r in decodeRows(
await _store.runScript(coDownloadPairsScript()),
))
if (r['a'] case final Object a)
if (r['b'] case final Object b) (a: a.toString(), b: b.toString()),
];
return detectCommunities(
memberships: memberships,
pairs: pairs,
minSize: minSize,
maxSize: maxSize,
maxGroupSize: maxGroupSize,
);
return (memberships: memberships, pairs: pairs);
}

/// The immediate graph neighborhood of media item [id] — its connected
Expand Down
2 changes: 2 additions & 0 deletions lib/features/dashboard/presentation/dashboard_screen.dart
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import 'package:grabbit/features/dashboard/presentation/widgets/duplicates_callo
import 'package:grabbit/features/dashboard/presentation/widgets/graph_entry_tile.dart';
import 'package:grabbit/features/dashboard/presentation/widgets/recent_activity_tile.dart';
import 'package:grabbit/features/dashboard/presentation/widgets/recent_media_row.dart';
import 'package:grabbit/features/dashboard/presentation/widgets/rediscover_row.dart';
import 'package:grabbit/features/dashboard/presentation/widgets/stat_card.dart';
import 'package:grabbit/features/dashboard/presentation/widgets/storage_donut_tile.dart';
import 'package:grabbit/features/dashboard/presentation/widgets/suggestions_tile.dart';
Expand Down Expand Up @@ -171,6 +172,7 @@ class _DashboardBody extends ConsumerWidget {
title: 'Recently opened',
provider: recentlyPlayedProvider,
),
const RediscoverRow(),
const RecentActivityTile(),
const SuggestionsTile(),
const DuplicatesCallout(),
Expand Down
43 changes: 43 additions & 0 deletions lib/features/dashboard/presentation/widgets/rediscover_row.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
import 'package:flutter/material.dart';
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:grabbit/core/theme/tokens.dart';
import 'package:grabbit/core/widgets/section_header.dart';
import 'package:grabbit/features/library/presentation/media_grid.dart';
import 'package:grabbit/features/library/presentation/rediscover_provider.dart';

/// A horizontal strip of **central-but-stale** media — items woven into the
/// library graph that haven't been opened lately (P13e-2). Mirrors
/// `RecentMediaRow`, but fed by the graph-derived `rediscoverProvider`; auto-
/// hides when empty (small/new library, or the graph is unavailable).
class RediscoverRow extends ConsumerWidget {
const RediscoverRow({this.cap = 12, super.key});

final int cap;

@override
Widget build(BuildContext context, WidgetRef ref) {
final tokens = GrabBitTokens.of(context);
final items = ref.watch(rediscoverProvider).asData?.value ?? const [];
if (items.isEmpty) return const SizedBox.shrink();

final shown = items.take(cap).toList();
return Column(
crossAxisAlignment: CrossAxisAlignment.stretch,
children: [
const SectionHeader('Rediscover', icon: Icons.auto_awesome_motion),
SizedBox(
height: 172,
child: ListView.separated(
scrollDirection: Axis.horizontal,
clipBehavior: Clip.none,
padding: EdgeInsets.symmetric(horizontal: tokens.spaceLg),
itemCount: shown.length,
separatorBuilder: (_, _) => SizedBox(width: tokens.spaceMd),
itemBuilder: (_, i) =>
SizedBox(width: 124, child: MediaTile(item: shown[i])),
),
),
],
);
}
}
4 changes: 4 additions & 0 deletions lib/features/library/presentation/library_view.dart
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import 'package:grabbit/core/widgets/empty_state.dart';
import 'package:grabbit/core/widgets/error_view.dart';
import 'package:grabbit/core/widgets/skeleton.dart';
import 'package:grabbit/core/db/database.dart';
import 'package:grabbit/features/dashboard/presentation/widgets/rediscover_row.dart';
import 'package:grabbit/features/library/data/metadata_repository.dart';
import 'package:grabbit/features/library/presentation/library_controller.dart';
import 'package:grabbit/features/library/presentation/library_filter_sheet.dart';
Expand Down Expand Up @@ -123,6 +124,9 @@ class _LibraryViewState extends ConsumerState<LibraryView> {
},
onFilters: () => showLibraryFilters(context),
),
// Resurface central-but-stale items, but only while browsing the full
// library (not mid-search/filter or selection) so it never intrudes.
if (!filtering && _selected.isEmpty) const RediscoverRow(),
Expanded(
child: RefreshIndicator(
onRefresh: () async => refresh(),
Expand Down
40 changes: 40 additions & 0 deletions lib/features/library/presentation/rediscover.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
/// Pure "Rediscover" ranking (P13e-2): combine graph **centrality** with
/// **staleness** to resurface central-but-faded items. No Flutter/Drift imports
/// so the scoring is unit-testable; the provider supplies centrality (from the
/// graph) and per-item last-touch times (from Drift).
library;

/// Ranks items for the Rediscover strip: `score = centrality × staleness`, where
/// staleness grows with days since the item was last touched ([lastTouchById] =
/// `lastAccessedAt ?? createdAt`). Items touched within [freshWindow] are
/// **excluded** (they already live in "Recently opened"); staleness saturates at
/// [stalenessCapDays]. Returns the top [limit] ids, most-relevant first (ties →
/// more-stale, then id for stability). Items without a centrality score or a
/// known touch time are skipped.
List<String> rankRediscover({
required Map<String, double> centrality,
required Map<String, DateTime> lastTouchById,
required DateTime now,
Duration freshWindow = const Duration(days: 14),
double stalenessCapDays = 30,
int limit = 12,
}) {
final scored = <({String id, double score, double days})>[];
centrality.forEach((id, rank) {
if (rank <= 0) return;
final touched = lastTouchById[id];
if (touched == null) return;
final days = now.difference(touched).inSeconds / Duration.secondsPerDay;
if (days < freshWindow.inSeconds / Duration.secondsPerDay) return;
final staleness = (days / stalenessCapDays).clamp(0.0, 1.0);
scored.add((id: id, score: rank * staleness, days: days));
});

scored.sort((a, b) {
final byScore = b.score.compareTo(a.score);
if (byScore != 0) return byScore;
final byDays = b.days.compareTo(a.days);
return byDays != 0 ? byDays : a.id.compareTo(b.id);
});
return [for (final s in scored.take(limit)) s.id];
}
37 changes: 37 additions & 0 deletions lib/features/library/presentation/rediscover_provider.dart
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import 'package:flutter_riverpod/flutter_riverpod.dart';
import 'package:grabbit/core/db/database.dart';
import 'package:grabbit/core/db/database_provider.dart';
import 'package:grabbit/core/graph/graph_query_provider.dart';
import 'package:grabbit/features/library/presentation/rediscover.dart';

// Hand-written (returns Drift `MediaItem` rows): central-but-stale items for the
// "Rediscover" strip (P13e-2).

/// Items to **resurface** — central in the library graph (PageRank) yet not
/// opened recently — for the Dashboard/Library "Rediscover" strip. Every-device
/// (graph only, no embedder). `[]` when the graph is unavailable or nothing
/// qualifies, so the strip simply hides.
final rediscoverProvider = FutureProvider<List<MediaItem>>((ref) async {
final centrality = await ref
.watch(graphQueryServiceProvider)
.itemCentrality();
if (centrality.isEmpty) return const [];

final db = ref.watch(appDatabaseProvider);
final found = await (db.select(
db.mediaItems,
)..where((t) => t.id.isIn(centrality.keys.toList()))).get();
final byId = {for (final m in found) m.id: m};

final ranked = rankRediscover(
centrality: centrality,
lastTouchById: {
for (final m in found) m.id: m.lastAccessedAt ?? m.createdAt,
},
now: DateTime.now(),
);
return [
for (final id in ranked)
if (byId[id] case final MediaItem m) m,
];
});
Loading
Loading