feat: Android taxonomy sync + hide built-in default chips

[djbclark]

Thank you for building and maintaining SuperBrain — it's a genuinely great project, and it's a pleasure to get to contribute back to it. No rush at all on review; happy to adjust anything.

What this adds

Defensive, gated support in the Android app for a config-driven category taxonomy, so a fork/deployment that adds GET /taxonomy on the backend can offer custom categories and always-fresh sync — without any backend change required to merge this, and with zero behavior change for everyone else.

Why

Categorization needs vary a lot between deployments (different content domains want different category sets), but the mainline app's categories are currently hardcoded. Rather than carry a growing fork-only diff against main, this PR proposes landing the client-side plumbing now — entirely inert until a backend opts in — so the taxonomy work stays upstream-compatible from day one instead of drifting further apart over time.

Why it's safe to merge before any backend taxonomy work exists

Every new code path is gated behind isTaxonomyApiActive(taxonomy). Concretely:

• Without GET /taxonomy (every deployment today): identical mainline behavior — built-in category chips, early-return after the local SQLite read, delta-only sync (including pull-to-refresh). Nothing observably changes.
• With GET /taxonomy (opt-in, fork/future upstream): always background-sync after the local paint, pull-to-refresh can trigger a full resync, taxonomy_version changes trigger an automatic full resync, and use_default_categories: false switches to server-configured chips only.
• If /taxonomy 404s, times out, or returns malformed data at any point, the client falls back to exact current mainline behavior rather than erroring.

Scope

App (client, the original scope of this PR):

• superbrain-app/src/constants/categories.ts — default-category fallback plumbing
• superbrain-app/src/screens/HomeScreen.tsx — gated taxonomy fetch/sync integration
• superbrain-app/src/screens/PostDetailScreen.tsx — configured-category display
• superbrain-app/src/services/api.ts — getTaxonomy(), paginated syncPosts()
• superbrain-app/src/services/localDb.ts — taxonomy_version persistence
• superbrain-app/src/services/syncService.ts — bounded, safe delta-sync pagination
• superbrain-app/src/services/taxonomySupport.ts — new, isTaxonomyApiActive gate + types
• superbrain-app/src/theme/index.ts — category color fallback

Backend — 2 files, 25 lines, fixing a pre-existing bug this PR is not responsible for (see below):

• backend/api.py — /sync now accepts and honors the offset param
• backend/core/database.py — get_posts_since() now supports real offset-based pagination with a has_more signal

⚠️ Pre-existing backend bug found and fixed along the way

This is not a taxonomy-feature change, and I want to be upfront that it goes beyond "Android app only": while hardening this PR's own delta-sync handling, I found that /sync never actually implemented the offset query param the Android app has already been sending on every sync, before this PR touched anything. The backend's get_posts_since() just ignored it — no OFFSET in the SQL, no has_more in the response — so offset was silently a no-op.

In practice this means: any single delta sync with more than 200 changed posts (the page size the client already uses) has always been unable to fetch past the first page, on the current main branch, independent of this PR. A large batch of changes (e.g. a big playlist import) landing between syncs would get truncated at 200 with no error and no way to catch up. My new client-side pagination-safety net (bounded loop + stall detection, see below) made this concrete: it correctly stops instead of looping forever, but a client-only fix can't retrieve data the server never sends — only a backend fix can.

The fix is small and purely additive: offset defaults to 0 and has_more is a new optional field, so nothing changes for any existing caller that doesn't use them. I mention this prominently because I know the intent was to keep this Android-only, and I want the "why" to be clear rather than have it read as scope creep.

Client-side hardening (the other 3 of 4 commits)

The original feature commit went through two rounds of independent automated review (CodeRabbit, then a second differently-sourced review as a deliberate second opinion) before I felt comfortable asking for your time on it. Both rounds found real, worthwhile issues, now fixed:

1. Pagination safety net — the delta-sync loop is now bounded (50 pages) and detects a non-advancing cursor, so it can never spin forever even against an unexpected backend. (This is what surfaced the pre-existing /sync bug above.)
2. Cursor-advance correctness — the sync cursor (lastSyncTime) only advances when a sync cycle genuinely completes; an early stop (cap hit, non-advancing cursor, or a failed page fetch) leaves it untouched so the next sync safely retries the same window instead of silently skipping data.
3. Removed a benign but real cache race — a taxonomy-fetch cache was accidentally shared across overlapping calls (e.g. screen-focus refresh vs. the poll interval); it's now correctly scoped per call.

Every fix above was independently re-verified (not just re-read) before this revision, and the app was rebuilt from a clean debug+release APK and confirmed working end-to-end. Happy to walk through any of this in more detail, or to split the backend fix into its own separate PR if you'd rather review/merge it independently of the taxonomy feature — just say the word.

Test plan

• Against upstream backend (no /taxonomy): chips and sync match current production APK exactly
• Against a taxonomy-enabled backend: custom chips appear when use_default_categories: false; a taxonomy_version change triggers a full resync
• Offline / failed connection: no extra taxonomy calls happen before the connectivity check
• Typechecks clean (tsc --noEmit) on the current commit
• New pagination/offset logic sanity-checked against an in-memory SQLite fixture (paged fetch of N rows returns all rows exactly once, in order, has_more flips correctly on the last page)

Thanks again for taking a look — genuinely appreciate the project and your time.

[djbclark]

Hey so sorry for the massive history here. This is my attempt to avoid asking for android client changes more than once. Happy to answer questions.

[djbclark]

@coderabbitai review

[djbclark]

Fixed in d629ccf per CodeRabbit's review on the sibling review-only PR: deltaSync no longer advances lastSyncTime when pagination stops early (page cap or non-advancing cursor) — previously that silently skipped unfetched changes on every subsequent sync. Cursor now only advances on genuine completion (empty page or hasMore === false).

[djbclark]

Ran an independent second-opinion review (different model) on top of the CodeRabbit round. It found a deeper issue than the pagination-cap fix addressed: /sync never actually implemented the offset param the client was already sending — get_posts_since() always queried from the same since with no OFFSET, and the endpoint never returned has_more. The client's safety nets correctly stopped the loop instead of spinning forever, but since the cursor never advanced either, every retry hit the identical wall — a guaranteed permanent sync stall for any backlog over 200 changed posts.

Fixed in 3acb694, pushed to this branch:

• Backend /sync + get_posts_since() now honor offset and return real has_more, with shortcode as a secondary sort key so pagination is stable when timestamps tie.
• Client syncPosts() now distinguishes a transient fetch failure from a legitimate empty/final page, so a network blip can't be mistaken for sync completion and silently skip unfetched changes.
• HomeScreen's taxonomy cache was a component-level ref that only ever lived within a single loadPosts call — turned into a local variable so overlapping calls (focus listener vs. poll-interval refresh) can't stomp each other's cache.

Verified the new pagination logic against an in-memory sqlite db (paged fetch returns all rows once, in order, has_more flips false correctly) and typechecked clean.

[djbclark]

Independent second-opinion re-verification (same model that caught the /sync offset bug) confirms all four issues are fixed in 3acb694, with no new issues introduced:

1. /sync offset/has_more — verified correct across page boundaries, including the exactly-limit-sized final page (off-by-one checked by hand).
2. Fetch-failure vs. end-of-data — verified the failed flag is checked first in the loop, before any cursor-advancing logic.
3. shortcode secondary sort key — confirmed unique (TEXT PRIMARY KEY) and cheap.
4. taxonomyRef race — confirmed the cache is now call-scoped (let inside loadPosts), no cross-call stomping possible.

This PR is in a good state from my side — ready whenever you want to take a look.