Skip to content

feat(search): offline search index for courses (#393)#431

Open
mmorgsmorgan wants to merge 1 commit into
rinafcode:mainfrom
mmorgsmorgan:feat/offline-search-index-393
Open

feat(search): offline search index for courses (#393)#431
mmorgsmorgan wants to merge 1 commit into
rinafcode:mainfrom
mmorgsmorgan:feat/offline-search-index-393

Conversation

@mmorgsmorgan
Copy link
Copy Markdown

Closes #393

Summary

Replaced the inline .includes() filtering in MobileSearch with a local offline-first search index for faster and more scalable search.

What changed

  • Added a pure JavaScript inverted index in src/services/searchIndex/

  • No new dependencies added

  • Added tokenization with:

    • diacritic stripping
    • stopword removal
    • 2-character minimum token length
  • Implemented:

    • AND-based token intersection
    • prefix matching on the final token for typeahead search
  • Added weighted ranking:

    • title: 5
    • category: 3
    • level: 2
    • body / instructor: 1
  • Added prefix match bonus scoring

Persistence

  • Search snapshots persist to AsyncStorage under:
    @teachlink_search_index_v1

  • Snapshot includes:

    • version
    • content hash
  • Added a 200KB size cap

    • oversized snapshots skip persistence
    • index rebuilds automatically on next launch

Search lifecycle

  • Added SearchIndexProvider in app/_layout.tsx

  • Provider flow:

    1. Hydrates from AsyncStorage on startup

    2. Rebuilds asynchronously from:

      • courseApi.getCourses()
      • fallback: offlineStorage.getAllCourses()
      • fallback: [sampleCourse]
  • After one successful online build, search works fully offline

Public API

useSearchIndex() exposes:

  • ready
  • size
  • search
  • rebuild

Incremental updates supported through:

  • searchIndex.update(doc)
  • searchIndex.remove(id)

User indexing

User search is intentionally out of scope for this PR.

The codebase currently includes getUser(id) but no list source. The IndexableDoc API already supports a 'user' type, so future support only requires:

  • one adapter
  • one indexing call

Acceptance Criteria

  • Build search index on app startup via SearchIndexProvider

  • Implement efficient search algorithm with:

    • inverted index
    • AND intersection
    • prefix lookup
    • weighted ranking
  • Instant search results (<100ms)

    • perf tests run against 500 synthetic docs
    • local average: ~1ms/query
  • Offline search support through AsyncStorage snapshot persistence

  • Index size management with 200KB payload cap

  • Incremental index updates through:

    • update(doc)
    • remove(id)
    • rebuild()
  • Added JSDoc documentation for public exports

Files

New

  • src/services/searchIndex/{types,tokenize,SearchIndex,persistence,courseAdapter,index}.ts
  • src/components/SearchIndexProvider.tsx
  • src/hooks/useSearchIndex.ts
  • src/__tests__/services/searchIndex/{tokenize,SearchIndex,persistence}.test.ts

Modified

  • src/components/mobile/MobileSearch.tsx

    • replaced filterCourse(sampleCourse, ...) with useSearchIndex().search(...)
  • app/_layout.tsx

    • added <SearchIndexProvider>
  • src/components/index.ts

  • src/hooks/index.ts

Test Plan

  • npm test -- searchIndex

    • 33/33 tests passing
    • includes <100ms performance budget validation
  • Full test suite completed

    • only existing failures remain:

      • DebounceIntegration
      • secureStorage
    • no regressions introduced

  • eslint --max-warnings=0

    • clean on all modified files

Manual Verification

  • Verify partial keyword search returns instant local results
  • Verify filter sheet still narrows results correctly
  • Launch app in airplane mode after one online build and confirm offline search works from snapshot
  • Modify sampleCourse, restart app, and confirm rebuild occurs instead of serving stale data

Adds a pure-JS inverted index that builds at app start, persists to
AsyncStorage, and replaces the inline filter in MobileSearch. Search
intersects per-token postings (AND), supports prefix matching on the
trailing token, and ranks by field-weighted scoring (title > category
> level > body). Falls back through courseApi -> offlineStorage ->
sampleCourse so the screen works offline once the snapshot has been
built. Snapshots persist under a versioned key with a 200KB cap.
@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented May 27, 2026

@mmorgsmorgan Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@RUKAYAT-CODER
Copy link
Copy Markdown
Contributor

kindly resolve conflict and fix workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement efficient search index for offline search capability

2 participants