Skip to content

Requirements [draft]

Jump to bottom
Danny Collier edited this page Jun 7, 2026 · 2 revisions

Beacon Requirements

(A working draft for review. Companion to the SkillRX wiki; this document assumes the SkillRX domain model and CMS behavior described there.)

Overview

#overview The Beacon is the next evolution of the device application that delivers continuing medical education to practitioners in low-resource locations. It runs on a minicomputer, replaces the current CMES-mini app, and reproduces that app's functionality while opening the door to new capabilities. Like its predecessors, it has two operating modes:

  • Syncing mode. The device connects to the internet and pulls the subset of content it is configured to hold, then updates its local database and files to match.
  • Serving mode. The device disconnects, broadcasts a local Wi-Fi network, and serves content to local users — typically on their phones — with no internet required. The defining architectural change is how content reaches the device. Today SkillRX publishes content to Azure file storage, and devices pull from Azure (minis via CSV + media, Pis via XML/TXT + media). The Beacon instead syncs directly from SkillRX over an API, using a versioned manifest. This removes Azure from the Beacon's path and removes the indirection that has complicated the existing workflow. Azure is not going away. SkillRX will continue publishing to Azure for the foreseeable future to serve the existing CMES-mini and CMES-Pi devices, which will be replaced slowly. Retiring Azure is out of scope for this project; direct sync and Azure delivery run in parallel. This document covers the Beacon application and the SkillRX changes needed to support it.

1. Core Decisions

#core-decisions

Build around functionality and SkillRX, not the legacy schemas

#build-around-functionality The Beacon's data model should be driven by two things only: the user-facing functionality below, and the realities of the SkillRX domain model and sync manifest. It does not need to preserve the legacy XML or CSV schemas. The current beacon tables were shaped to ingest the legacy feed we are now replacing, so a few of them reflect that feed rather than the functionality or SkillRX.

This is not a parity exercise against the SkillRX schema. The Beacon needs only the tables that hold the synced content that drives the local app, plus a small amount of state to drive its own sync. SkillRX keeps its additional tables for features specific to it (device registration, per-Beacon configuration, import reporting, taggings/cognates, etc.); none of those belong on the device. The column-level mapping is in Schema & Sync Mapping below.

Changes from the as-built beacon tables:

  • Topic should carry SkillRX's real published_at rather than year/month/volume/issue as strings (the CSV shape). Volume and issue are defunct in SkillRX and not needed here. (Per-topic language and state are not needed on the device — language is a device-level configuration concern, and the manifest already pre-filters to active topics; see Schema & Sync Mapping.)
  • Files should support everything SkillRX supports — see §4.4. The CSV's 1=PDF, 2=MP3 encoding is a legacy limit, not a requirement. Files carry SkillRX's content_type and key on blob.id + checksum.
  • Authors are defunct in SkillRX v3; the beacon's Author model is a carryover and should be retired unless a user requirement reintroduces it. (Confirm.)
  • Language and Region are device-configuration and sync concerns, not user-facing. A Beacon holds one language's content for an optionally-configured region, and end users never select or see either; they simply get the topics and files that result (§2). The Beacon only needs whatever minimal representation its configuration and sync require — the manifest already carries both — not SkillRX's full multi-language/multi-region modeling.

Identifiers — lean entirely on SkillRX IDs

#identifiers In the Beacon world we are free of the two constraints that made identity painful in SkillRX:

  1. No Azure, so no identity in filenames. SkillRX had to embed topic IDs in media filenames ([topic_id]_[filename]) because content traveled through Azure as files, and the devices keyed off those names. The Beacon receives files directly with a manifest that describes them, so we can name files whatever we want. Filenames become a display/storage detail, not a key.
  2. No attachment-before-save problem. SkillRX had to contend with the Rails workflow uploading an attachment before its Topic was saved — i.e. before the Topic had an ID to build a filename from. The manifest hands us server-assigned IDs and per-file blob.id + checksum that exist independent of upload ordering, so the Beacon never has to construct identity itself. Decision: the Beacon adopts SkillRX's IDs as its own. When we sync a provider, topic, tag, or file, we persist it under the SkillRX-assigned ID and reconcile with find_or_create_by on that ID. Change detection on files keys on blob.id + checksum. This greatly simplifies updates, uniqueness, and delete reconciliation: a record is the same record on the device as it is in SkillRX, by ID. Discipline required: we must only create records with SkillRX-supplied IDs in a synching context. No locally-created, non-synced data may ever mint an ID into a table that holds synced content. In practice the purely-synced tables (providers, topics, tags, files) are populated only by sync, and the local-only tables (users, favorites, activity logs, Local Folder) never share those tables — so no collision arises. This becomes a live concern only if locally-curated content is ever folded into a synced table (see N-9). To confirm: that Rails/Postgres handle this cleanly — explicit non-sequential primary keys, find_or_create_by(id:), and the fact that a synced table's sequence is never advanced by a local insert (because we never do local inserts there). We believe this is a non-issue but should verify the sequence behavior so a stray local insert can't ever collide.

Schema & Sync Mapping

#schema-sync-mapping This subsection makes the decisions above concrete at the column level. The driver is the sync manifest, not the SkillRX schema — the Beacon consumes the manifest produced by Beacons::ManifestBuilder, which is leaner than SkillRX's tables and is the actual contract for what the device must persist. Manifest shape:

manifest_version "v3"   manifest_checksum "sha256:…"   generated_at
language { id, code, name }     region { id, name }     tags [ { id, name } ]
providers [ { id, name,
  topics [ { id, name, tag_ids[],
    files [ { id(=blob.id), filename, path, checksum, size_bytes, content_type, updated_at } ] } ] } ]
total_size_bytes   total_files

How today's beacon tables sort out

Beacon table Category Disposition
content_providers Synced content Keep; key on SkillRX provider id
topics Synced content Keep; restructure columns
topic_files Synced content Keep; restructure columns
tags, topic_tags Synced content Keep; key on SkillRX tag id
authors, topic_authors Synced content Retire — no authors in SkillRX v3 / manifest
users, favorites, user_activity_logs Local-only Keep as-is (not synced)
admins, admin_activity_logs, local_files Local-only Keep as-is (not synced)
(none today) Device config Add minimal language/region config
(none today) Sync cursor Add device sync-state

SkillRX-only tables the Beacon does not need: beacons, beacon_topics, beacon_providers, branches, regions/languages (full modeling), contributors, import_reports, import_errors, sessions, taggings, tag_cognates, and providers.file_name_prefix/provider_type/old_id.

content_providers — adopt SkillRX provider.id as the primary key; keep name. Do not add file_name_prefix/provider_type/old_id (SkillRX-only; filenames are display-only here). Optionally rename to providers.

topics — adopt SkillRX topic.id as the primary key; drop topic_external_id (redundant once PK = SkillRX id); drop year, month, volume, issue (legacy CSV strings); keep title (manifest name) and content_provider_id; keep view_count as a local-only engagement metric (Top Topics, §4.7 — never synced). Add published_at (datetime) to drive the Month/Year browse hierarchy (§4.3) and Latest Content (§4.7) — this requires a manifest extension (see §3). description (text) optional, for the metadata sidebar / N-1. Do not add per-topic state (manifest sends only active topics; deletes handled by absence) or language_id (one language per device).

topic_files — this is where the §4.4 video/image gap lives. Adopt blob.id as the key (PK, or a unique blob_id column); keep filename as display-only; rename file_size → size_bytes (bigint); replace file_type (pdf/mp3 enum) with content_type (MIME string); add checksum and path. Replace the TopicFile validation inclusion: %w[pdf mp3] with the full SkillRX set: image/jpeg, image/png, image/svg+xml, image/webp, image/avif, image/gif, video/mp4, application/pdf, audio/mpeg. This is the schema half of closing the §4.4 parity gap (viewer work is separate).

tags / topic_tags — adopt SkillRX tag.id as the key; keep name; populate topic_tags from the manifest's topic.tag_ids. SkillRX taggings/tag_cognates stay server-side, flattened in the manifest.

authors / topic_authors — retire both tables, the Author/TopicAuthor models, and the topic↔author associations. The browse sidebar's Author field (§4.3) drops with them. (Confirm — O-5.)

Device configuration (new, minimal) — the Beacon has no language/region table today. Store the device's one required language and optional region as device configuration (a singleton settings row, or columns on the sync-state record below) — not SkillRX's full multi-language/multi-region tables. Just enough to record what this device is configured for.

Device sync-state (new) — to drive the ETag conditional sync (§3), the device must persist its own copy of the sync cursor (SkillRX's beacons table is the server's view of each device; the device needs a local one). Add a singleton record holding: manifest_version, manifest_checksum (sent as If-None-Match / If-Match), last_sync_at, last_sync_status, last_sync_error, files_count, total_size_bytes, API-key storage, and device_info — the device-side mirror of what the Beacon already reports to POST /api/v1/beacons/sync_statuses.

Sync mechanism — replace the current XML importer (ContentImporter + ContentXmlParser, reconciling on topic_external_id and filename) with a manifest consumer that reconciles providers/topics/tags via find_or_create_by(id:) on SkillRX IDs, keys files on blob.id + checksum, and handles deletes by removing local records absent from the manifest. Retire/rewrite the content *_xml_parser.rb / *_importer.rb services accordingly (the admin/user XML importers are a separate migration concern).

Rails app decisions

#rails-app-decisions The Beacon is a Rails app (passwordless local-user auth, Hotwire, standard Active Storage). Packaging moves the device to SQLite + Docker (see N-3). Other framework choices follow Rails conventions unless there's a reason to differ.

On-device file storage — decided (pending dev gut-check): Active Storage. The Beacon stores synced files via Active Storage (Disk service), with sync implemented as a differential reconcile (download-to-temp → verify checksum → attach; deletes by absence), not the versioned-directory + symlink-swap model from the developer's storage doc. Rationale on two practical criteria: (1) usability by the local app — Active Storage gives native in-app serving, content types, and HTTP Range/streaming for MP4/MP3 with no custom code, directly serving §4.4; (2) manageability during sync — file bookkeeping and relational metadata stay together so a sync reconciles in one DB transaction, whereas a filesystem/symlink swap doesn't cover the SQLite metadata and leaves two stores to coordinate. This also keeps both apps on Active Storage. To confirm with the dev: that no constraint behind the filesystem design (whole-dataset rollback, SQLite/disk limits with large media) should change the call. The manifest's path field is advisory under this approach.

2. Beacon Configuration & Content Scoping

#configuration A Beacon's content is defined by cumulative (AND) filters set in SkillRX. Topics and tags are the derived result of those filters — they are not hand-picked.

  • Language — required. Exactly one per Beacon. English and Spanish are available now; SkillRX supports adding languages, so the set is extensible.
  • Region — optional. Exactly one if set. If set, restricts content to that region's topics.
  • Provider — optional. If set, restricts to one or more providers. Selections are cumulative. For example, a Beacon may be configured for English + Asia region + three specific providers, and will sync only English-language, Asia-region content from those three providers. Decided: topics (and the tags derived from them) are derived from the filters above, not manually assigned per device. The developer protocol docs described assigning topics/tags directly to a device; that is superseded — the SkillRX admin sets the filters and the topic/tag set is the materialized result. The BeaconTopic join is the derived result, not a manual override. Provider relates to Region via Branch, so the Provider choices can themselves be filtered by the chosen Region. Note — a key change from the model doc. The docs/model_relationships.md draft (and the current schema) make a Beacon's Region required. The rule above makes Region optional, Language required. The SkillRX model and the doc both need updating to match.

3. Synchronization & Serving

#synchronization

The detailed protocol lives on a companion page: Beacon Sync Protocol. That page is the canonical, code-matched specification (endpoints, manifest schema, conditional requests, file download/resume, differential update, error handling). This section stays a product-level summary.

Responsibilities. SkillRX owns the configuration of what each Beacon may receive (§2) and serves that as a manifest on request; it does not push to devices. Each Beacon initiates its own sync — it "phones home," authenticates with its API key, requests its manifest, applies the result, and reports status back. SkillRX prepares and waits; the Beacon drives. (When configuration changes, SkillRX only rebuilds that Beacon's manifest in its own database; the change reaches the device on the device's next sync.)

Terminology. "Manifest" means the manifest of files for a sync — the content list (providers/topics/tags/files), not a description of the device. (The software-update mechanism, N-3/N-4, uses a separate release descriptor that should not be called a "manifest.")

Syncing mode

#syncing-mode The Beacon pulls the subset defined in §2 directly from SkillRX.

  • Manifest (SkillRX → Beacon). Beacons::ManifestBuilder produces a versioned manifest (manifest_version, manifest_checksum) at GET /api/v1/beacons/manifest, enumerating the language, the single optional region, tags, providers, topics, and files (with blob.id, checksum, content_type, size_bytes, and a path). It uses ETag conditional requests: the device sends If-None-Match with its cached version and gets 304 when nothing has changed; during a sync it sends If-Match and gets 412 if the manifest changed mid-sync, so the device aborts and restarts. (Already built on the SkillRX side.)
  • Manifest extension needed. The manifest's topic node currently carries only id, name, tag_ids, and files — it does not include published_at (or description). The Beacon needs published_at to build the Month/Year browse hierarchy (§4.3) and Latest Content (§4.7); falling back to the device's local created_at is unreliable (it resets on re-sync/rebuild). Extend ManifestBuilder#build_topic to include published_at (and optionally description). (SkillRX-side change.)
  • File download. GET /api/v1/beacons/files/:id serves file content with Range / resumable download support (206 Partial Content), important for low-connectivity links and large media. Access is scoped to the Beacon's assigned content.
  • Differential update. The device reconciles its local data against the manifest and handles new, changed, and deleted files, keying on SkillRX IDs and blob.id + checksum (§1). Deletes are handled by absence from the manifest. (Beacon-side consumer not yet built; verify delete handling.)
  • Status (Beacon → SkillRX). POST /api/v1/beacons/sync_status reports status (synced / syncing / outdated / error), manifest_version, manifest_checksum, synced_at, files_count, total_size_bytes, device_info, progress_percent (while syncing), and error_message. A device-facing GET /api/v1/beacons/status also exists (purpose to confirm — see Open Questions). (Already built.)
  • Auth. Each device authenticates with a revocable per-device API key, stored hashed server-side (api_key_digest + api_key_prefix). (Already built.)
  • On completion, the device updates its local database and files and returns to serving mode. Note. The skillrx-beacon repo's current importer parses the Pi XML (Server_XML.xml), not the manifest. The manifest consumer described here is the work to build. (Confirm the plan to replace the XML importer with a manifest consumer.)

Serving mode

#serving-mode The device disconnects, broadcasts local Wi-Fi, and serves content from the local database and files (and the local search index, N-1). All §4 functionality runs offline.

SkillRX device-management changes

#device-management

  • Management UI (beacons_controller): register a Beacon, set its Language (required) and optional Region/Provider, view sync status and last-seen, and revoke keys. (Verify completeness.)
  • API: the manifest and sync-status endpoints above. (Already built.)
  • Manifest content: extend the topic node with published_at (± description) per §3. (SkillRX-side change.)
  • Remote update of device software and installations to the extent possible (ties to N-3 and N-4).

4. Feature Requirements — Parity with CMES-mini

#parity The Beacon must reproduce all existing CMES-mini functionality before new capability is layered on. Much of this is already scaffolded in the beacon repo; status is noted per item.

4.1 Identity & access

#identity Local end users sign up with a first and last name, which generates a firstname.lastname user ID, and log in by that ID. Authentication is passwordless — identity only, suited to a shared device. The logged-in user's name shows in the header, with logout. (Already built.) Confirm passwordless is still acceptable. Device administration is a separate, credentialed mode — see §4.10.

4.2 Home & navigation

#home Primary navigation: All Topics, Latest Content, Favorites, Top Topics, Local Folder. The home view also shows a Top Users leaderboard ranked by engagement. (Already built.)

4.3 Browse "All Topics"

#browse Users browse the hierarchy Provider → Month/Year → Topic → file(s), with periods listed newest-first. A topic detail page lists every file with its format and size, and shows a metadata sidebar. Breadcrumb and Back navigation move up the hierarchy. (Already built, except video — see §4.4.) Note: the Month/Year level depends on topic published_at, which must be added to the manifest (§3). Changes from the existing app: the sidebar's Author field should be dropped if authors are retired (§1); Tags remain (and become optional filters once semantic search lands — N-1).

4.4 Content viewing

#viewing Content opens in the app — never redirecting to a separate page — using in-app viewers for PDF, audio, and video (MP4). PDF and audio players are built. Video is a gap: the beacon's TopicFile allows only pdf/mp3, while SkillRX supports MP4 and image types. Closing this is required for true parity and should also cover image content types SkillRX delivers. The schema change is in §1 (replace file_type with content_type over the full SkillRX MIME set). (Gap — high priority.)

4.5 Search

#search A persistent search bar is available throughout the app; results are presented like a browsed result. Baseline search is keyword/tag-based. (Already built — superseded by N-1.)

4.6 Favorites

#favorites Users can favorite content and view a Favorites page of saved items. The legacy app favorites at both the topic and individual file level; the beacon currently favorites at the topic level only. (File-level favoriting is a gap.)

4.7 Discovery

#discovery Latest Content (newest across providers), Top Topics (most viewed this month), and Top Users (ranked by engagement) all rely on usage events recorded per user and topic. (Already built — usage tracking extended by N-5.)

4.8 Local Folder

#local-folder The device offers a Local Folder of content uploaded by a local administrator at the clinic — not synced from SkillRX. Admins can upload, list, and delete local files and folders; local files use the same in-app viewers. (Already built, pending video.) Discussion — is this the right model? The legacy Local Folder conflates two distinct needs: (a) device-local curation by a clinic admin (never synced), and (b) centrally-curated, synced non-topic content from SkillRX (a user manual, standalone references, the CDS tools in N-8). Whether (b) should be a new first-class Resource type rather than living in the Local Folder is open — see N-9. (Raise with Yan.)

4.9 CMES Extras

#cmes-extras A CMES Extras section carries pre-packaged content bundles (e.g. Alfred/POCUS, vENC neonatal care). This content must open in-app like everything else (§4.4); some open as separate pages today. (Partial.)

4.10 Device-local admin

#admin We expect a separate admin mode for managing the device locally, distinct from the passwordless end-user login (§4.1) and using a real username and password. At a minimum, an admin can manage the Local Folder (§4.8); a dashboard with user and admin activity logs is also part of this. (Admin login with credentials is already built; confirm the intended scope of local admin capabilities — this is my current assumption and may need refining.)

5. New Capability

#new-capability These are prioritized goals beyond parity. Each needs acceptance criteria and sizing during design.

  • N-1 · Semantic on-device search. Offline, on-device search that returns meaning-based results (not just keyword/tag) and produces summary responses grounded in the local knowledge base. Free-text input (reuse the search bar); hybrid keyword + semantic retrieval; local index build/refresh as part of the device update; manual tagging no longer required (existing tags become optional filters); summaries via a quantized edge LLM (evaluate NVIDIA NIMs / Ollama / Copilot). Open: model footprint vs. device resources, latency, how summaries cite sources.
  • N-2 · PWA proof-of-concept. Validate an installable, web-based mobile experience for one core workflow end-to-end before committing to a native rebuild; document feasibility and constraints. Rails' built-in PWA support — the generated app/views/pwa/manifest.json.erb and service worker, already present in the repo — is the natural starting point. Open: offline caching on phones, iOS PWA limits.
  • N-3 · Packaging & deployment. A repeatable packaging/release process for the device software; evaluate at least two approaches (Docker, Ansible, others); produce a stored, distributable artifact and a documented process a non-expert can run. Docker work (SQLite, compose files) is already underway.
  • N-4 · "Connect to Update" experience. A clear, error-free update flow with real-time status during sync (not feasible today). Builds on the §3 status reporting.
  • N-5 · Deeper usage tracking. Beyond "opened": capture progress (PDF page, audio playback), closed / session-end, and completion, with a completion definition per content type. The beacon logs opened/view/search/favorite today; progress and completion are the gap.
  • N-6 · In-app continuity. All CME content, including CMES Extras, opens in-app with no separate-page redirects. Same intent as §4.4 and §4.9.
  • N-7 · Updated sync. Covered in §3.
  • N-8 · Clinical Decision Support tools (offline). List and make accessible under CMES Extra: CorePendium, Core Ultrasound, RCH Pediatric care, LITFL. Build ingestion pipelines: CorePendium → CMES cloud storage for sync; Core Ultrasound → an "alfred"-style repo for the Fundamental Course plus an offline question bank for assessment; LITFL → a scraper that loads marked content to SkillRX. Open: licensing per source, update cadence, question-bank data model. Likely depends on N-9.
  • N-9 · Synced non-topic Resource type (proposed). A first-class Resource synced from SkillRX that sits outside the Provider → Topic → Document model, for content that doesn't fit a topic (standalone references, curated extras, possibly the N-8 tools). Distinct from the device-local Local Folder (§4.8). Open: does it replace or complement Local Folder; data model and manifest representation; SkillRX curation UI; how it appears in Beacon navigation; and it must respect the §1 ID discipline (synced Resources carry SkillRX IDs; local content does not). Raise with Yan.

6. Milestones

#milestones Beacon

  1. [SkillRx] Admin can add new beacon device
  2. [SkillRx] Admin can manage content settings for beacon devices (language, provider, region)
  3. [Beacon] Device can be configured with rails app [lots more goes in here that is more tasky than milestoney]
  4. [Beacon] Admin can see manifest pulled from SkillRx
  5. [Beacon] Admin can see list of topics
  6. [Beacon] Admin can see list of topics matching that beacon's SkillRx configuration setttings
  7. [Beacon] Admin can see an individual topic's document names
  8. [Beacon] Admin can view documents (might be multiple milestones since diff doc types have different viewing requirements)
  9. [Beacon] Admin can do a simple search and see matching results
  10. Further milestones TBD
  11. [Beacon] New capabilities, prioritized: video parity → N-1 search → N-5 tracking → N-6 continuity → N-8 CDS. SkillRX
  • Device-management UI complete; manifest + sync-status API hardened; manifest topic node extended with published_at; remote update support; Azure publishing continues in parallel for legacy devices. Packaging
  • Repeatable, documented packaging so a non-expert can configure a Beacon (N-3).

7. Open Questions

#open-questions

  1. Passwordless local-user auth — confirm it stays.
  2. Video/MP4 (and image) support — required for parity (§4.4).
  3. BeaconTopic — derived vs manual override Resolved: topics/tags are derived from filters, not manually assigned (§2).
  4. Region becomes optional, Language required — update the SkillRX model and model_relationships.md. Resolved: exactly one optional region (§2).
  5. Retire the Author model unless a user requirement reintroduces it (§1, §4.3).
  6. Edge LLM choice and resource budget (N-1).
  7. Whether usage analytics (N-5) report back to SkillRX.
  8. CDS source licensing (N-8).
  9. Packaging: Docker vs. Ansible vs. hybrid (N-3) — also gates the software-update protocol (separate release descriptor, not a "manifest").
  10. Synced non-topic content — new first-class Resource type, or keep in Local Folder? (N-9; raise with Yan.)
  11. Manifest must add topic published_at (± description) for Month/Year browse and Latest Content (§3, §4.3). (For Dmitry.)
  12. Confirm per-topic state is unnecessary on the device (manifest pre-filters to active; deletes by absence) (§1).
  13. topic_files key — adopt blob.id as the primary key, or keep a synthetic PK with a unique blob_id column? (§1)
  14. Device sync-state — a dedicated singleton table, or columns on an existing device/config record? (§1)
  15. Checksum algorithm — manifest examples say sha256: but Active Storage blob.checksum is base64 MD5; add a real SHA256 column or accept MD5 and relabel? (For Dmitry.)
  16. On-device storage — confirm Active Storage over the versioned-filesystem model, or name the constraint that favors the filesystem. (For Dmitry — §1 Rails app decisions.)
  17. GET /api/v1/beacons/status — confirm purpose and document. (For Dmitry.)

See the Beacon Sync Protocol companion page for the full code-matched protocol and the consolidated questions for Dmitry.

Clone this wiki locally