Repository navigation
Data Storage
KHost Cue is local/offline-only: every list, cache, and setting lives in the app's private on-device storage behind a small set of interfaces. This page documents the storage pattern, the exact files on disk, and how to add a new persisted type safely.
Every JSON-backed store derives from JsonFileStore<TItem, TCache> (src/KHost.Mobile.Infrastructure/Services/JsonFileStore.cs), which owns:
- The
Gate(aSemaphoreSlim(1, 1)) that serializes every public method. - The in-memory cache, and the rule that
LoadAsync/SaveAsyncassume the caller already holds the gate. - The crash-safe write (
.tmpsibling + atomic rename, via the injectedIAtomicFileWriter) and the corrupt-file quarantine — aJsonExceptionmoves the bad bytes to a.corruptsibling and the store starts empty rather than crashing. - The
Changedevent, raised throughRaiseChanged()after the gate is released and only when something actually changed.
The store classes themselves (JsonFileSongListStore, JsonFileTonightStore, JsonFileSingerStore, JsonFileVenueStore, JsonFileLyricsCache, AtomicFileWriter, SingerFileNames) are internal sealed — KHost.Mobile.Infrastructure.csproj grants <InternalsVisibleTo> to both KHost.Mobile.UnitTests and KHost.Mobile.IntegrationTests, which is how the integration suite drives them directly while every other project sees only the I* interface in Abstractions.
A subclass supplies which file, which serializer context, and a label for log lines. Most stores cache the stored list as-is and use the one-parameter JsonFileStore<T>:
internal sealed class JsonFileFooStore : JsonFileStore<Foo>, IFooStore
{
private readonly string _filePath;
public JsonFileFooStore(IAppDataDirectory paths, ILogger<JsonFileFooStore>? logger = null, IAtomicFileWriter? writer = null)
: base(logger ?? NullLogger<JsonFileFooStore>.Instance, writer)
=> _filePath = Path.Combine(paths.AppDataDirectory, "foo.json");
protected override JsonTypeInfo<List<Foo>> TypeInfo => FooJsonContext.Default.ListFoo;
protected override string Label => "Foos";
protected override string PathFor(Guid? key) => _filePath;
public async Task<IReadOnlyList<Foo>> GetAllAsync()
{
await Gate.WaitAsync();
try { return await LoadAsync(); }
finally { Gate.Release(); }
}
public async Task AddAsync(Foo item)
{
await Gate.WaitAsync();
try
{
var items = await LoadAsync();
items.Add(item);
await SaveAsync(items);
}
finally { Gate.Release(); }
RaiseChanged(); // after release, and a real change happened
}
}Three optional overrides cover the stores that differ:
| Override | Used by | Why |
|---|---|---|
CurrentKey |
JsonFileSongListStore, JsonFileTonightStore
|
Returns the active singer, so the cache invalidates on a switch and the file path follows. See Per-singer namespacing. |
Project / Flatten
|
JsonFileLyricsCache |
Its in-memory shape is a Dictionary keyed by title+artist while the file is a flat array — the only store whose cache isn't the list. It derives from the two-parameter JsonFileStore<TItem, TCache>. |
Adding is never an overwrite. JsonFileSingerStore and JsonFileVenueStore both take an id from outside (an imported profile or venues file carries the ids it was exported with), and both re-key a newcomer whose id is already in use. Two rows sharing an id would be indistinguishable to UpdateAsync, which edits the first, and to RemoveAsync, which deletes every match. A non-colliding id is kept, which is what lets a venue import keep performances resolving to the venue they were tagged at.
The per-singer write rule is enforced by the base: LoadAsync records the key it read under, and SaveAsync writes to that key's path — never a freshly re-read active-singer id. A singer switch landing between a load and its write cannot put one singer's songs into another singer's file.
Plus the source-gen context (see Serialization) and a services.AddSingleton<IFooStore, JsonFileFooStore>(); line in AddKHostInfrastructure (src/KHost.Mobile.Infrastructure/Project.cs) — the registration has to live there, not in the head's MauiProgram.cs, because the store classes are internal and only their own project can name them.
One DI wrinkle: JsonFileSongListStore is registered twice in AddKHostInfrastructure — once as itself, once as ISongListStore resolving to the same instance (sp.GetRequiredService<JsonFileSongListStore>()) — so callers that need the concrete type (the profile export/import path) and callers going through the interface share one singleton rather than getting two independent caches.
Everything below lives under the app's private data directory (see On-device paths) unless noted otherwise.
| File | Holds | Store | Serializer context |
|---|---|---|---|
song-list-{id}.json (per singer) / song-list.json (no singer active) |
List<SongListItem> — the wishlist |
JsonFileSongListStore |
SongListJsonContext |
tonight-{id}.json (per singer) / tonight.json (no singer active) |
List<TonightEntry> — tonight's on-deck set |
JsonFileTonightStore |
TonightJsonContext |
singers.json |
List<Singer> — the shared roster |
JsonFileSingerStore |
SingerJsonContext |
venues.json |
List<Venue> — the shared venue list |
JsonFileVenueStore |
VenueJsonContext |
lyrics-cache.json |
List<LyricsCacheEntry> (in memory: Dictionary<string, LyricsCacheEntry> keyed by trimmed, lowercased title+artist) |
JsonFileLyricsCache |
LyricsCacheJsonContext |
album-art/<SHA-256 hex of source URL> |
raw cover-image bytes, one file per unique artwork URL (no extension) | AlbumArtCache |
— binary blob, not JSON |
<file>.tmp |
in-flight write; renamed over the target on success (see Atomic writes) | AtomicFileWriter.WriteAsync |
n/a |
<file>.corrupt |
a file that failed to parse, moved aside instead of being silently overwritten | AtomicFileWriter.Quarantine |
n/a |
Settings are not in this table — they use MAUI Preferences, a separate mechanism (see Settings storage).
Import/Export (src/KHost.Mobile.UI/Components/Pages/ImportExport.razor, src/KHost.Mobile.Infrastructure/Services/SingerProfileCodec.cs) writes/reads JSON too, but to a user-chosen location via the OS share/save picker, not into AppDataDirectory:
| Export / import | Shape | Codec method | Serializer context |
|---|---|---|---|
| Singer profile |
SingerProfile — one Singer + their full List<SongListItem> (including sung history) |
SingerProfileCodec.Serialize/ParseProfile
|
ProfileJsonContext |
| Venues export | bare List<Venue>
|
SerializeVenues/ParseVenues
|
VenueJsonContext |
SingerProfileCodec.Detect(json) classifies a picked profile-import file by its JSON root — an object carrying a Singer property is ProfileFileKind.Profile; anything else (or invalid JSON) is ProfileFileKind.Invalid — so the import page routes to ParseProfile without the caller re-parsing. ProfileFileKind (src/KHost.Mobile.Abstractions/Models/ProfileFileKind.cs) is a standalone enum, alongside every other persisted model.
SingerProfile excludes the Tonight set (on-the-night, device-local), Venues (shared, exported separately) and Settings — a profile import/export moves identity + wishlist only.
The song-list and Tonight stores are namespaced per singer through the injected ISingerFileNames (src/KHost.Mobile.Abstractions/Services/ISingerFileNames.cs, implemented by internal sealed class SingerFileNames in src/KHost.Mobile.Infrastructure/Services/SingerFileNames.cs):
string SongList(Guid singerId) => $"song-list-{singerId:N}.json";
string Tonight(Guid singerId) => $"tonight-{singerId:N}.json";{singerId:N} is the dash-less GUID format ("N"), e.g. song-list-3fa85f6457174562b3fc2c963f66afa6.json. JsonFileSongListStore, JsonFileTonightStore and JsonFileSingerStore (which deletes a removed singer's files) all take an optional ISingerFileNames? names = null constructor parameter, falling back to new SingerFileNames() — the same optional-DI-parameter shape as the logger and the clock.
Both stores take an optional IAppSession? session in their constructor:
-
With a session wired in (the real app),
LoadAsyncreadssession.ActiveSingerIdand resolves the singer's own file viaPathFor(singerId). It caches_loadedForalongside the in-memory list; a mismatch between_loadedForand the currentActiveSingerIdinvalidates the cache and reloads.SaveAsyncwrites to_loadedFor(the singer the data was loaded for, captured before anyawait), never a freshly re-readActiveSingerId— so a singer switch landing mid-operation can't write one singer's data into another singer's file. -
With no session (
session is null— the integration-test path, or pre-seed before any singer exists),PathFor(null)falls back to the unsuffixed file name (SingerFileNames.DefaultSongList="song-list.json",DefaultTonight="tonight.json"). - The store subscribes to
session.ActiveSingerChangedand re-raises its ownChangedevent on a switch, so UI bound toChangedreloads the new singer's data exactly as it would after any mutation.
tests/KHost.Mobile.IntegrationTests/Infrastructure/Services/PerSingerStoreTests.cs exercises this directly: two singers get independent lists and independent on-disk files, a switch fires Changed exactly once (not on a same-singer "switch"), and the session-less constructor falls back to song-list.json.
IAtomicFileWriter (src/KHost.Mobile.Abstractions/Services/IAtomicFileWriter.cs), implemented by internal sealed class AtomicFileWriter (src/KHost.Mobile.Infrastructure/Services/AtomicFileWriter.cs), gives every store crash-safe writes. JsonFileStore<TItem, TCache> takes it as an optional constructor parameter (IAtomicFileWriter? writer = null, defaulting to new AtomicFileWriter()) and exposes it to subclasses as the protected Writer property:
public async Task WriteAsync(string path, Func<Stream, Task> writeContents)
{
var tmp = path + ".tmp";
await using (var stream = File.Create(tmp))
await writeContents(stream);
File.Move(tmp, path, overwrite: true); // same-volume rename → atomic
}The .tmp file must sit in the same directory as the target so File.Move is a same-volume atomic rename, not a cross-volume copy. An app kill or power loss mid-write leaves only the .tmp file — the rename either completes (new data) or never happens (old data intact), so the previous good file always survives.
Writer.Quarantine(path) is the complementary half: when a file fails to parse on load, it's moved aside to <file>.corrupt rather than left in place to be silently clobbered by the next successful save — preserving the bad bytes for manual recovery, and it logs a warning (ILogger<AtomicFileWriter>, optional/defaulted the same way as every store's logger) naming the file it quarantined, so a corrupt-file event leaves a breadcrumb instead of degrading silently. Best-effort (returns false on a locked/vanished file, logged as a second warning); the caller starts empty either way.
Every persisted type goes through a System.Text.Json source-generation context ([JsonSourceGenerationOptions(WriteIndented = true)] [JsonSerializable(typeof(List<T>))] public partial class FooJsonContext : JsonSerializerContext;) rather than the reflection-based serializer. This keeps (de)serialization trimming-/AOT-friendly on the MAUI heads — the source generator emits the metadata the trimmer/AOT compiler needs at build time, instead of JsonSerializer reflecting over types at runtime (which the trimmer can strip and NativeAOT can't do at all). WriteIndented = true throughout so every on-disk file stays human-readable for debugging.
When adding a new persisted type, you must add it to a JsonSerializerContext — miss it and it fails at runtime, not compile time. JsonSerializer.SerializeAsync/DeserializeAsync calls in this codebase always pass an explicit SomeContext.Default.SomeTypeName metadata argument (never the ambient-reflection overload), so a type the generator doesn't know about throws NotSupportedException: … is not compatible with source generation the first time a store tries to read or write it — not when the project builds. Concretely:
- Add
[JsonSerializable(typeof(List<Foo>))]to the owning context (a new one, or an existing one if the type is naturally grouped — e.g.SongListJsonContextalso declaresPerformancebecause it's nested insideSongListItem). - If the type is only ever serialized as a bare element inside another serialized type (like
PerformanceinsideSongListItem), the generator still needs its own[JsonSerializable(typeof(Performance))]line if anything serializes/deserializes it standalone (SongListJsonContextdoes, for exactly this reason). - Existing contexts, one per feature:
SongListJsonContext,TonightJsonContext,SingerJsonContext,VenueJsonContext,LyricsCacheJsonContext,ProfileJsonContext(all insrc/KHost.Mobile.Infrastructure/Serialization/).
Not every type in src/KHost.Mobile.Abstractions/Models/ needs a context — RatingConfig, RatingContext and SurpriseOptions are pure records built at read time from IAppSettings (see Ratings & history, Surprise Me), never written to a file of their own, so they carry no [JsonSerializable] entry.
Services/IAppSettings.cs + Services/MauiAppSettings.cs back every toggle and tunable (auto-fill, YouTube/Spotify/lyrics links, Tonight, Surprise, per-performance rating, update checks, undo window, launch destination, rating weighting, venue detection radius/history, spelling-suggestion level, catalogue region, import lookup delay, haptics, 12/24-hour time, etc.) via MAUI Preferences — a flat, synchronous key/value store, not the JSON-file mechanism above. Every property is Preferences.Default.Get(key, defaultValue) / .Set(key, value), string-keyed ("settings.autofill_metadata", …), with each default matching the hardcoded value it stands in for, so a fresh install is unchanged.
Consequences of that separate storage mechanism:
-
Not in a profile export.
SingerProfile(see File inventory) carries onlySinger+List<SongListItem>— Settings never travel with an exported/imported profile. -
Not per-singer. Unlike the song-list/Tonight stores,
MauiAppSettingshas no singer-awareness at all — every setting is one flat value shared by every singer on the device (confirmed in Singers: Settings sits in the "Shared across everyone" column, alongside Venues). -
Platform-native storage, not a KHost Cue-owned file: on Android this is
shared_prefs/*.xmlunder the app's private data directory (see On-device paths);scripts/backup-device-data.cscaptures it alongsidefiles/*.jsonfor exactly this reason. It is not readable/editable as plain JSON the way the stores above are. -
LastActiveSingerId(which singer to reopen on launch) andVenuesSeeded/TutorialCompleted/etc. (one-time bootstrap flags) live here too — they're settings-shaped state, not domain data, even though they're not user-facing toggles.
Lyrics (Services/JsonFileLyricsCache.cs) follows the plain JSON-store pattern above — lyrics-cache.json, keyed by trimmed, lowercased title+artist (not the Matching/ normalizer — no accent-stripping or punctuation-folding here), including cached misses (Found = false) so a confirmed "no lyrics" answer never re-hits LRCLIB. Cleared via Settings' danger zone (ClearAsync); CountAsync backs the "X cached" display.
Album art is two layers, not one:
-
Services/AlbumArtCache.cs(IAlbumArtCache, registered singleton) is the disk layer: each cover is a raw image file underalbum-art/, named by the SHA-256 hex hash of its source URL (no in-memory copy — a repeat request just re-opens the cheap on-disk file).ClearAsyncdeletes every file in the folder (Settings' "Clear album art");CountAsyncwalks the folder- Covers are written through
IAtomicFileWriter(a.tmpsibling, then an atomic rename), the same as the JSON stores. "Is it cached?" isFile.Exists, so a write torn by an app kill would leave a truncated image that counts as a cache hit forever: the card renders broken and never re-downloads. A failed atomic write never reaches the rename, so no target file exists and the next request fetches it again. The read path also treats a 0-byte file as not cached.
- Covers are written through
-
Services/AlbumArtService.cs(IAlbumArtService, registered scoped — it talks to the WebView throughIJSRuntime, itself scoped per Blazor Hybrid WebView) is the in-memory layer on top: it owns a song-id →blob:object-URL map internally, but the UI never reads a URI directly — it calls the publicViewFor(song), which returns anAlbumArtView(a ready--kh-card-art: url('…')CSS declaration, aLoadingflag, orAlbumArtView.None) and, by asking, kicks off discovery/download/caching for a cover that isn't ready yet. (The privateUriFor(song)helper behindViewForis what actually reads/populates that URL map.) This map is not persisted — it's rebuilt from the disk cache (or a fresh network lookup) as the UI re-requests covers, and is held until a singer switch, an edit, or a cache clear.
The in-memory map is capped, viewport-aware rather than keyed on the rendered card count — My Songs keeps every scrolled-past card in the DOM and re-requests its cover on every render, so a cap keyed off rendered count would evict a cover and have the very next render immediately re-request it, forever. wwwroot/js/art-visibility.js runs one IntersectionObserver over every element carrying data-art-song and reports the on-screen set to AlbumArtService.SetVisibleAsync/VisibleArtChanged; nothing currently visible is ever evicted. Off-screen covers get OffScreenCovers (40) worth of headroom for scrolling back before EvictOffScreenAsync drops the longest-gone first. See DEVELOPMENT.md → Design notes for more.
| Platform |
FileSystem.AppDataDirectory (JSON stores + album-art/) |
Preferences (Settings) |
|---|---|---|
| Windows (unpackaged) |
%LOCALAPPDATA%\KHost\khost.mobile\Data\ — the Data subfolder, not the parent khost.mobile\ (see the gotcha below) |
app-local settings store |
| Android | /data/data/khost.mobile/files/ |
/data/data/khost.mobile/shared_prefs/*.xml |
| iOS / Mac Catalyst | the app's private sandboxed container (MAUI Essentials-managed; no exact path is documented in this repo, and there's no adb-equivalent backup tool for it here) |
platform-native preference store |
FileSystem.AppDataDirectory is the Data SUBFOLDER on Windows, i.e. %LOCALAPPDATA%\KHost\khost.mobile\Data\ — the parent folders are the appxmanifest PublisherDisplayName (KHost) and ApplicationId (khost.mobile). Seeding or inspecting persisted state on Windows must target Data\, not the parent khost.mobile\. A build from before the publisher/id rename wrote to the legacy %LOCALAPPDATA%\<User Name>\com.companyname.khost.mobile\Data\, which current builds ignore.
Backing up device data (Android only today) — scripts/backup-device-data.cs, a .NET 10 file-based script run with dotnet run, pulls files/ + shared_prefs/ off the device via adb run-as khost.mobile (Debug builds only, no root needed) into a gitignored device-backups/*.tar.gz:
dotnet run scripts/backup-device-data.cs -- backup # timestamped .tar.gz -> device-backups/
dotnet run scripts/backup-device-data.cs -- list # what backups exist
dotnet run scripts/backup-device-data.cs -- inspect <file.tar.gz> # peek inside one
dotnet run scripts/backup-device-data.cs -- restore <file.tar.gz> # push a backup back onto the device
# -s, --serial <serial> target one of several attached devices (also honors $ANDROID_SERIAL)
# restore also takes -y/--yes to skip the confirmationAlways back up before a device deploy that might reinstall (a signing-key mismatch from a fresh machine/regenerated debug keystore, a package-id change, or a manual uninstall-to-fix-a-crash all silently wipe on-device data — a redeploy alone does not). See Building & testing for the full deploy workflow this fits into.
KHost.Mobile.IntegrationTests exercises the JSON stores against real file I/O (not mocks) via TempAppDataDirectory (tests/KHost.Mobile.IntegrationTests/TempAppDataDirectory.cs) — an IAppDataDirectory backed by a fresh, unique temp folder per test instance, deleted on Dispose():
public sealed class TempAppDataDirectory : IAppDataDirectory, IDisposable
{
public string AppDataDirectory { get; } // Path.Combine(Path.GetTempPath(), "khost-tests", Guid.NewGuid():N)
public string FilePath(string fileName) => Path.Combine(AppDataDirectory, fileName);
// Dispose(): best-effort Directory.Delete(AppDataDirectory, recursive: true)
}FilePath(name) lets a test seed or hand-corrupt the raw JSON before a store ever touches it, exercising the load/migrate/corrupt-file paths directly rather than only through the store's own API. Coverage by file:
-
JsonFileSongListStoreTests.cs,JsonFileTonightStoreTests.cs,JsonFileSingerStoreTests.cs,JsonFileVenueStoreTests.cs,JsonFileLyricsCacheTests.cs,AlbumArtCacheTests.cs— each store's own CRUD/corrupt-file/quarantine behavior. -
PerSingerStoreTests.cs— the namespacing contract described above (independent files per singer,Changedon a real switch only, unsuffixed-file fallback with no session). -
SingerFileNamesTests.cs— the{id:N}file-naming helpers. -
SingerProfileImportTests.cs— the profile import/export round-trip (SingerProfileCodec). -
AtomicWriteTests.cs—AtomicFileWriter's crash-safety contract directly. -
ProjectTests.cs— both DI entry points (AddKHostInfrastructure,AddKHostClients): every registration resolves from a container built withValidateOnBuild, the stores are singletons, the song-list store is one instance behind both of its registrations, and each typed client reaches the host's handler hook. - Each store's tests also pin that a file written by an earlier build still loads — literal PascalCase JSON, not a re-serialized list, since a naming policy added to a context would round-trip through itself while orphaning every file already on a device.
The integration test project targets plain net10.0 (no MAUI workload needed) and pulls in the MAUI-free source it covers (models, stores) via ordinary project references to KHost.Mobile.Abstractions, .Common, .Infrastructure and .Clients — none of which touch the MAUI workload. See Building & testing for the run commands and the unit-vs-integration split.
- Architecture — where these stores sit in the app's overall shape.
- Singers — the multi-singer model this per-singer namespacing serves.
- Import & export — the profile/venue file round-trip that sits on top of these stores.
- Building & testing — running the integration test suite, and the device-deploy data-loss gotchas.
- Conventions — the broader local-store and client-backend coding patterns.