0.24.0

Release Highlights

This release includes 140+ pull requests and 25,000+ lines of changes across the server, Web UI, AI pipeline, storage engine, and device sync layer, along with contributions from 6 new contributors.

🤖 Multi-Provider AI Backends & Structured Configuration

• Pluggable AI Model Architecture: Introduced task-specific AI model interfaces (OCR, summarization, embeddings) with structured ai provider and model configuration in config.yaml and per-model concurrency limits (#306, #312, #313, #324, #328, #329).
• OpenAI-Compatible Model Support: Added support for OpenAI-compatible model backends (e.g., Ollama, vLLM, LiteLLM, OpenAI) alongside Google Gemini (#259).
• Search & Summary Pipeline Reliability: Wait for page OCR completion before generating summaries (#362), automatically refresh summaries when note pages change (#361), and exclude deleted files from semantic search results (#254).

🛡️ Storage Integrity, Garbage Collection & Caching

• Two-Way Storage Integrity Verification: Added admin API endpoints and supernote admin CLI commands (integrity, tasks) with per-file issue reporting, remediation hints, pipeline/cache artifact verification, summary blob/hash validation, duplicate case-variant root folder detection, and --full-disk orphan blob/chunk-set scanning (#355, #369, #371, #376, #377, #378, #398, #431).
• Recycle Bin, Summary & Staging Retention Cleanup: Added permanent recycle bin purging with reference-counted blob and derived data cleanup (#288), opportunistic pruning of expired recycle bin entries (#253), folder soft-delete cascade and selective restore (#286), soft-deleted summary retention with async blob cleanup (#383, #393, #438), and automatic pruning of expired staging files, abandoned upload chunks, and superseded conversion artifacts (#295, #416).
• PNG Generation & Caching Fixes: Resolved issues with notebook PNG generation and caching, isolating derived note conversions into a dedicated cache bucket and preventing cache key collisions (#344, #417).

🖥️ Web UI & Knowledge Hub Enhancements

• Rich File Previews & Downloads: Added inline previews for images, PDFs, and text files in FileViewer and FileCard (#391), configurable derived PNG notebook previews (#279), and direct file downloads from the Web UI (#255).
• UI, Navigation, Folder Size & Caching Fixes: Fixes for the Web UI including URL history and tab navigation (#397, #460), subfolder drill-down and breadcrumb navigation when moving files (#458), correcting the display of recursive folder sizes (#456), non-blocking toast notifications (#457), and static asset caching fixes so browser updates apply immediately (#455).

🔄 Device, Partner App & Real-Time Sync

• Real-Time Sync Across Devices, Partner App & Web UI: Completed full Socket.IO compatibility so Supernote devices, the Partner App, and the Web UI stay connected reliably and receive instant real-time updates whenever files, tasks, or digests are synced (#331, #430, #437, #462).
• Root Folder Naming Bug Fix: Fixed a bug in root folder naming so system folders (Note, Document, Export, MyStyle, INBOX, SCREENSHOT) match the device firmware's exact names and existing mixed-case duplicate root folders are automatically merged (#256, #272, #275, #311).
• Schedule & Summary Sync Fixes: Numerous fixes for scheduling and summary sync between Supernote devices and the Partner App, including Inbox task syncing, moving tasks between groups, re-adding deleted summaries, timestamp handling, iCal feed exports, and preventing redundant full-device re-syncs on upgrade (#314, #318, #319, #321, #323, #327, #332, #345, #349, #368, #408, #432, #438, #461, #468).

🔔 Outbound Webhooks & MCP Improvements

• Outbound Webhook Dispatcher: Added an outbound webhook dispatcher for note sync (note.synced) and processing completion (note.processing_completed) events with per-endpoint user email scoping, bounded concurrency, URL validation, and credential redaction (#217, #277, #280, #289, #290, #291, #293, #294).
• MCP Server Enhancements: Added the list_notebooks MCP tool (#292), fixed incorrect integer conversions for file IDs in MCP models and tools (#410), and added the mcp_enabled server configuration option to gate MCP and OAuth startup (#429).

⚡ Performance & Memory Footprint Optimizations

• Dramatically Lower Memory & Faster Conversions: Reduced server idle RAM to ~160 MB (and even lower for bare sync servers when mcp_enabled: false) via lazy top-level package imports (#427), layer streaming and bounded page concurrency during PNG conversion (#418, #425), replacing pypng with Pillow in PngDecoder (#309), and 1 MB blob streaming chunks with hashlib.file_digest (#420).
• VFS SQL Query Optimization: Eliminated $N+1$ path lookup queries when listing and searching files (#424) and optimized recursive directory listings with a single query (#421).

🔒 Security & Authentication Hardening

• Session & Credential Security: Persisted auto-generated JWT secrets in the database (#467), bound sessions to immutable user_id with revocation on credential changes and transparent legacy token migration (#267, #270, #466), enforced single-use login challenges with timestamp validation (#268, #395), shared IP rate limiting across public auth endpoints (#343), and prevented bindEquipment from overwriting devices bound to another user (#422).
• Storage, OAuth & Parser Guards: Verified blob storage path containment (#285), enforced strict signed URL validation with Range download support (#278, #281), enforced single-use MCP OAuth authorization codes and PKCE validation (#298, #305), and added stream/decompression bounds and malformed bitmap/layer validation in the notebook parser (#307, #454, #459).

---

What's Changed

• Fix semantic search returning deleted files by @BorgTen in #254
• test(server): verify semantic search excludes inactive files by @allenporter in #257
• feat(web): download a file from the web UI by @TWDickson in #255
• chore(deps): update dependency ty to v0.0.86 by @renovate[bot] in #260
• fix(files): protect Supernote system folders by location, not by name by @TWDickson in #256
• Apply cruft updates by @allenporter in #261
• Refactor session creation into UserService.create_session by @allenporter in #267
• Enforce single-use login challenges and constant-time digest checks by @allenporter in #268
• Harden auth route DTO validation and update_email checks by @allenporter in #269
• Bind sessions to immutable user_id and revoke on credential changes by @allenporter in #270
• Add logout and account clear endpoints and client wiring by @allenporter in #271
• Enforce stateful session verification in SocketIOServerManager by @allenporter in #273
• Clean up UserService methods, return types, and profile conversion by @allenporter in #274
• feat(server): add outbound webhook dispatcher for note sync events by @proscar87 in #217
• Make SqliteCoordinationService.pop_value atomic with DELETE ... RETURNING by @allenporter in #276
• refactor(webhook): type dispatch payload and reuse client session across retries by @allenporter in #277
• Enforce strict path and query validation in UrlSigner.verify by @allenporter in #278
• docs(webhook): document outbound webhook configuration and payload format by @allenporter in #280
• Make web UI and notebook preview configurable to derive PNG conversion by @allenporter in #279
• feat(files): create system root folders with the device firmware's names by @TWDickson in #272
• Allow reusing signed download URLs for multi-request Range downloads by @allenporter in #281
• feat(mcp): add list_notebooks tool to discover notebook IDs and recent updates by @allenporter in #292
• Prune expired keys in SqliteCoordinationService.set_value by @allenporter in #284
• feat(webhook): add constant-time verify_signature helper by @allenporter in #287
• fix(webhook): validate endpoint URLs and disable redirect following by @allenporter in #291
• feat(webhook): emit note.processing_completed event when note processing pipeline finishes by @allenporter in #290
• Scope Extended system task and processing status APIs to authenticated user by @allenporter in #297
• fix(webhook): share ClientSession lifecycle and bound request concurrency by @allenporter in #293
• fix(webhook): redact credentials and query params from logged URLs by @allenporter in #289
• fix(server): resolve and verify blob storage path containment by @allenporter in #285
• Validate redirect_uri and code_challenge in MCP OAuth login-bridge by @allenporter in #298
• feat(server): prune expired staging files and abandoned upload chunks in LocalBlobStorage (#249) by @allenporter in #295
• feat(webhook): require per-endpoint user email scoping by @allenporter in #294
• feat(vfs): enforce folder soft-delete cascade and selective restore by @allenporter in #286
• feat(server): permanent recycle bin purge with ref-counted blob and derived data cleanup by @allenporter in #288
• feat(server): opportunistically prune expired recycle bin entries by @allenporter in #253
• Enforce single-use MCP OAuth authorization codes and implement token revocation by @allenporter in #305
• Enforce stream and decompression bounds in notebook parser and decoders by @allenporter in #307
• perf(notebook): replace pypng with Pillow in PngDecoder by @allenporter in #309
• feat(db): merge mixed-case system root folders into the firmware's names by @TWDickson in #275
• Update schedule task timestamp on batch updates by @allenporter in #310
• fix(db): fix collision, recycle, and cleanup edge cases in system root migration by @allenporter in #311
• Introduce AI model interfaces, prompt types, and fake implementations by @allenporter in #306
• Add Gemini AI model implementations and factory functions by @allenporter in #312
• Migrate AI processing callers to task-specific model interfaces by @allenporter in #313
• fix: align summary HTTP methods and Socket.IO status code/channels with API spec by @allenporter in #314
• fix: align OpenAPI spec and Python models by @allenporter in #316
• docs: add software design heuristics rule and role cohesion checks by @allenporter in #317
• feat(db): track embedding_model provenance on NotePageContentDO and support backfill by @allenporter in #315
• feat(config): add structured AI provider and model configuration by @allenporter in #324
• Make lastModified optional in ClearScheduleTaskGroupDTO by @allenporter in #322
• Update PyPI author metadata and clean up notebook parser credit by @allenporter in #325
• fix(db,config): backfill embedding_model from startup config and preserve omitted ai capabilities by @allenporter in #326
• Unify schedule task_id and task_list_id as String columns by @allenporter in #327
• feat(ai): wire model factories to structured config.ai and per-model concurrency by @allenporter in #328
• docs(ai): document provider backend contract and structured ai configuration by @allenporter in #329
• Fix iCal export for ungrouped Inbox tasks by @allenporter in #321
• Persist recurrence, completedTime, and lastModified on task creation by @allenporter in #323
• Persist schedule task group taskListId, createTime, and lastModified by @allenporter in #319
• Support filePath and taskTitle in ScheduleTaskLinkDTO by @allenporter in #320
• Support taskListId null for Inbox tasks and moving tasks between groups by @allenporter in #318
• fix(socket): complete Socket.IO v2 (EIO=3) default namespace handshake and client PING/PONG by @allenporter in #331
• Make schedule task_list_id nullable for Inbox tasks by @allenporter in #332
• Scope summary and summary tag unique_identifier by user_id by @allenporter in #334
• Add database schema conformance tests for user_id scoping by @allenporter in #342
• Share IP rate limiter across public auth endpoints by @allenporter in #343
• Fix summary query pagination and unpaginated batch/hash queries by @allenporter in #345
• Extract trace middleware and redact credentials in trace logs by @allenporter in #348
• Persist comment, handwriting, source_type, and author fields on summary/group updates by @allenporter in #349
• Preserve hierarchical keys in LocalBlobStorage to prevent cache collisions by @allenporter in #344
• Require authentication and scope account ownership on equipment unlink by @allenporter in #351
• Implement equipment binding query by serial number by @allenporter in #353
• Standardize on /api/user/account/clear and require confirm=True in AdminClient by @allenporter in #352
• feat(server): support OpenAI-compatible model backends by @littleblackfish in #259
• fix(ai): harden exception handling, summary JSON parsing, and vector validation by @allenporter in #356
• test(server): replace mocks with fakes across AI and processor tests by @allenporter in #360
• fix(server): propagate missing-blob and parse errors in PageHashingModule by @allenporter in #363
• fix(server): wait for page OCR completion before running SummaryModule by @allenporter in #362
• fix(server): invalidate summary task when note pages change by @allenporter in #361
• Return UserCheckVO from check/exists/server and add user check/login routes by @allenporter in #357
• refactor(tests): replace remaining service and admin route mocks with real fixtures by @allenporter in #366
• Revert authentication requirement on POST /api/terminal/equipment/unlink by @allenporter in #365
• feat(admin): expose storage integrity and system tasks endpoints and CLI commands by @allenporter in #355
• refactor(cli): remove dead list_users call and replace CLI test mocks with real server by @allenporter in #367
• Preserve client lastModified timestamp when updating schedule tasks by @allenporter in #368
• feat(integrity): add per-file issue details and file_id filtering by @allenporter in #369
• Allow clearing nullable fields when updating schedule tasks by @allenporter in #370
• Include partUploadUrl and standard innerName in web upload apply response by @allenporter in #372
• feat(integrity): verify note processing pipeline and cache artifacts by @allenporter in #371
• refactor(integrity): push file_id filter and orphan parent check into SQL by @allenporter in #374
• Drop existing primary key before recreating composite primary key in migrations by @allenporter in #375
• feat(integrity): detect duplicate case-variant root folders by @allenporter in #376
• feat: add IntegrityIssueType enum and server remediation hints by @allenporter in #377
• fix: handle page_ task key prefix in integrity pipeline check by @allenporter in #379
• feat(integrity): validate SummaryDO blobs, hashes, and parent/file references by @allenporter in #378
• fix: align TaskType.PAGE_HASHING and processor modules on TaskType enum by @allenporter in #380
• fix: register ephemeral bootstrap user as admin with system folders by @allenporter in #382
• fix(summary): cascade soft-deletion to child summaries when deleting a group by @allenporter in #383
• refactor: replace individual integrity report counters with issue_counts by @allenporter in #384
• feat(web): support previewing images, PDFs, and text files in FileViewer and FileCard by @allenporter in #391
• fix: include exception type in last_error and increment retry_count on task failure by @allenporter in #392
• fix(oss,web): use strict inline MIME allowlist, sandbox CSP, and direct PDF iframe URL by @allenporter in #394
• fix(auth): validate login challenge timestamp and log auth failure diagnostics by @allenporter in #395
• refactor: use email instead of user_id in admin integrity API and CLI by @allenporter in #398
• feat(web): persist folder navigation and open file state in URL history by @allenporter in #397
• fix(sync): require completed sync for synType and track account soldOutCount by @allenporter in #408
• fix(mcp): serialize file_id as string in MCP models and tools by @allenporter in #410
• test(summary): verify SummaryModule failure marks SystemTaskDO FAILED by @allenporter in #411
• fix(storage): delete superseded blobs and stale conversion artifacts when overwriting a file by @allenporter in #416
• perf(blob): use 1 MB streaming chunks and hashlib.file_digest for MD5 by @allenporter in #420
• perf(notebook): optimize single-page loading and layer PNG conversion by @allenporter in #418
• perf(vfs): optimize recursive directory listing with a single CTE query by @allenporter in #421
• perf(processor): trigger device note processing on SyncCompletedEvent by @allenporter in #419
• fix(auth): prevent bindEquipment from overwriting a device bound to another user by @allenporter in #422
• docs(readme): update server idle memory footprint to ~160MB by @allenporter in #423
• perf(vfs): eliminate N+1 get_full_path queries in query_file_list and search_files by @allenporter in #424
• Extract ConversionService and store note conversions in CACHE_BUCKET by @allenporter in #417
• docs(readme): update Python developer API snippets to match public package exports by @allenporter in #426
• perf(processor): reduce peak PNG conversion RAM via layer streaming and page concurrency by @allenporter in #425
• feat(server): add mcp_enabled config option to gate MCP server and OAuth startup by @allenporter in #429
• perf: lazily import top-level subpackages in supernote.init by @allenporter in #427
• docs: update RAM guidance with full vs bare sync server setups by @allenporter in #433
• feat(sync): move sync_locks to CoordinationService and align Socket.IO models by @allenporter in #428
• test(socket): add 1:1 unit test suites for SupernoteSocketClient and LocalEventBus by @allenporter in #436
• feat(socket): publish domain events for TASK-SYN, DIGEST-SYN, and FILE-SYN notifications by @allenporter in #437
• fix(socket): handle optional payload in Ratta ping and client_log events by @kandrilenas in #430
• fix(schedule): normalize millisecond timestamps and local date format… by @kandrilenas in #432
• feat(summary): add async blob cleanup and soft-deleted summary retention by @allenporter in #393
• fix(summary): support re-adding summaries and groups after soft-deletion by @allenporter in #438
• Add --full-disk two-way storage integrity check and chunk-set TTL evaluation by @allenporter in #431
• fix(web): load directory when switching to Files tab from #/tasks by @allenporter in #460
• fix(notebook): reject unsupported bitmap bit depths by @kocaemre in #454
• fix(notebook): raise ParserException on malformed layer info payload by @nosmile99 in #459
• fix(auth): return UserErrorCodeEnum codes (E0019/E0018/E0045) with HTTP 200 on login by @allenporter in #463
• fix(schedule): use null taskListId for Inbox tasks and validate group existence by @allenporter in #461
• feat(web): compute and display recursive folder sizes in the file browser by @allenporter in #456
• fix(web): set Cache-Control headers on /static/ assets to prevent stale ES module caching by @allenporter in #455
• feat(web): replace blocking window.alert() error popups with non-blocking toast notifications by @allenporter in #457
• feat(web): connect Socket.IO client and re-poll on sync events by @allenporter in #462
• feat(web): support subfolder drill-down and breadcrumb navigation in MoveModal by @allenporter in #458
• Persist auto-generated JWT secret key in database via CoordinationService by @allenporter in #467
• fix(auth): support and transparently migrate legacy session tokens by @allenporter in #466
• fix(server): seed sync:initialized for existing users on upgrade by @allenporter in #468

New Contributors

• @BorgTen made their first contribution in #254
• @TWDickson made their first contribution in #255
• @littleblackfish made their first contribution in #259
• @kandrilenas made their first contribution in #430
• @kocaemre made their first contribution in #454
• @nosmile99 made their first contribution in #459

Full Changelog: 0.23.0...0.24.0