Skip to content

Resolve #330: interactive MistDemo (web toggle + native app refresh)#332

Merged
leogdion merged 9 commits into
v1.0.0-beta.1from
330-interactive-mistdemo
May 14, 2026
Merged

Resolve #330: interactive MistDemo (web toggle + native app refresh)#332
leogdion merged 9 commits into
v1.0.0-beta.1from
330-interactive-mistdemo

Conversation

@leogdion
Copy link
Copy Markdown
Member

@leogdion leogdion commented May 12, 2026

Summary

Closes out the full #330 plan on the umbrella branch 330-interactive-mistdemo. All six sub-issues land here.

Phase 1 — Foundation refactors

Phase 2 — Web CRUD + comparison

Phase 3 — Database selection

Phase 4 — Native app rework

Adjacent polish that rode along

Test plan

  • swift build clean on the branch
  • swift test — full suite green at each sub-PR merge
  • ./Scripts/lint.sh — clean
  • CI green on this PR
  • Manual smoke: mistdemo web against a real container, exercise both MistKit and CloudKit JS toggles on public and private DBs
  • Manual smoke: MistDemoApp on iOS — confirm public/private picker behaves

Resolves #288
Resolves #289
Resolves #274
Resolves #329
Resolves #275
Resolves #328
Resolves #330

🤖 Generated with Claude Code

leogdion and others added 2 commits May 11, 2026 17:06
…kenCommand

AuthTokenCommand now only owns lifecycle and channel plumbing. Route
construction moves to a reusable AuthTokenServer; loopback validation
moves to a standalone LoopbackAuthority helper. Both gain dedicated
unit + router-level tests via HummingbirdTesting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CloudKit auth-flow page moves out of four Swift raw-string files
and into a single auth-token-index.html resource in MistDemoKit's
bundle. AuthTokenIndexHTML becomes a thin Bundle.module loader.

The original CodeSign concern doesn't apply: MistDemoApp (the iOS
target) has no dependency on MistDemoKit, so the resource never ships
in an iOS app bundle. The mistdemo CLI executable that does consume
MistDemoKit is macOS / Linux only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 12, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d51650b3-a2cb-4226-9167-e20e71c08436

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 330-interactive-mistdemo

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

❌ Patch coverage is 73.75887% with 333 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.53%. Comparing base (a28ab3c) to head (6be4939).

Files with missing lines Patch % Lines
...Demo/Sources/MistDemoKit/Commands/WebCommand.swift 0.00% 70 Missing ⚠️
.../Sources/MistDemoKit/Configuration/WebConfig.swift 0.00% 62 Missing ⚠️
...ources/MistDemoKit/Commands/AuthTokenCommand.swift 0.00% 47 Missing ⚠️
...stDemo/Sources/MistDemoKit/Server/WebBackend.swift 0.00% 34 Missing ⚠️
...Sources/MistDemoKit/Server/WebBackendFactory.swift 11.76% 15 Missing ⚠️
...ces/MistDemoKit/Commands/DemoInFilterCommand.swift 0.00% 11 Missing ⚠️
...ources/MistDemoKit/Commands/DemoErrorsRunner.swift 0.00% 8 Missing ⚠️
...rces/MistDemoKit/Commands/UploadAssetCommand.swift 0.00% 8 Missing ⚠️
...emoKit/Integration/Phases/ModifyRecordsPhase.swift 0.00% 5 Missing ⚠️
...ce/Extensions/CloudKitService+Classification.swift 28.57% 5 Missing ⚠️
... and 30 more
Additional details and impacted files
@@                Coverage Diff                @@
##           v1.0.0-beta.1     #332      +/-   ##
=================================================
+ Coverage          69.46%   70.53%   +1.06%     
=================================================
  Files                529      551      +22     
  Lines              14457    15426     +969     
=================================================
+ Hits               10042    10880     +838     
- Misses              4415     4546     +131     
Flag Coverage Δ
mistdemo-spm-macos 60.04% <67.13%> (+1.40%) ⬆️
mistdemo-swift-6.2-jammy 60.03% <67.13%> (+1.38%) ⬆️
mistdemo-swift-6.2-noble 60.03% <67.13%> (+1.35%) ⬆️
mistdemo-swift-6.3-jammy 60.03% <67.13%> (+1.38%) ⬆️
mistdemo-swift-6.3-noble 60.03% <67.13%> (+1.38%) ⬆️
spm 52.58% <82.40%> (+0.90%) ⬆️
swift-6.1-jammy 52.41% <86.11%> (+0.73%) ⬆️
swift-6.1-noble 52.66% <82.40%> (+0.95%) ⬆️
swift-6.2-jammy 52.28% <82.40%> (+0.91%) ⬆️
swift-6.2-noble 52.69% <82.40%> (+1.38%) ⬆️
swift-6.3-jammy 52.47% <83.33%> (+1.16%) ⬆️
swift-6.3-noble 52.28% <82.40%> (+0.93%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread Examples/MistDemo/Sources/MistDemoKit/Server/AuthTokenIndexHTML.swift Outdated
- AuthTokenIndexHTML drops the guard-let-else pyramid in favor of a
  single try! / force-unwrap; the resource is shipped in MistDemoKit's
  bundle so failure here is a build-system bug, not a runtime condition.
  (review nit r3225504184)
- AuthTokenCommandTests+Timeout's "throws on timeout" duplicates
  AsyncHelpersTests+Timeout's coverage but lacked its withKnownIssue
  gate. Under visionOS-simulator CI load the operation's single 1s
  Task.sleep can outrun the polling timeout's many short sleeps, so the
  test recorded "Should have timed out" instead of catching
  AsyncTimeoutError. Mirror the AsyncHelpers gate.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@claude
Copy link
Copy Markdown

claude Bot commented May 12, 2026

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

MistKit is a Swift Package for Server-Side and Command-Line Access to CloudKit Web Services. It targets cross-platform Swift (including Linux, WASI, and Windows) using modern Swift concurrency and code generated from Apple's CloudKit Web Services OpenAPI specification.

Key Project Context

  • Purpose: Provides a Swift interface to CloudKit Web Services (REST API) rather than the CloudKit framework
  • Target Platforms: Cross-platform including macOS, iOS, tvOS, watchOS, visionOS, Linux, WASI, and Windows
  • Default Branch: main
  • API Reference: The openapi.yaml file contains the OpenAPI 3.0.3 specification for Apple's CloudKit Web Services
  • Code Generation: Generated client code lives in Sources/MistKit/Generated/ and is not committed

Development Commands

Swift Package Commands

# Build the package
swift build

# Run tests
swift test

# Run tests with coverage
swift test --enable-code-coverage

# Build for release
swift build -c release

# Clean build artifacts
swift package clean

# Update dependencies
swift package update

# Resolve package dependencies
swift package resolve

# Generate Xcode project (if needed)
swift package generate-xcodeproj

OpenAPI Code Generation

# Generate OpenAPI client code (run this after modifying openapi.yaml)
./Scripts/generate-openapi.sh

# Or manually with swift-openapi-generator
swift run swift-openapi-generator generate \
    --output-directory Sources/MistKit/Generated \
    --config openapi-generator-config.yaml \
    openapi.yaml

Development Workflow

# Run specific test
swift test --filter TestClassName.testMethodName

# Run tests in parallel
swift test --parallel

# Show test output
swift test --verbose

# Format + lint
# swift-format, swiftlint, periphery, and swift-openapi-generator are pinned
# in mise.toml — do NOT invoke them from PATH directly. Run them THROUGH mise:
mise exec -- swift-format -i -r Sources/ Tests/
mise exec -- swiftlint              # lint
mise exec -- swiftlint --fix        # auto-fix

# Or run the full lint pipeline (build + swiftlint + header.sh + periphery):
./Scripts/lint.sh

MistDemo Commands

# MistDemo is located in Examples/MistDemo and must be run from there
cd Examples/MistDemo

# Build MistDemo
swift build

# Run MistDemo commands
swift run mistdemo --help
swift run mistdemo auth-token
swift run mistdemo current-user
swift run mistdemo query
swift run mistdemo lookup
swift run mistdemo create
swift run mistdemo update
swift run mistdemo modify
swift run mistdemo delete
swift run mistdemo upload-asset
swift run mistdemo lookup-zones
swift run mistdemo fetch-changes
swift run mistdemo demo-in-filter
swift run mistdemo demo-errors
swift run mistdemo test-integration
swift run mistdemo test-private

# Run with specific configuration
swift run mistdemo --config-file ~/.mistdemo/config.json query

Architecture Considerations

FieldValue Type Architecture

MistKit uses separate types for requests and responses at the OpenAPI schema level to accurately model CloudKit's asymmetric API behavior:

Type Layers:

  1. Domain Layer: FieldValue enum - Pure Swift types, no API metadata (Sources/MistKit/FieldValue.swift)
  2. API Request Layer: FieldValueRequest - No type field, CloudKit infers type from value structure
  3. API Response Layer: FieldValueResponse - Optional type field for explicit type information

Why Separate Request/Response Types?

  • CloudKit API has asymmetric behavior: requests omit type field, responses may include it
  • OpenAPI schema accurately models this asymmetry (openapi.yaml:867-920)
  • Swift code generation produces type-safe request/response types
  • Compiler prevents accidentally using response types in requests
  • Cleaner architecture without nil type values in conversion code

Generated Types:

  • Components.Schemas.FieldValueRequest - Used for modify, create, filter operations
  • Components.Schemas.FieldValueResponse - Used for query, lookup, changes responses
  • Components.Schemas.RecordRequest - Records in request bodies
  • Components.Schemas.RecordResponse - Records in response bodies

Conversion:

  • Request conversion: Extensions/OpenAPI/Components+FieldValue.swift converts domain FieldValueFieldValueRequest
  • Response conversion: Service/FieldValue+Components.swift converts FieldValueResponse → domain FieldValue

Modern Swift Features to Utilize

  • Swift Concurrency (async/await) for all network operations
  • Structured concurrency with TaskGroup for parallel operations
  • Actors for thread-safe state management
  • Result builders for query construction
  • Property wrappers for CloudKit field mapping

Package Structure

MistKit/
├── Sources/
│   └── MistKit/
│       ├── Generated/      # Auto-generated OpenAPI client code (not committed)
│       └── MistKitClient.swift  # Main client wrapper
├── Tests/
│   └── MistKitTests/
├── Scripts/
│   └── generate-openapi.sh # Script to generate OpenAPI code
├── openapi.yaml           # CloudKit Web Services OpenAPI specification
└── openapi-generator-config.yaml # Configuration for code generation

CloudKitService Operations

CloudKitService operations are split across focused extension files:

File Operations
CloudKitService+Initialization.swift initializer overloads (API token, web auth token, server-to-server)
CloudKitService+Operations.swift queryRecords, queryAllRecords, lookupRecords
CloudKitService+WriteOperations.swift modifyRecords, createRecord, updateRecord, deleteRecord
CloudKitService+ZoneOperations.swift listZones, lookupZones(zoneIDs:), fetchZoneChanges(syncToken:)
CloudKitService+SyncOperations.swift fetchRecordChanges(recordType:syncToken:), fetchAllRecordChanges(recordType:syncToken:)
CloudKitService+UserOperations.swift fetchCaller(), discoverUserIdentities(lookupInfos:), discoverAllUserIdentities() (unavailable — pending #28), lookupUsersByEmail(_:), lookupUsersByRecordName(_:), fetchCurrentUser() (deprecated, forwards to fetchCaller)
CloudKitService+AssetOperations.swift uploadAssets, requestAssetUploadURL
CloudKitService+AssetUpload.swift uploadAssetData
CloudKitService+RecordManaging.swift record-managing convenience surface
CloudKitService+Classification.swift operation classification (batch sync result tracking)
CloudKitService+ErrorHandling.swift error mapping helpers

Sync/Change Operations:

  • fetchRecordChanges(recordType:syncToken:)/records/changes — returns RecordChangesResult with records, syncToken, moreComing
  • fetchAllRecordChanges(recordType:syncToken:) — convenience wrapper that auto-paginates using moreComing
  • fetchZoneChanges(syncToken:)/zones/changes — returns ZoneChangesResult
  • lookupZones(zoneIDs:)/zones/lookup — returns [ZoneInfo]
  • discoverUserIdentities(lookupInfos:) → POST /users/discover — takes [UserIdentityLookupInfo], returns [UserIdentity]

User-Identity Operations (public DB + web-auth required):

  • fetchCaller()/users/caller — returns UserInfo. Replaces deprecated fetchCurrentUser() / users/current. Only valid against the public database with web-auth credentials.
  • discoverAllUserIdentities() → GET /users/discover — returns [UserIdentity] for every discoverable user in the caller's address book.
  • lookupUsersByEmail(_:) → POST /users/lookup/email — returns [UserIdentity].
  • lookupUsersByRecordName(_:) → POST /users/lookup/id — returns [UserIdentity].

In MistDemo, integration runs targeting these endpoints use PhaseContext.userContextService (a public+web-auth CloudKitService) which is built from CLOUDKIT_API_TOKEN + CLOUDKIT_WEB_AUTH_TOKEN regardless of the primary --database selection. The DatabaseConfiguration / AuthenticationCredentials types in Examples/MistDemo/Sources/MistDemoKit/Configuration/ enforce valid database+auth combinations at construction time.

Result Types (Sources/MistKit/Service/):

  • QueryResultrecords: [RecordInfo], continuationMarker: String?
  • RecordChangesResultrecords: [RecordInfo], syncToken: String?, moreComing: Bool
  • ZoneChangesResultzones: [ZoneInfo], syncToken: String?, moreComing: Bool
  • UserIdentityuserRecordName: String?, nameComponents: NameComponents?, lookupInfo: UserIdentityLookupInfo?
  • UserIdentityLookupInfoemailAddress: String?, phoneNumber: String?, userRecordName: String?
  • NameComponents — full personal name parts (givenName, familyName, nickname, etc.)

Protocols:

  • RecordTypeIterating (Sources/MistKit/Protocols/RecordTypeIterating.swift) — forEach(_ action:) to iterate over CloudKit record types; used by fetchAllRecordChanges

Key Design Principles

  1. Protocol-Oriented: Define protocols for all major components (TokenManager, NetworkClient, etc.)
  2. Dependency Injection: Use initializer injection for testability
  3. Error Handling: Use typed errors conforming to LocalizedError
  4. Sendable Compliance: Ensure all types are Sendable for concurrency safety

Logging

MistKit uses swift-log for cross-platform logging support, enabling usage on macOS, Linux, Windows, and other platforms.

Key Logging Components:

  • MistKitLogger - Centralized logging infrastructure with subsystems for api, auth, and network
  • Environment-based privacy control via MISTKIT_DISABLE_LOG_REDACTION environment variable
  • SecureLogging utilities for token masking and safe message formatting
  • Structured logging in LoggingMiddleware for HTTP request/response debugging (DEBUG builds only)

Logging Subsystems:

MistKitLogger.api      // CloudKit API operations
MistKitLogger.auth     // Authentication and token management
MistKitLogger.network  // Network operations

Helper Methods:

MistKitLogger.logError(_:logger:shouldRedact:)    // Error level
MistKitLogger.logWarning(_:logger:shouldRedact:)  // Warning level
MistKitLogger.logInfo(_:logger:shouldRedact:)     // Info level
MistKitLogger.logDebug(_:logger:shouldRedact:)    // Debug level

Privacy Controls:

  • By default, logs use SecureLogging.safeLogMessage() to redact sensitive information
  • Set MISTKIT_DISABLE_LOG_REDACTION=1 to disable redaction for debugging
  • Tokens, keys, and secrets are automatically masked in logged messages

Asset Upload Transport Design

⚠️ CRITICAL WARNING: Transport Separation

When providing a custom AssetUploader implementation:

  • NEVER use the CloudKit API transport (ClientTransport) for asset uploads
  • MUST use a separate URLSession instance, NOT shared with api.apple-cloudkit.com
  • MUST NOT share HTTP/2 connections between CloudKit API and CDN hosts
  • Custom uploaders should ONLY be used for testing or specialized CDN configurations
  • Production code should use the default implementation (URLSession.shared)

Why URLSession instead of ClientTransport?

Asset uploads use URLSession.shared directly rather than the injected ClientTransport to avoid HTTP/2 connection reuse issues:

  1. Problem: CloudKit API (api.apple-cloudkit.com) and CDN (cvws.icloud-content.com) are different hosts
  2. HTTP/2 Issue: Reusing the same HTTP/2 connection for both hosts causes 421 Misdirected Request errors
  3. Solution: Use separate URLSession for CDN uploads, maintaining distinct connection pools

Design:

  • AssetUploader closure type allows dependency injection for testing
  • Default implementation uses URLSession.shared.upload(_:to:) with separate connection pool
  • Tests provide mock uploader closures without network calls
  • Platform-specific: WASI compilation excludes URLSession code via #if !os(WASI)
  • CRITICAL: Custom uploaders must maintain connection pool separation from CloudKit API

Implementation Details:

  • AssetUploader type: (Data, URL) async throws -> (statusCode: Int?, data: Data)
  • Defined in: Sources/MistKit/Core/AssetUploader.swift
  • URLSession extension: Sources/MistKit/Extensions/URLSession+AssetUpload.swift
  • Upload orchestration:
    • uploadAssets() - Complete two-step upload workflow → Sources/MistKit/Service/CloudKitService+AssetOperations.swift
    • requestAssetUploadURL() - Step 1: Get CDN upload URL → Sources/MistKit/Service/CloudKitService+AssetOperations.swift
    • uploadAssetData() - Step 2: Upload binary data to CDN → Sources/MistKit/Service/CloudKitService+AssetUpload.swift

Future Consideration:
A ClientTransport extension could provide a generic upload method, but would need to:

  • Handle connection pooling separately for different hosts
  • Provide platform-specific implementations (URLSession, custom transports)
  • Maintain the same testability via dependency injection

FilterBuilder Extensions

FilterBuilder is split across extension files for maintainability:

  • Sources/MistKit/Helpers/FilterBuilder.swift — core comparators (EQUALS, NOT_EQUALS, LESS_THAN, etc.) and IN/NOT_IN
  • Sources/MistKit/Helpers/FilterBuilder+StringFilters.swift — string-specific: beginsWith, notBeginsWith, containsAllTokens
  • Sources/MistKit/Helpers/FilterBuilder+ListMemberFilters.swift — list-specific: listContains, etc.

IN/NOT_IN serialization: Uses ListValuePayload (Components.Schemas.ListValuePayload) to wrap array values. The fix in v1.0.0-alpha.5 ensures the correct value key structure is used when serializing list comparators.

CloudKit Web Services Integration

  • Base URL: https://api.apple-cloudkit.com
  • Authentication:
    • Public database: CLOUDKIT_KEY_ID + CLOUDKIT_PRIVATE_KEY or CLOUDKIT_PRIVATE_KEY_PATH → server-to-server signing
    • Private database: CLOUDKIT_API_TOKEN + CLOUDKIT_WEB_AUTH_TOKEN → web authentication
  • All operations should reference the OpenAPI spec in openapi.yaml
  • URL Pattern: /database/{version}/{container}/{environment}/{database}/{operation}
  • Supported databases: public, private, shared
  • Environments: development, production

Testing Strategy

  • Use Swift Testing framework (@Test macro) for all tests
  • Unit tests for all public APIs
  • Integration tests using mock URLSession
  • Use #expect() and #require() for assertions
  • Async test support with async let and await
  • Parameterized tests for testing multiple scenarios
  • See testing-enablinganddisabling.md for Swift Testing patterns

Asset Upload Testing

Integration Test Requirements:

  • Verify connection pool separation between CloudKit API and CDN
  • Test HTTP/2 connection reuse prevention
  • Validate 421 Misdirected Request error handling
  • Mock uploaders should simulate realistic HTTP responses

Test Files:

  • Tests/MistKitTests/Service/CloudKitServiceUpload/CloudKitServiceTests.Upload+*.swift
  • Tests/MistKitTests/Service/AssetUploadTokenTests.swift

MistDemo Integration Test Runner

Examples/MistDemo/Sources/MistDemoKit/Integration/ provides a live end-to-end test suite that runs against a real CloudKit container:

  • IntegrationTestRunner.swift — orchestrates all operations (query, create, update, lookup, upload, fetchChanges, lookupZones, discoverUserIdentities)
  • IntegrationTestData.swift — seed data for integration tests
  • IntegrationTestError.swift — typed errors for test failures
  • IntegrationTest.swift, PhasedIntegrationTest.swift, and Tests/ subdirectory — protocol-based phase pipeline introduced in Refactor IntegrationTestRunner into protocol-based phase pipeline (#254) #283

Run via swift run mistdemo test-integration or swift run mistdemo test-private (private database variant). Both commands require valid CloudKit credentials in the config file.

Important Implementation Notes

  1. Async/Await First: All network operations should use async/await, not completion handlers
  2. Codable Compliance: All models should be Codable with custom CodingKeys when needed
  3. CloudKit Types: Map CloudKit types (Asset, Reference, Location) to Swift types appropriately
  4. Error Context: Include request/response details in error types for debugging
  5. Pagination: Implement AsyncSequence for paginated results (queries, list operations)

OpenAPI-Driven Development

The Swift package uses Apple's swift-openapi-generator to create type-safe client code from the OpenAPI specification. Generated code is placed in Sources/MistKit/Generated/ and should not be committed to version control.

IMPORTANT: Never manually edit files in Sources/MistKit/Generated/. These files are auto-generated from openapi.yaml. Any manual edits will be lost when code is regenerated. Instead, modify openapi.yaml and regenerate using ./Scripts/generate-openapi.sh.

The openapi.yaml file serves as the source of truth for:

  • All available endpoints and their HTTP methods
  • Request/response schemas and models
  • Authentication requirements
  • Error response formats

Key endpoints documented in the OpenAPI spec:

  • Records: /records/query, /records/modify, /records/lookup, /records/changes
  • Zones: /zones/list, /zones/lookup, /zones/modify, /zones/changes
  • Subscriptions: /subscriptions/list, /subscriptions/lookup, /subscriptions/modify
  • Users: /users/caller, /users/discover (POST + GET), /users/lookup/email, /users/lookup/id
  • Assets: /assets/upload
  • Tokens: /tokens/create, /tokens/register

Reference Documentation

Apple's official CloudKit documentation is available in .claude/docs/ for offline reference during development:

When to Consult Each Document

webservices.md (289 KB) - CloudKit Web Services REST API

  • Primary use: Implementing REST API endpoints
  • Contains: Authentication, request formats, all endpoints, data types, error codes
  • Consult when: Writing API client code, handling authentication, debugging responses

cloudkitjs.md (188 KB) - CloudKit JS Framework

  • Primary use: Understanding CloudKit concepts and operation flows
  • Contains: Container/database patterns, operations, response objects, error handling
  • Consult when: Designing Swift types, implementing queries, working with subscriptions

testing-enablinganddisabling.md (126 KB) - Swift Testing Framework

  • Primary use: Writing modern Swift tests
  • Contains: @Test macros, async testing, parameterization, migration from XCTest
  • Consult when: Writing or organizing tests, testing async code

swift-openapi-generator.md (235 KB) - Swift OpenAPI Generator Documentation

  • Primary use: Understanding code generation configuration and features
  • Contains: Generator configuration, type overrides, middleware system, transport protocols, API stability
  • Consult when: Configuring openapi-generator-config.yaml, implementing middleware, troubleshooting generated code

See .claude/docs/README.md for detailed topic breakdowns and integration guidance.

MistDemo Documentation

  • Swift Configuration Reference (.claude/docs/mistdemo/swift-configuration-reference.md) - Guide for using Swift Configuration in MistDemo
  • Official Swift Configuration Docs (.claude/docs/https_-swiftpackageindex.com-apple-swift-configuration-1.0.0-documentation-configuration.md) - Full API reference

CloudKit Schema Language

cloudkit-schema-reference.md - CloudKit Schema Language Quick Reference

  • Primary use: Working with text-based .ckdb schema files
  • Contains: Complete grammar, field options, data types, permissions, common patterns, MistKit-specific notes
  • Consult when: Reading/modifying schemas, understanding indexing, designing record types

sosumi-cloudkit-schema-source.md - Apple's Official Schema Language Documentation

  • Primary use: Authoritative reference for CloudKit Schema Language
  • Contains: Full grammar specification, identifier rules, system fields, permission model
  • Consult when: Understanding schema fundamentals, resolving syntax questions

Comprehensive Schema Guides

For detailed schema workflows and integration:

  • AI Schema Workflow (Examples/CelestraCloud/.claude/AI_SCHEMA_WORKFLOW.md) - Comprehensive guide for understanding, designing, modifying, and validating CloudKit schemas with text-based tools
  • Quick Reference (Examples/SCHEMA_QUICK_REFERENCE.md) - One-page cheat sheet with syntax, patterns, cktool commands, and troubleshooting

Additional Notes

@claude
Copy link
Copy Markdown

claude Bot commented May 12, 2026

Review: PR #332 — Extract auth-token server scaffolding + resource bundle

Overall this is a solid refactor. The architecture is cleaner and testability is improved.

What's good

  • Separation of concerns: AuthTokenCommand now delegates to AuthTokenServer; route assembly and loopback validation are independently testable
  • internal import: Explicitly scoping module imports prevents leaking dependencies through the public API
  • Resource bundle: Moving the raw-string HTML/JS constants into Resources/auth-token-index.html removes ~500 LOC of awkward Swift string escaping and makes the HTML editable with proper tooling
  • Test coverage: 26-case parameterized loopback test + 4 router smoke tests is thorough

Minor observations

LoopbackAuthority on non-Hummingbird platforms: The utility lives in Sources/MistDemoKit/Utilities/ outside a #if canImport(Hummingbird) guard. Verify this compiles cleanly on WASI/platforms where Hummingbird is unavailable, or add a compile guard if it is only ever used by Hummingbird-gated code.

AuthTokenIndexHTML.content error path: Using preconditionFailure for a missing bundle resource is fine for a demo tool, but a comment clarifying the intentional crash-vs-silent-failure choice would help future readers.

Import access in test files: The new test files import without internal qualifiers, inconsistent with the source files in this PR. Non-critical since tests are module-internal, but consistency is nice.

These are all minor — the refactoring is well-executed.


Note: there is also a stale comment above this one that was posted by mistake — it can be ignored.

@claude
Copy link
Copy Markdown

claude Bot commented May 12, 2026

Review: PR #332 — Extract auth-token server scaffolding + resource bundle

Summary: Phase 1 of #330. Two issues resolved (#288, #289). Routes extracted from AuthTokenCommand into WebServer, loopback validation into standalone LoopbackAuthority, and the auth-flow HTML/JS moved from raw Swift strings into a Bundle.module-loaded resource file. 907 tests pass, lint clean.


What's well done

Architecture

  • Clean separation of concerns: AuthTokenCommand is now lifecycle-only, WebServer owns routing, LoopbackAuthority owns host validation. Exactly the right decomposition.
  • WebAuthTokenStore as an actor with an AsyncStream continuation is a solid pattern — thread-safe, one-shot consumers can await iterator.next(), and deinit finishes the continuation correctly.
  • WebBackend protocol is narrow and testable. MockBackend in tests eliminates the live CloudKit dependency entirely.
  • WebServer.runOperation catching all backend errors and serializing them as {"message":"..."} JSON keeps the browser UI from parsing raw HTTP errors — good defensive design for a demo.
  • nonisolated let tokenUpdates on WebAuthTokenStore is the right way to expose the stream without requiring await at the call site.

Testing

  • 26-case parameterized LoopbackAuthorityTests — the rejection matrix covers 10.0.0.1, 192.168.x.x, 0.0.0.0, IPv6 non-loopback, and subdomain-bypass attempts (e.g. localhost.evil.com). Very thorough.
  • WebServerTests / WebServerTests+CRUD use HummingbirdTesting to exercise the full router without a live port — the right approach.
  • The intermittent timeout test is now wrapped in withKnownIssue(isIntermittent: true) and disabled on wasm32 — pragmatic fix for cooperative-executor flakiness.

Resource bundle

  • Moving four concatenated raw-string constants into a single Resources/index.html is clearly better. The iOS CodeSign concern was correctly investigated and shown to be moot (MistDemoApp has no dependency on MistDemoKit).

Issues / suggestions

1. HummingbirdTesting added to the executable target (Package.swift)

HummingbirdTesting is a test-only framework — bundling it into the MistDemo production executable is unexpected. It bloats the binary and may cause issues if Hummingbird's testing utilities make assumptions about the process environment. If WebServerTests needs it, it should live in the test target's dependency list, not the executable's. Worth confirming this is in the right target.

2. WebBackend hard-codes .private database

All four CloudKitService: WebBackend methods hard-code database: .private. Fine for the current demo scope, but a short comment explaining the intentional constraint would prevent future confusion when someone tries to adapt it for the public database.

3. LoopbackOnlyMiddleware — absent Host header is blocked

request.head.authority ?? "" calls LoopbackAuthority.isLoopback("") when the header is absent, correctly returning false and yielding 403. Right security behavior, but worth a code comment so future maintainers don't "fix" the ?? "" to something permissive.

4. host(in:)?? authority fallback is dead code

let host = authority.split(separator: ":", maxSplits: 1).first
return host.map(String.init) ?? authority

The only time first is nil is when authority == "", making ?? authority return "" anyway. The fallback is unreachable in practice. Minor nit — could guard on empty input explicitly instead.

5. CloudKitClientConfig.environment is String

The struct stores the enum's string representation. Make sure the conversion from MistKit.Environment to its raw value is explicit at construction time so it doesn't silently break if the enum's raw value ever changes.

6. runOperation exposes localizedDescription to the browser

For a loopback demo this is acceptable and useful for debugging. Worth a note in the README security section if this surface ever expands beyond loopback.


Summary

The refactoring is clean, well-tested, and directionally correct. The main item worth resolving before merge is issue #1 (HummingbirdTesting in the executable target). The rest are minor observations or documentation suggestions. High-quality work overall.

Generated with Claude Code

@claude claude Bot mentioned this pull request May 12, 2026
leogdion and others added 2 commits May 12, 2026 18:36
…d/Modified (#336)

* Rework mistdemo web UI to table+form; sortable Created/Modified

Iterates on top of #329's CloudKit JS mode toggle. The single-mode JSON-
textarea CRUD grid is replaced by a Notes table beside a Title/Index
form: clicking a row loads it for edit, per-row Delete buttons, "New"
to clear. Auto-refreshes after every mutation and after mode switches,
so the same notes can be observed fetched through either backend.

WebUI:
- Two-column responsive layout: Notes table left, edit/create form
  right; stacks to one column below 820px.
- Created and Modified columns formatted with the locale's
  dateStyle:short/timeStyle:short (e.g., "5/12/26, 4:30 PM"); full
  ISO is in each cell's tooltip.
- Clickable Created/Modified column headers cycle unsorted →
  ascending → descending. Sort forwards to both backends:
  MistKit body `sortBy:[{field, ascending}]`, CloudKit JS
  `sortBy:[{fieldName, ascending}]`. Default is no sort, so the demo
  still lists records before the new schema deploys.
- Record name is removed as a column and surfaced as a row tooltip.

Note schema:
- Drop custom `createdAt` (TIMESTAMP) and `modified` (INT64) — they
  duplicated CloudKit's system metadata. CKRecord.creationDate /
  .modificationDate and the Web Services `created.timestamp` /
  `modified.timestamp` cover the same information without manual
  bookkeeping. Schema, native Note model, RecordDetailView,
  QueryView, NativeCloudKitService, integration phases, README,
  and the CLI query examples are updated.
- Add `___createTime` and `___modTime` to the schema with QUERYABLE
  SORTABLE so the sort feature actually works against the live
  container (system fields default to non-sortable; the schema must
  explicitly opt them in).

Server:
- New WebJSON.encoder()/.decoder() with .millisecondsSince1970 date
  strategy. The browser receives created/modified timestamps as plain
  epoch-millis numbers, matching CloudKit JS's Date shape.
- WebRequests.Update + .Delete grow optional `recordChangeTag` — the
  browser holds it from the last query, so MistKit-mode update/delete
  no longer need a server-side fetch round-trip. Fixes CloudKit's
  `BadRequestException: missing required field 'recordChangeTag'`.
- WebRequests.Create + .Update.fields are `[String: FieldValue]`
  decoded through MistKit's FieldValue Codable (which accepts raw
  JSON primitives — string/int/double). Fixes the 400 thrown when
  the form sent `"index": 5` (a JSON number) against the prior
  `[String: String]` type.
- WebRequests.Query gains `sortBy: [QuerySortField]?`; WebBackend
  takes the request-shape sort directly (no MistKit-internal type
  leakage). CloudKitService extension is the only site that knows
  about MistKit's QuerySort.

WebCommand:
- Catch AsyncTimeoutError.cancelled so Ctrl+C is a normal shutdown
  rather than a top-level fatal error.

Tests:
- WebServerTests+QuerySort — sort forwarding + nil default.
- Updated CRUD tests cover recordChangeTag forwarding (update +
  delete), mixed-type fields (int + double in create), and absent
  recordChangeTag tolerance on update.
- MockBackend.QueryCall.sortBy captures the request-shape sort;
  flatten() handles int64/double for assertion.

The new SORTABLE system fields in schema.ckdb need to be deployed
to the live CloudKit container before sort works end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Address PR #336 review: route table-Delete status; tidy comments/tests

- index.html: deleteNote(note, statusEl = tableStatusEl) so per-row
  Delete feedback lands above the table; the form panel's Delete
  passes formStatusEl explicitly. Removes a dead recordType fallback
  reachable only if normalizeRecords() dropped recordType — which it
  doesn't. Adds a comment that per-row Delete intentionally skips
  confirm(); the raw response panel makes accidents visible.
- WebJSON.swift: removes unused decoder() (FieldValue's own Codable
  handles request-side dates; Hummingbird's request.decode runs the
  framework decoder). Encoder remains as the singular response-side
  contract; docstring updated accordingly.
- WebRequests.swift: doc comment on QuerySortField.field flagging
  that CloudKit JS calls the same concept `fieldName` and the browser
  maps between them.
- MockBackend.swift: comment on flatten()'s default case explaining
  the intentional drop of asset/date/reference/location/list/bytes —
  tests needing those should inspect the FieldValue directly.
- WebJSONTests.swift (new): locks the encoder's epoch-millis contract
  with a round-trip test. The browser's `toDate(value)` in index.html
  depends on receiving plain millis numbers.

Skipped from the review with reasoning preserved in the plan file
(/Users/leo/.claude/plans/async-wibbling-pearl.md):
  - server-side recordChangeTag guard (CloudKit's 400 is already
    clear, browser is the only realistic caller)
  - generic CancellationError catch in WebCommand (the explicit
    AsyncTimeoutError.cancelled is more meaningful)
  - sort response with timestamps test (encoder round-trip covers
    the same contract more directly)
  - schema.ckdb trailing blank line (was already present pre-PR)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Resolve #275 (web side): public/private database picker

Adds a database picker to the long-running `mistdemo web` demo so the
mode toggle's four profiles can be exercised side-by-side:

  - MistKit + private  → API token + browser-captured web-auth token
  - MistKit + public   → server-to-server signing (CLOUDKIT_KEY_ID +
                          CLOUDKIT_PRIVATE_KEY[_PATH])
  - CloudKit JS + private → API token + web-auth (shared with MistKit)
  - CloudKit JS + public  → API token only (browser → CloudKit directly)

WebConfig now accepts optional key-id / private-key inputs and computes
`publicDatabaseAvailable`. WebBackendFactory.live builds the
CloudKitService from `Credentials` so a single service can route
operations to either database based on the request's `database` field.
The `/api/config` endpoint advertises `publicDatabaseAvailable` so the
UI disables the "MistKit + Public" option when the server isn't holding
S2S credentials. CloudKit-JS-mode requests pick
`container.publicCloudDatabase` vs `privateCloudDatabase` based on the
toggle. Unknown database values return 400 rather than silently
defaulting.

The app side of #275 is intentionally not addressed here; it's absorbed
by #328 (replace NativeCloudKitService with the CloudKit framework).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* WebUI: loading states, post-create delay, "You" badge, default sort

Four UX papercuts surfaced in first hands-on use of the new
public/private database picker:

- Switching databases triggered a query with no feedback while the
  network round-trip was in flight. queryNotes now sets a `.status.loading`
  message ("Loading <db> via <mode>") and disables refresh/db/mode/save/
  delete buttons for the duration; a finally block re-enables them and
  re-runs refreshDatabasePicker so the public-availability gate wins.

- Auto-refresh after Create raced CloudKit's eventual consistency on
  Public — the new record was often missing. saveNote now pauses
  REFRESH_DELAY_MS (1.2s) on Create only, with a visible
  "waiting for CloudKit to settle" status. Update/Delete still refresh
  immediately.

- No way to tell which records the signed-in user owned (most useful on
  Public). handleAuthentication now stores userIdentity.userRecordName
  in currentUserRecordName; normalizeRecords projects createdBy from
  the MistKit-mode `created.userRecordName` envelope; renderRows
  appends a green "You" badge when they match. Cleared on sign-out.
  CloudKit-JS mode doesn't surface the creator on records, so the badge
  is MistKit-only — refreshDatabasePicker now spells that out as a hint
  in CloudKit-JS mode.

- Default sort was "whatever CloudKit returned." currentSort now
  initializes to ___createTime descending, and refreshSortIndicators
  fires once during init so the column arrow renders before the first
  query.

Coverage: WebServerTests+Index gains "Index HTML carries the
post-database-picker UX additions" asserting all four new strings
(.status.loading, REFRESH_DELAY_MS, currentUserRecordName/badge-you,
the default-sort initializer). 931 tests pass; swiftlint clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* WebUI: rework database-picker hint after MistKit/CloudKit-JS identity audit

Two corrections to the hint that appears under the database toggle:

- Drop the special CloudKit-JS branch that claimed CloudKit JS doesn't
  surface the record creator. A real-session diagnostic showed the
  opposite: CloudKit JS returns `created.userRecordName` in the same
  envelope shape MistKit does (alongside `timestamp` and `deviceID`),
  and `normalizeRecords` was already picking it up correctly.

- Add a new MistKit + Public hint explaining the iCloud-vs-S2S identity
  split: records you write via that path are owned by the S2S key's
  effective identity, not your iCloud user, so they intentionally don't
  carry the "You" badge. Tracked for broader follow-up in #338.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@claude
Copy link
Copy Markdown

claude Bot commented May 13, 2026

Code Review PR 332 - Resolve 288 289: extract auth-token server scaffolding and resource bundle. Overview: Extracts the embedded HTTP server logic from AuthTokenCommand into a reusable WebServer struct, introduces WebCommand for a long-running interactive CRUD demo, and moves the HTML page into a resource bundle. Two commands now share one server implementation distinguished by the terminatesAfterAuth flag. Architecture Strengths: Clean separation of concerns across WebServer, WebBackend protocol, and WebBackendFactory. WebAuthTokenStore is correctly an actor preventing data races. LoopbackOnlyMiddleware is scoped to the /api router group so the HTML index stays accessible while the sensitive API surface is restricted to loopback. terminatesAfterAuth elegantly drives the 205 vs 204 behavior. Suggestions: (1) WebIndexHTML.loadContent() uses preconditionFailure - a descriptive message hinting at the resource target would help contributors who hit this during development. (2) All CRUD routes use POST even for delete and update - worth a brief comment so future contributors do not accidentally fix it and break the client JS. (3) runOperation maps all errors to 500 - CloudKit typed errors would benefit from more specific status codes in a future pass. (4) The 300-second timeout in captureToken is hardcoded with no user-visible progress feedback - a log line at start would improve UX. (5) MockBackend.stubRecord uses string interpolation for JSON which is fragile with special characters in record names. Security: LoopbackAuthority correctly handles IPv6 bracketed form, trailing ports, and empty authorities. API token is only served behind the loopback guard. Session token in WebAuthTokenStore has no expiry which is acceptable for a local demo but worth noting in the help text. Test Coverage: Coverage is solid. Gap: LoopbackOnlyMiddleware is only tested in isolation via LoopbackAuthorityTests - a routing-level test that sends a request with a non-loopback Host header and expects 403 would close this. Minor nits: addIndexEndpoint captures indexBytes via an extra closure layer that is not necessary since ByteBuffer is Sendable. WebRequests.decodeDatabase is fileprivate on an enum where private would be equally correct. Summary: Clean architecture, good test coverage, follows project conventions. The WebAuthTokenStore actor plus AsyncStream pattern for token capture is particularly clean. Three items worth addressing before leaving DRAFT: (1) routing-level test for LoopbackOnlyMiddleware rejecting non-loopback requests, (2) user-visible log message for the auth timeout countdown, (3) comment on the POST-for-all-verbs decision.

leogdion and others added 2 commits May 14, 2026 08:37
…) (#339)

* Resolve #328: MistDemoApp CloudKit refresh (CKRecord-first, @observable, public/private picker)

- Rename `NativeCloudKitService`/`Error` to `CloudKitStore`/`Error` — the
  app target no longer depends on MistKit, so the "Native" disambiguator
  is dead weight; "Store" reads as the SwiftUI source-of-truth idiom.
- `Note` wraps `CKRecord` instead of copying fields out of it. Update is
  now "mutate the held record, save" — no extra fetch round-trip to
  refresh the change tag.
- `@Observable` + `@MainActor` on `CloudKitStore`; views use
  `@Environment(CloudKitStore.self)` and `@Bindable` for the picker.
  App entry switches to `@State` + `.environment(_:)`.
- Public/private database picker in `AccountView`; `QueryView` and
  `ZoneListView` re-fetch on `.onChange(of: store.databaseScope)` and
  show the active scope in their navigation title.
- Drop web-auth-token UI (`AccountView+Actions.swift`, related state)
  and the `CLOUDKIT_API_TOKEN` scheme env var — the native app authenticates
  via the signed-in iCloud user.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* [CodeFactor] Apply fixes

* Gate WebBackendFactory on canImport(Hummingbird) to fix wasm build

WebBackendFactory.live calls CloudKitService's URLSession-defaulted
convenience init, which is gated behind #if !os(WASI). The rest of
the Server/ folder is already wrapped in #if canImport(Hummingbird);
this file was missed. Wrapping it the same way unblocks the wasm,
wasm 6.2, and wasm-embedded CI jobs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Address PR #339 review: roll back Note CKRecord wrapper, restore web-auth-token UI

Two review comments from #pullrequestreview-4286058024:

1. Note: revert from CKRecord wrapper back to value-struct (id/title/index/
   imageAssetURL + system metadata). updateNote now fetches by ID before
   apply+save instead of mutating the original record in place; deleteNote
   reconstructs CKRecord.ID from the recordName. Views switch from
   note.recordName to note.id.

2. AccountView: restore the API-token TextField, "Fetch Web Auth Token"
   button, copyable token display, and CLOUDKIT_API_TOKEN env-var seed,
   ported from the pre-#328 NativeCloudKitService design onto the new
   @observable CloudKitStore + @Environment binding. Database picker
   stays. Adds CloudKitStore.fetchWebAuthToken via
   CKFetchWebAuthTokenOperation and a webAuthTokenUnavailable error case.
   Recreates AccountView+Actions.swift (deleted in #328).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Resolve #338: per-call PublicAuthPreference encoded in Database (#340)

* Address PR #339 review: use CKDatabase.Scope, fix web-auth-token routing + scheme env

- Replace CloudKitStore.DatabaseScope with CKDatabase.Scope; new
  CKDatabaseScope+Demo.swift extension provides the demo-scoped
  selectable list ([.public, .private]) and label.
- Route CKFetchWebAuthTokenOperation through container.privateCloudDatabase
  unconditionally; the operation is documented to require the private
  database and was previously running against the user-selected scope.
- Migrate fetchWebAuthTokenCompletionBlock -> fetchWebAuthTokenResultBlock
  (the completion-block API is deprecated in macOS 12+); drop the now-
  unreachable webAuthTokenUnavailable error case.
- Bake CLOUDKIT_API_TOKEN into the macOS + iOS scheme run actions so
  xcodegen substitutes the .env value AccountView already reads from
  ProcessInfo at launch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Mark CloudKitStore.fetchWebAuthToken nonisolated to fix CK callback crash

The continuation body inherited @mainactor isolation from CloudKitStore,
which tripped a dispatch_assert_queue assertion on com.apple.cloudkit.callback
when CKFetchWebAuthTokenOperation's result block fired — crashing with
EXC_BREAKPOINT in _dispatch_assert_queue_fail on macOS 26.5. Marking the
bridge nonisolated lets the operation enqueue + callback dispatch run off
the main actor.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Add owner "You" badge and newest-first sort to native MistDemo

Mirrors the web demo: track the signed-in user's record name via
CKContainer.userRecordID, capture each note's creator from
CKRecord.creatorUserRecordID, and tag matching rows in QueryView.
Also sorts Notes by creationDate desc with modificationDate desc as
the tiebreaker, matching the web demo's default ordering.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: codefactor-io <support@codefactor.io>
@leogdion leogdion marked this pull request as ready for review May 14, 2026 14:37
@leogdion leogdion changed the title Resolve #288, #289: extract auth-token server scaffolding + resource bundle Resolve #330: interactive MistDemo (web toggle + native app refresh) May 14, 2026
@leogdion leogdion added this to the v1.0.0-beta.1 milestone May 14, 2026
@leogdion leogdion merged commit d65d20b into v1.0.0-beta.1 May 14, 2026
73 checks passed
@leogdion leogdion deleted the 330-interactive-mistdemo branch May 14, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant