diff --git a/docs/README.md b/docs/README.md index 13c208bf..d8ba84e8 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,3 +1,4 @@ # Documentation - **[cloudkit-guide/](cloudkit-guide/)** — Content reference, talk prep, and marketing materials for the server-side CloudKit speaking series +- **[why-mistkit.md](why-mistkit.md)** — Use-case catalog of server-side CloudKit patterns (public database, private database, web app bridge, data aggregation) diff --git a/docs/cloudkit-guide/README.md b/docs/cloudkit-guide/README.md index 88931d24..1e3480fb 100644 --- a/docs/cloudkit-guide/README.md +++ b/docs/cloudkit-guide/README.md @@ -1,4 +1,38 @@ -# MistKit Content & Talks +# CloudKit as Your Backend: From iOS to Server-Side Swift + +## Presentation Description from Swift Craft 2026 + +> CloudKit is great for iOS apps. How about backend services? I rebuilt a production CloudKit library and learned the patterns Apple doesn't document: three auth methods, type safety, error handling. Real deployments. Learn the whys and hows of using CloudKit on the backend. +> CloudKit has excellent documentation for iOS and macOS client development. But backend services—podcast aggregation, RSS readers, data processing—face APIs that Apple barely documents. I rebuilt a comprehensive CloudKit library using AI-generated OpenAPI specifications. The result: type-safe Swift code supporting three authentication methods (server-to-server, web authentication token, and API token), typed error handling for 9 HTTP status codes, and production deployments. + +> This talk fills the gaps with real production patterns: + +> Three Authentication Methods: - Server-to-Server: Autonomous services (podcast aggregation, cron jobs) - Web Authentication Token: User operations from backend (on behalf of signed-in users) - API Token: Development and debugging (CloudKit Dashboard) + +> Each method includes key generation, request signing, token handling, and failure recovery that Apple's documentation glosses over. You'll learn when to use each method and how to implement them with a unified ClientMiddleware pattern. + +> Type System Challenges: Solving CloudKit's dynamically-typed fields in Swift's statically-typed system with discriminated unions and type-safe record builders. + +> Production Error Handling: CloudKit returns 9+ HTTP status codes. Implementing typed error hierarchies, retry logic for transient failures, conflict resolution for concurrent modifications. + +> When to Use CloudKit: Decision framework comparing CloudKit vs. Firebase vs. custom backends with real production examples. + +> Drawing from production deployments (podcast backend, RSS sync service), attendees at all experience levels learn authentication patterns, type safety, error handling, and informed backend decisions. No prior CloudKit server-side experience required. + +--- + +## Core Narrative & Hook + +**Opening hook** (works for talks, videos, threads): +> "Raise your hand if you've used CloudKit from an iOS app. Keep it up if you've used CloudKit from a backend service. Yeah, that's the problem." + +**The problem**: CloudKit server-side is Apple's worst-documented feature. 2016-era docs. Auth barely explained. No error handling examples. Type system challenges unaddressed. Stack Overflow full of unanswered questions. + +**The solution story**: Built two production backends (BushelCloud + CelestraCloud) that required solving all of this. Then rebuilt MistKit from scratch using AI-generated OpenAPI specs to give others the patterns Apple didn't document. + +**Key insight on AI**: AI excels at documentation→OpenAPI spec translation. Human expertise required for architecture, error patterns, and API design. + +--- Educational content, reference material, and talk prep for an ongoing series about [MistKit](../README.md) and server-side CloudKit — covering what Apple's documentation leaves out. @@ -13,22 +47,61 @@ CloudKit Web Services is a REST API that works on any platform: server-side Swif --- -## Core Narrative & Hook +## Outline -**Opening hook** (works for talks, videos, threads): -> "Raise your hand if you've used CloudKit from an iOS app. Keep it up if you've used CloudKit from a backend service. Yeah, that's the problem." +### Why CloudKit -**The problem**: CloudKit server-side is Apple's worst-documented feature. 2016-era docs. Auth barely explained. No error handling examples. Type system challenges unaddressed. Stack Overflow full of unanswered questions. +#### iOS App 101 -**The solution story**: Built two production backends (BushelCloud + CelestraCloud) that required solving all of this. Then rebuilt MistKit from scratch using AI-generated OpenAPI specs to give others the patterns Apple didn't document. +#### CloudKit on the Server -**Key insight on AI**: AI excels at documentation→OpenAPI spec translation. Human expertise required for architecture, error patterns, and API design. +##### Why CloudKit on the Server + +* Web Application +* Background Job + +**Production Examples**: + +| App | Purpose | Auth | Real Challenges | +|---|---|---|---| +| BushelCloud | Syncs macOS/Swift/Xcode version data for Bushel VM | Server-to-server | Concurrent updates from multiple version sources | +| CelestraCloud | Syncs RSS feeds for Celestra RSS reader | Server-to-server | 15-min polling, aggressive rate limiting, conflict resolution | + +**Stats for credibility**: +- **MistKit**: actively maintained open-source library — see the [repo](../../) for current stats +- Built using AI-assisted OpenAPI generation — significantly faster than manual implementation +- **BushelCloud** and **CelestraCloud** are production deployments, each requiring substantial schema migrations + +**When to Use CloudKit**: + +Use CloudKit when: +- Building backend for an iOS/macOS app +- Data sync for indie/small team +- Zero server management preferred +- Already in the Apple ecosystem + +Consider alternatives when: +- Android support needed → Firebase +- Complex relational queries → PostgreSQL/Supabase +- Real-time updates → Firebase +- Full backend control → Vapor/Hummingbird + +**Reality check**: CloudKit's "free" tier has limits. Rate limiting (429) is real at scale. Factor in discovery time for undocumented auth patterns. --- -## Key Technical Topics +##### Understanding CloudKit -### 1. Three Authentication Methods +| Theme | What It Covers | +|---|---| +| **Server-to-Server Auth** | Key pair generation, ECDSA request signing, credential lifecycle, what Apple's docs omit | +| **Type Safety** | CloudKit's dynamic fields vs. Swift's static types — discriminated unions, OpenAPI `oneOf` | +| **Error Handling** | 9 HTTP status codes, retry logic, exponential backoff, conflict resolution | +| **API Ergonomics** | Three-layer architecture: generated OpenAPI → abstraction → user-facing Swift API *(see Integrating MistKit)* | + +###### Authentication + +**Three Authentication Methods**: | Method | Use Case | Status | |---|---|---| @@ -50,7 +123,7 @@ CloudKit Web Services is a REST API that works on any platform: server-side Swif --- -### 2. Type System Polymorphism +###### Data Types **The problem**: CloudKit fields are runtime-dynamic JSON. Swift is statically typed. Mismatch. @@ -84,7 +157,7 @@ Custom type overrides in `openapi-generator-config.yaml` improve ergonomics. Com --- -### 3. Production Error Handling +###### Error Codes CloudKit returns 9 HTTP status codes, each requiring specific handling: @@ -116,7 +189,13 @@ Each error carries nested JSON: `ckErrorCode`, `serverRecord` (on 409), `reason` --- -### 4. API Ergonomics: Three-Layer Architecture +##### Integrating MistKit + +###### Web Application + +###### Background Job + +**Three-Layer Architecture**: **Problem**: OpenAPI-generated code is verbose and low-level. @@ -128,74 +207,19 @@ let request = Operations.SaveRecordsRequest( ) ``` -**Three layers**: - | Layer | Responsibility | |---|---| | **Generated client** (Layer 1) | Auto-generated from OpenAPI spec. Never edit. Low-level REST. | | **MistKit abstraction** (Layer 2) | Auth middleware, retry logic, error handling, response unwrapping, domain type conversion | | **User-facing API** (Layer 3) | Swift-native, intuitive, feels like native CloudKit framework | -**After all three layers**: +After all three layers: ```swift try await database.save(record) // 5 lines, type-safe, production-ready ``` --- -## Production Examples - -| App | Purpose | Auth | Real Challenges | -|---|---|---|---| -| BushelCloud | Syncs macOS/Swift/Xcode version data for Bushel VM | Server-to-server | Concurrent updates from multiple version sources | -| CelestraCloud | Syncs RSS feeds for Celestra RSS reader | Server-to-server | 15-min polling, aggressive rate limiting, conflict resolution | - -**Stats for credibility**: -- **MistKit**: actively maintained open-source library — see the [repo](../../) for current stats -- Built using AI-assisted OpenAPI generation — significantly faster than manual implementation -- **BushelCloud** and **CelestraCloud** are production deployments, each requiring substantial schema migrations - ---- - -## Learning Outcomes - -Audience leaves able to: -1. Implement server-to-server CloudKit auth — key pairs, request signing, environment switching -2. Design type-safe APIs for CloudKit's dynamic fields using OpenAPI discriminated unions -3. Handle all 9 CloudKit HTTP status codes with appropriate retry logic -4. Build the three-layer architecture to make generated code feel Swift-native -5. Decide when CloudKit is the right backend vs. Vapor, Firebase, or Supabase - ---- - -## When to Use CloudKit - -**Use CloudKit when**: -- Building backend for an iOS/macOS app -- Data sync for indie/small team -- Zero server management preferred -- Already in the Apple ecosystem - -**Consider alternatives when**: -- Android support needed → Firebase -- Complex relational queries → PostgreSQL/Supabase -- Real-time updates → Firebase -- Full backend control → Vapor/Hummingbird - -**Reality check**: CloudKit's "free" tier has limits. Rate limiting (429) is real at scale. Factor in discovery time for undocumented auth patterns. - ---- - -## Memorable Phrases - -- *"Apple's worst-documented feature"* — server-to-server authentication -- *"The patterns Apple's documentation doesn't cover"* -- *"AI excels at docs→spec translation; humans needed for architecture"* -- *"Compiler catches type errors, not runtime surprises"* -- *"Zero overlap, complete coverage"* — useful when pairing with a client-side CloudKit talk - ---- - ## Talk Structure Five acts, scalable to any length. @@ -356,6 +380,27 @@ Execute a working query: request signed → response decoded → type-safe field --- +## Memorable Phrases + +- *"Apple's worst-documented feature"* — server-to-server authentication +- *"The patterns Apple's documentation doesn't cover"* +- *"AI excels at docs→spec translation; humans needed for architecture"* +- *"Compiler catches type errors, not runtime surprises"* +- *"Zero overlap, complete coverage"* — useful when pairing with a client-side CloudKit talk + +--- + +## Learning Outcomes + +Audience leaves able to: +1. Implement server-to-server CloudKit auth — key pairs, request signing, environment switching +2. Design type-safe APIs for CloudKit's dynamic fields using OpenAPI discriminated unions +3. Handle all 9 CloudKit HTTP status codes with appropriate retry logic +4. Build the three-layer architecture to make generated code feel Swift-native +5. Decide when CloudKit is the right backend vs. Vapor, Firebase, or Supabase + +--- + ## In This Directory ``` diff --git a/docs/cloudkit-guide/articles/authenticating-cloudkit-backend-services.md b/docs/cloudkit-guide/articles/authenticating-cloudkit-backend-services.md new file mode 100644 index 00000000..92df0f45 --- /dev/null +++ b/docs/cloudkit-guide/articles/authenticating-cloudkit-backend-services.md @@ -0,0 +1,230 @@ +--- +title: Beyond the MistKit Tutorials: Authenticating CloudKit from Backend Services +date: 2026-01-01 00:00 +description: [FILL IN: 1-2 sentence description covering the three auth methods and who this is for] +featuredImage: /media/tutorials/[FILL IN: path to hero image] +subscriptionCTA: [FILL IN: CTA tied to article topic] +--- + + + +[FILL IN: Opening hook — what frustration or friction does a developer hit first when trying to connect a backend service to CloudKit? What question does this article answer that Apple's docs don't?] + +--- + +**In this series:** + +* [Rebuilding MistKit with Claude Code (Part 1)](/tutorials/rebuilding-mistkit-claude-code-part-1/) +* [Rebuilding MistKit with Claude Code (Part 2)](/tutorials/rebuilding-mistkit-claude-code-part-2/) +* _Beyond the MistKit Tutorials: Authenticating CloudKit from Backend Services_ + +--- + +- [Why CloudKit Auth is Different on the Backend](#why-cloudkit-auth-is-different) +- [Method 1: API Token](#method-1-api-token) +- [Method 2: Web Auth Token](#method-2-web-auth-token) + - [Via Browser Redirect (Web Apps)](#getting-web-auth-token-browser) + - [Via iOS App (CKFetchWebAuthTokenOperation)](#getting-web-auth-token-from-ios) +- [Method 3: Server-to-Server (ECDSA)](#method-3-server-to-server) +- [Choosing the Right Method](#choosing-the-right-method) +- [Configuring MistKit](#configuring-mistkit) +- [Production Considerations](#production-considerations) + + +## Why CloudKit Auth is Different on the Backend + +[FILL IN: Explain why this isn't obvious — on-device CloudKit auth is handled transparently by the framework. On the backend, the developer must explicitly manage credentials. Mention that Apple's docs assume a browser/JS context in several places, which adds confusion.] + +[QUESTION: Do you want to mention the asymmetry here — that public database uses server-to-server while private database requires web auth token? This is the single most counterintuitive thing for new users.] + +CloudKit's REST API offers three distinct authentication methods: + +| Method | Database | Use Case | +|--------|----------|----------| +| API Token | Public (limited) | Prerequisite for Web Auth Token; limited standalone access to public data | +| Web Auth Token | Private | Access a specific user's private database (paired with API Token) | +| Server-to-Server | Public | Backend services, daemons, and CLI tools writing to the public database | + + +## Method 1: API Token + + + +An API Token identifies your CloudKit container but grants limited access on its own. Its primary role in backend auth is as a required companion to the Web Auth Token — without an API Token, you can't initiate the web auth flow at all. + +### Creating an API Token in CloudKit Dashboard + +[FILL IN: Step-by-step — where in the CloudKit Dashboard UI the user goes to create a token, what options/scopes to select, and where to copy the token value from] + +[QUESTION: Is there anything non-obvious about token naming, expiry, or scope that burned you or a user?] + +### Limitations + +An API Token alone cannot access the private database. To read or write a user's private data from a backend service, you must pair it with a Web Auth Token obtained from the user's iCloud session. + + +## Method 2: Web Auth Token + + + +[QUESTION: Is Web Auth Token actually applicable to your backend CLI use case, or is it primarily for web apps with a browser? If the latter, note that clearly upfront so readers don't waste time on it.] + + +### Via Browser Redirect (Web Apps) + +#### The Auth Flow + +[FILL IN: Walk through the redirect-based sign-in flow: +1. App/web page requests sign-in URL from CloudKit +2. User is redirected to Apple sign-in +3. Apple redirects back with a ckWebAuthToken +4. App stores the token for subsequent API calls] + +#### The `AUTHENTICATION_REQUIRED` Response + +[FILL IN: Explain the `redirectURL` field in error responses — when CloudKit returns 401 with `AUTHENTICATION_REQUIRED`, the `redirectURL` is where you send the user. This is the main integration point.] + +```swift +// FILL IN: Show what the error response looks like and how MistKit surfaces it +``` + +#### Pairing with the API Token + +[FILL IN: Clarify that both `ckAPIToken` and `ckWebAuthToken` are required together — the API token identifies the container, the web auth token identifies the user] + + +### Via iOS App (CKFetchWebAuthTokenOperation) + + + +When your backend needs to access a user's private CloudKit database, the token doesn't come from a browser redirect — it comes from the iOS app itself. The app uses `CKFetchWebAuthTokenOperation` to obtain a short-lived token from the CloudKit framework (which already has the user's iCloud session), then sends it to your server. + +The flow looks like this: + +1. **iOS app** calls `CKFetchWebAuthTokenOperation` with your API token +2. **CloudKit framework** exchanges it for a `ckWebAuthToken` tied to the signed-in iCloud account +3. **iOS app** sends the token to your backend (over your own API) +4. **Backend** uses MistKit with both the API token and the received web auth token to read/write the user's private database + +```swift +// FILL IN: Show the iOS-side CKFetchWebAuthTokenOperation call — +// instantiate with the API token, set the fetchWebAuthTokenCompletionBlock, +// add to CKContainer.default().privateCloudDatabase, and send the resulting +// webAuthToken string to your server +``` + +[FILL IN: Note the token's lifetime — how long is it valid? Does it need to be refreshed, and if so how? Does CloudKit return a new one on each call or cache it?] + +[QUESTION: In your experience with Celestra or Bushel, did you use this iOS → backend token handoff pattern, or did you only use server-to-server? If you haven't used this pattern, note that it's the intended path for user-specific private DB access from a server.] + +[QUESTION: Is the web auth token scoped to a specific container, or is it usable across containers? This affects whether you need one token per container.] + + +## Method 3: Server-to-Server (ECDSA) + + + +Server-to-server authentication uses ECDSA P-256 signing to authenticate as your server rather than as a user. This is the method for daemons, CLI tools, and scheduled jobs that write to the public database. + +### Setting Up in CloudKit Dashboard + +[FILL IN: Step-by-step: +1. Navigate to the correct section in CloudKit Dashboard (API Access? Server-to-Server Keys?) +2. Generate the key pair — does Apple generate it, or do you upload your own public key? +3. Download the private key file (.pem format?) +4. Copy the Key ID shown in the Dashboard] + +[QUESTION: Is the private key generated by Apple and downloaded once, or do you generate it yourself and upload the public key? This matters a lot for key management.] + +### What Gets Signed + +[FILL IN: Describe the signing payload — the exact string that gets signed, which typically includes: +- HTTP method +- Request path +- ISO 8601 timestamp +- Body hash (SHA-256?) +Explain why this prevents replay attacks] + +### The Authorization Header Format + +[FILL IN: Show the exact format of the Authorization header value that CloudKit expects] + +``` +Authorization: [FILL IN: exact header format] +``` + +### Key File Management + +[FILL IN: How you store the private key — file path vs environment variable containing the key contents. Reference `CLOUDKIT_PRIVATE_KEY_PATH` vs `CLOUDKIT_PRIVATE_KEY`.] + +[QUESTION: What's the recommended approach for production — file on disk, env var with PEM contents, or secrets manager? What do you actually use for Celestra/Bushel?] + + +## Choosing the Right Method + +[FILL IN: Decision guide. A simple flowchart or set of questions: +- "Do you need to access the private database?" → Web Auth Token +- "Are you running a server daemon or CLI?" → Server-to-Server +- "Do you just need read access to the public database?" → API Token may be sufficient] + +[QUESTION: Is there a case where someone would use API Token alone for a backend service, or should you always use server-to-server if you're writing to the public DB?] + + +## Configuring MistKit + + + +### The `TokenManager` Protocol + +[FILL IN: Brief explanation of the protocol — MistKit's abstraction that accepts credentials and produces the right auth headers at runtime] + +### API Token Configuration + +```swift +// FILL IN: Show how to initialize MistKit with an API token only +``` + +### Web Auth Token Configuration + +```swift +// FILL IN: Show how to initialize MistKit with both API token and web auth token +``` + +### Server-to-Server Configuration + +```swift +// FILL IN: Show how to initialize MistKit with key ID and private key (both file path and inline variants) +``` + +### Reading Credentials from the Environment + +[FILL IN: Show the MistDemo pattern for reading `CLOUDKIT_KEY_ID`, `CLOUDKIT_PRIVATE_KEY`, `CLOUDKIT_PRIVATE_KEY_PATH` from environment variables — this is what the CLI tools do] + +```swift +// FILL IN: Environment variable reading example from MistDemo +``` + + +## Production Considerations + +### Key Rotation _(Server-to-Server)_ + +[FILL IN: How/when to rotate server-to-server keys. Is there a key expiry? What's the process in the Dashboard?] + +[QUESTION: Have you dealt with key rotation in Celestra or Bushel? Any gotchas?] + +### Securing Credentials in CI/CD _(Server-to-Server)_ + +[FILL IN: Brief guidance on not committing keys, using secret managers, passing as env vars to cloud functions / GitHub Actions / etc.] + +### Local Development vs Production + +[FILL IN: How to use the `development` environment with your credentials during development, switch to `production` for release. This applies to all three authentication methods.] + +[QUESTION: Do development and production use the same set of keys, or do you need separate credentials per environment?] + +--- + +[FILL IN: Closing — what the reader can now do, pointer to MistDemo examples in the repo as working reference implementations] + +📚 **[View Documentation](https://swiftpackageindex.com/brightdigit/MistKit/documentation)** | 🐙 **[GitHub Repository](https://github.com/brightdigit/MistKit)** diff --git a/docs/cloudkit-guide/articles/rebuilding-mistkit-claude-code-part-1.md b/docs/cloudkit-guide/articles/rebuilding-mistkit-claude-code-part-1.md new file mode 100644 index 00000000..03cb6b26 --- /dev/null +++ b/docs/cloudkit-guide/articles/rebuilding-mistkit-claude-code-part-1.md @@ -0,0 +1,535 @@ +--- +title: Rebuilding MistKit with Claude Code - From CloudKit Docs to Type-Safe Swift (Part 1) +date: 2025-12-01 00:00 +description: Follow the journey of rebuilding MistKit using Claude Code and swift-openapi-generator. Learn how OpenAPI specifications transformed Apple's CloudKit documentation into a type-safe Swift client, and discover the challenges of mapping CloudKit's quirky REST API to modern Swift patterns. +featuredImage: /media/tutorials/rebuilding-mistkit-claude-code/mistkit-rebuild-part1-hero.webp +subscriptionCTA: Want to learn more about AI-assisted Swift development? Sign up for our newsletter to get notified when Part 2 drops. +--- + +In my previous article about [Building SyntaxKit with AI](https://brightdigit.com/tutorials/syntaxkit-swift-code-generation/), I explored how with the help of [Claude Code](https://claude.ai/claude-code) I could transform SwiftSyntax's 80+ lines of verbose API calls into 10 lines of elegant, declarative Swift. + +I saw how Claude Code could easily replace and understand patterns. That's when I decided to explore the idea of updating [MistKit](https://github.com/brightdigit/MistKit), my library for server-side CloudKit application and see how Claude Code can help. + +--- + +**In this series:** + +* [Building SyntaxKit with AI](/tutorials/syntaxkit-swift-code-generation/) +* _Rebuilding MistKit with Claude Code (Part 1)_ +* [Rebuilding MistKit with Claude Code (Part 2)](/tutorials/rebuilding-mistkit-claude-code-part-2/) + +--- + +📚 **[View Documentation](https://swiftpackageindex.com/brightdigit/MistKit/documentation)** | 🐙 **[GitHub Repository](https://github.com/brightdigit/MistKit)** + +- [The Decision to Rebuild](#the-decision-to-rebuild) + - [The Game Changer: swift-openapi-generator](#the-game-changer-swift-openapi-generator) + - [Learning from SyntaxKit's Pattern](#learning-from-syntaxkits-pattern) +- [Building with Claude Code](#building-with-claude-code) + - [Why OpenAPI + swift-openapi-generator?](#why-openapi--swift-openapi-generator) + - [Challenge #1: Type System Polymorphism](#challenge-1-type-system-polymorphism) + - [Challenge #2: Authentication Complexity](#challenge-2-authentication-complexity) + - [Challenge #3: Error Handling](#challenge-3-error-handling) + - [Challenge #4: API Ergonomics](#challenge-4-api-ergonomics) + - [The Iterative Workflow with Claude](#the-iterative-workflow-with-claude) +- [What's Next](#whats-next) + + +## The Decision to Rebuild + +I had a couple of use cases where MistKit running in the cloud would allow me to store data in a public database. However I hadn't touched the library in a while. + +By now, [Swift had transformed](https://brightdigit.com/tutorials/swift-6-async-await-actors-fixes/) while MistKit stood still: +- **Swift 6** with strict concurrency checking +- **async/await** as standard (not experimental) +- **Server-side Swift maturity** (Vapor 4, swift-nio, AWS Lambda) +- **Modern patterns** expected (Result types, AsyncSequence, property wrappers) + +MistKit, frozen in 2021, couldn't take advantage of any of this. + +> youtube https://youtu.be/_-k97s1ZPzE + + +### The Game Changer: [swift-openapi-generator](https://github.com/apple/swift-openapi-generator) + +At [WWDC 2023](https://developer.apple.com/videos/play/wwdc2023/10171/), Apple announced [`swift-openapi-generator`](https://github.com/apple/swift-openapi-generator)—a tool that reads OpenAPI specifications and automatically generates type-safe Swift client code. This single tool made the MistKit rebuild feasible. What was missing was an OpenAPI spec. If I had that I could easily create a library which made the necessary calls to CloudKit as needed, as well as compatibility with [server-side (AsyncHTTPClient)](https://github.com/swift-server/swift-openapi-async-http-client) or [client-side (URLSession)](https://github.com/apple/swift-openapi-urlsession) APIs . + +That's where [Claude Code](https://claude.ai/claude-code) came in. + + +### Learning from SyntaxKit's Pattern + +With my work on SyntaxKit, I could see that if I fed sufficient documentation on an API to an LLM, it can understand how to develop against it. There may be issues along the way. However, any failures come with the ability to learn and adapt either with internal documentation or writing sufficient tests. + +Just as I was able to simplify SwiftSyntax into a simpler API with [SyntaxKit](https://github.com/brightdigit/SyntaxKit), I can have an LLM create an OpenAPI spec for CloudKit. + +--- + +The pattern was clear: **give Claude the right context, and it could translate Apple's documentation into a usable OpenAPI spec**. SyntaxKit taught me that code generation works best when you have a clear source of truth—for SyntaxKit it was SwiftSyntax ASTs, for MistKit it would be CloudKit's REST API documentation. The abstraction layer would come later. + +The rebuild was ready to begin. + +![CloudKit Web Services Documentation Site](/media/tutorials/rebuilding-mistkit-claude-code/cloudkit-documentation.webp) + + +## Building with [Claude Code](https://claude.ai/claude-code) + +I needed a way for Claude Code to understand how the CloudKit REST API worked. There was one main document I used—the [CloudKit Web Services Documentation Site](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/). The [CloudKit Web Services Documentation](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/) Site, **which hasn't been updated since June of 2016**, contains the most thorough documentation on how the REST API works and hopefully can provide enough for Claude to start crafting the OpenAPI spec. + +By running the site (as well as the swift-openapi-generator documentation) through llm.codes, saving the exported markdown documentation in the `.claude/docs` directory and letting Claude Code know about it (i.e. add a reference to it in Claude.md), I could now start having Claude Code translate the documentation into a usable API. + +### Setting Up Claude Code for MistKit + +Before diving in, here's what you need to understand about working with Claude Code: + +**Documentation Export with llm.codes** +I used [llm.codes](https://llm.codes) (mentioned in my [SyntaxKit article](https://brightdigit.com/tutorials/syntaxkit-swift-code-generation/)) to convert Apple's web documentation into markdown format that Claude can easily understand. This tool crawls documentation sites and exports them as clean markdown files. It also works with DocC documentation from Swift packages, making it easy to give Claude context about any Swift library's API. + +**Claude Code's Context System** +Claude Code uses a simple but powerful context system: +- `.claude/docs/` - Store reference documentation (like CloudKit API docs, swift-openapi-generator guides) +- `.claude/CLAUDE.md` or `CLAUDE.md` - Reference these docs so Claude knows to use them as context + +This gives Claude the context it needs to understand CloudKit's API without you having to paste documentation repeatedly in every conversation. + +``` +.claude/docs +├── cktool-full.md # Complete CloudKit CLI tool documentation +├── cktool.md # Condensed CloudKit CLI reference +├── cktooljs-full.md # Full CloudKitJS documentation +├── cktooljs.md # CloudKitJS quick reference +├── cloudkit-public-database-architecture.md +├── cloudkit-schema-plan.md +├── cloudkit-schema-reference.md +├── cloudkitjs.md # JavaScript SDK documentation +├── data-sources-api-research.md +├── firmware-wiki.md +├── https_-swiftpackageindex.com-apple-swift-log-main-documentation-logging.md +├── https_-swiftpackageindex.com-apple-swift-openapi-generator-1.10.3-documentation-swift-openapi-generator.md +├── https_-swiftpackageindex.com-brightdigit-SyndiKit-0.6.1-documentation-syndikit.md +├── mobileasset-wiki.md +├── protocol-extraction-continuation.md +├── QUICK_REFERENCE.md +├── README.md +├── schema-design-workflow.md +├── sosumi-cloudkit-schema-source.md +├── SUMMARY.md +├── testing-enablinganddisabling.md +└── webservices.md # Primary CloudKit Web Services REST API documentation +``` + +Note: Files with "-full" suffix contain complete documentation exported from llm.codes, while shorter versions are condensed for quicker reference. The swift-openapi-generator docs were essential for understanding type overrides and middleware configuration. + + +### Why OpenAPI + [swift-openapi-generator](https://github.com/apple/swift-openapi-generator)? + +With [`swift-openapi-generator`](https://github.com/apple/swift-openapi-generator) available (announced WWDC 2023), the path forward became clear: + +1. **Create OpenAPI specification from CloudKit documentation** + - Translate Apple's prose docs → Machine-readable YAML + - Every endpoint, parameter, response type formally defined + +2. **Let swift-openapi-generator generate the client** + - Run `swift build` → 10,476 lines of type-safe networking code appear + - Request/response types (Codable structs) + - API client methods (async/await) + - Type-safe enums, JSON handling, URL building + - Configuration: `openapi-generator-config.yaml` + Swift Package Manager build plugin + +3. **Build clean abstraction layer on top** + - Wrap generated code in friendly, idiomatic Swift API + - Add TokenManager for authentication + - CustomFieldValue for CloudKit's polymorphic types + +By following [spec-driven development](https://brightdigit.com/tutorials/swift-openapi-generator/), we had many benefits: + +- Type safety (if it compiles, it's valid CloudKit usage) +- Completeness (every endpoint defined) +- Maintainability (spec changes = regenerate code) +- No manual JSON parsing or networking boilerplate +- Cross-platform (macOS, iOS, Linux, server-side Swift) + + +### Challenge #1: Type System Polymorphism +[CloudKit fields](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/Types.html#//apple_ref/doc/uid/TP40015240-CH3-SW2) are dynamically typed—one field can be STRING, INT64, DOUBLE, TIMESTAMP, BYTES, REFERENCE, ASSET, LOCATION, or LIST. But [OpenAPI is statically typed](https://spec.openapis.org/oas/latest.html). How do we model this polymorphism? + +```no-highlight +Me: "Here's CloudKit's field value structure from Apple's docs. + A field can have value of type STRING, INT64, DOUBLE, TIMESTAMP, + BYTES, REFERENCE, ASSET, LOCATION, LIST..." + +Claude: "This is a discriminated union. Try modeling with oneOf in OpenAPI: + The value property can be oneOf the different types, + and the type field acts as a discriminator." + +Me: "Good start, but there's a CloudKit quirk: ASSETID is different + from ASSET. ASSET has full metadata, ASSETID is just a reference." + +Claude: "Interesting! You'll need a type override in the generator config: + typeOverrides: + schemas: + FieldValue: CustomFieldValue + Then implement CustomFieldValue to handle ASSETID specially." + +Me: "Perfect. Can you generate test cases for all field types?" + +Claude: "Here are test cases for STRING, INT64, DOUBLE, TIMESTAMP, + BYTES, REFERENCE, ASSET, ASSETID, LOCATION, and LIST..." +``` + +Having developed MistKit previously, I understood the challenge of various field types and the difficulty in expressing that in Swift. This is a common challenge in Swift with JSON data. + +Claude's suggestion of [`typeOverrides`](https://swiftpackageindex.com/apple/swift-openapi-generator/1.10.3/documentation/swift-openapi-generator/configuring-the-generator#Type-overrides) was the breakthrough—instead of fighting OpenAPI's type system, we'd let the generator create basic types, then override with our custom implementation that handles CloudKit's quirks. + +#### Understanding ASSET vs ASSETID + +CloudKit uses two different type discriminators for [asset fields](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/Types.html#//apple_ref/doc/uid/TP40015240-CH3-SW2): + +**[ASSET](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/Types.html#//apple_ref/doc/uid/TP40015240-CH3-SW2)** - Full asset metadata returned by CloudKit +- Appears in: Query responses, lookup responses, modification responses +- Contains: `fileChecksum`, `size`, `downloadURL`, `wrappingKey`, `receipt` +- Use case: When you need to download or verify the asset file + +**[ASSETID](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/Types.html#//apple_ref/doc/uid/TP40015240-CH3-SW2)** - Asset reference placeholder +- Appears in: Record creation/update requests +- Contains: Same structure as ASSET, but typically only `downloadURL` populated +- Use case: When you're referencing an already-uploaded asset + +At the end of the day, both decode to the same `AssetValue` structure, but CloudKit distinguishes them with different type strings (`"ASSET"` vs `"ASSETID"`). Our custom implementation handles this elegantly: + +```swift +internal struct CustomFieldValue: Codable, Hashable, Sendable { + internal enum FieldTypePayload: String, Codable, Sendable { + case asset = "ASSET" + case assetid = "ASSETID" // Both decode to AssetValue + case string = "STRING" + case int64 = "INT64" + // ... more types + } + + internal let value: CustomFieldValuePayload + internal let type: FieldTypePayload? +} +``` + +Using the `CustomFieldValue` with the power of openapi-generator `typeOverides` allows us to implement the specific quirks of CloudKit field values. + + +### Challenge #2: Authentication Complexity + +The next challenge was dealing with the 3 different methods of authentication: + +1. **API Token** - Container-level access + - Query parameter: `ckAPIToken` + - Simplest method + - A starting point for **Web Auth Token** + +2. **[Web Auth Token](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/SettingUpWebServices.html#//apple_ref/doc/uid/TP40015240-CH24-SW2)** - User-specific access + - Two query parameters: `ckAPIToken` + `ckWebAuthToken` + - For private database access + +3. **[Server-to-Server](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/SettingUpWebServices.html#//apple_ref/doc/uid/TP40015240-CH24-SW6)** - Public Database Access + - ECDSA P-256 signature in Authorization header + - Most complex, most secure + + +This became a complexity problem when trying to model it in OpenAPI. What Claude suggested was to use the [ClientMiddleware API](https://swiftpackageindex.com/apple/swift-openapi-runtime/1.8.3/documentation/openapiruntime/clientmiddleware) to handle authentication dynamically rather than relying on generator's built-in auth. The meant we used: + +1. **OpenAPI**: Define all three `securitySchemes` but make endpoint security optional (`security: []`) +2. **Middleware**: Implement `AuthenticationMiddleware` that inspects `TokenManager` at runtime +3. **TokenManager Protocol**: Three implementations (API, WebAuth, ServerToServer) +4. **Runtime Selection**: Client chooses auth method via TokenManager injection + +Here's what it looks like: + +```swift +internal struct AuthenticationMiddleware: ClientMiddleware { + internal let tokenManager: any TokenManager + + func intercept(...) async throws -> (HTTPResponse, HTTPBody?) { + guard let credentials = try await tokenManager.getCurrentCredentials() else { + throw TokenManagerError.invalidCredentials(.noCredentialsAvailable) + } + + var modifiedRequest = request + + switch credentials.method { + case .apiToken(let token): + // Add ckAPIToken to query parameters + case .webAuthToken(let apiToken, let webToken): + // Add both tokens to query parameters + case .serverToServer(let keyID, let privateKey): + // Sign request with ECDSA + // Add Authorization header with signature + } + + return try await next(modifiedRequest, body, baseURL) + } +} +``` + +This helps because: + +- ✅ Generator doesn't need to handle auth complexity +- ✅ We control authentication at runtime +- ✅ Easy to test (inject mock TokenManager) +- ✅ Supports all three methods seamlessly +- ✅ Can switch auth methods without code changes + +While this looks like it will work I wanted to make absolute sure. When using an LLM **it's important to have code which actually proves that it works and not just unit test.** So I created a simple command line tool which would to both write and read from a public and private database: + +```no-highlight +Me: "Can you run MistDemo to test the actual functionality?" + +Claude: *[Runs demo successfully]* +"Perfect! The MistDemo ran successfully: + - Web Authentication - Used API token + web auth token + - CloudKit API Calls - Made multiple successful requests + - HTTP Integration - AuthenticationMiddleware properly added all required parameters + - Data Retrieval - Successfully retrieved TodoItem records from CloudKit + + The authentication layer is working as designed. All three authentication methods have a solid foundation!" +``` + +We'll touch more on this approach in [the next blog post.](/tutorials/rebuilding-mistkit-claude-code-part-2/) + + +### Challenge #3: Error Handling + +[CloudKit returns over 9 different HTTP status codes](https://developer.apple.com/library/archive/documentation/DataManagement/Conceptual/CloudKitWebServicesReference/ErrorCodes.html), each with nested error details including `serverErrorCode`, `reason`, `uuid`, and sometimes `redirectURL` or `retryAfter`. What would be nice is if we can parse these in a Swift-y way, taking advantage of Swift 6 features like typed throws for more precise error handling. + +According to Apple's Documentation: + +> **Record Fetch Error Dictionary** +> +> The error dictionary describing a failed operation with the following keys: + + - `recordName`: The name of the record that the operation failed on. + - `reason`: A string indicating the reason for the error. + - `serverErrorCode`: A string containing the code for the error that occurred. For possible values, see Error Codes. + - `retryAfter`: The suggested time to wait before trying this operation again. + - `uuid`: A unique identifier for this error. + - `redirectURL`: A redirect URL for the user to securely sign in. + +Based on this, I had Claude create an openapi entry on this: + +```yaml +components: + schemas: + ErrorResponse: + type: object + description: Error response object + properties: + uuid: + type: string + description: Unique error identifier for support + serverErrorCode: + type: string + enum: + - ACCESS_DENIED + - ATOMIC_ERROR + - AUTHENTICATION_FAILED + - AUTHENTICATION_REQUIRED + - BAD_REQUEST + - CONFLICT + - EXISTS + - INTERNAL_ERROR + - NOT_FOUND + - QUOTA_EXCEEDED + - THROTTLED + - TRY_AGAIN_LATER + - VALIDATING_REFERENCE_ERROR + - ZONE_NOT_FOUND + reason: + type: string + redirectURL: + type: string + + responses: + BadRequest: + description: Bad request (400) - BAD_REQUEST, ATOMIC_ERROR + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + + Unauthorized: + description: Unauthorized (401) - AUTHENTICATION_FAILED + content: + application/json: + schema: + $ref: '#/components/schemas/ErrorResponse' + + # ... additional error responses for 403, 404, 409, 412, 413, 421, 429, 500, 503 +``` + +Claude was able to translate the documentation into: + +1. **Error Code Enum**: Converted prose list of error codes to explicit enum +2. **HTTP Status Mapping**: Created reusable response components for each HTTP status +3. **Consistent Schema**: All errors use same `ErrorResponse` schema +4. **Status Documentation**: Linked HTTP statuses to CloudKit error codes in descriptions + +This enables: +- **Type-Safe Error Handling**: Generated code includes all possible error codes +- **Automatic Deserialization**: Errors automatically parsed to correct type +- **Centralized Definitions**: Define once, reference everywhere + +Here's how it's mapped: + +| HTTP Status | CloudKit Error Codes | Client Action | +|-------------|---------------------|---------------| +| **400 Bad Request** | `BAD_REQUEST`, `ATOMIC_ERROR` | Fix request parameters or retry non-atomically | +| **401 Unauthorized** | `AUTHENTICATION_FAILED` | Re-authenticate or check credentials | +| **403 Forbidden** | `ACCESS_DENIED` | User lacks permissions | +| **404 Not Found** | `NOT_FOUND`, `ZONE_NOT_FOUND` | Verify resource exists | +| **409 Conflict** | `CONFLICT`, `EXISTS` | Fetch latest version and retry, or use force operations | +| **412 Precondition Failed** | `VALIDATING_REFERENCE_ERROR` | Referenced record doesn't exist | +| **413 Request Too Large** | `QUOTA_EXCEEDED` | Reduce request size or upgrade quota | +| **429 Too Many Requests** | `THROTTLED` | Implement exponential backoff | +| **500 Internal Error** | `INTERNAL_ERROR` | Retry with backoff | +| **503 Service Unavailable** | `TRY_AGAIN_LATER` | Temporary issue, retry later | + +This structured [error handling](https://brightdigit.com/articles/swift-error-handling/) enables the generated client to provide specific, actionable error messages rather than generic HTTP failures. Developers get type-safe error codes, HTTP status mapping, and clear guidance on how to handle each error condition. + + +### Challenge #4: API Ergonomics + +The generated OpenAPI client works, but it's not exactly ergonomic. Here's what a simple query looks like with the raw generated code: + +```swift +// Verbose generated API +let input = Operations.queryRecords.Input( + path: .init( + version: "1", + container: "iCloud.com.example.MyApp", + environment: Components.Parameters.environment.production, + database: Components.Parameters.database._private + ), + headers: .init(accept: [.json]), + body: .json(.init( + query: .init(recordType: "User") + )) +) + +let response = try await client.queryRecords(input) + +switch response { +case .ok(let okResponse): + let queryResponse = try okResponse.body.json + // Process records... +default: + // Handle errors... +} +``` + +The problem is there's too much boilerplate for simple operations when we can clean this up with a nicer abstraction. The solution was to build a three-layer architecture that keeps the generated code internal and exposes a clean public API: + +Three-layer architecture showing User Code (public API), MistKit Abstraction (internal), and Generated OpenAPI Client (internal) + +So now it can look something like this: + +```swift +// Clean, idiomatic Swift +let service = try CloudKitService( + container: "iCloud.com.example.MyApp", + environment: .production, + database: .private, + tokenManager: tokenManager +) + +let records = try await service.queryRecords( + recordType: "User", + filter: .equals("status", .string("active")) +) + +// Type-safe field access +for record in records { + if let name = record.fields["name"]?.stringValue { + print("User: \(name)") + } +} +``` + +In this case, we create a few abstraction to help: + +- `FieldValue` enum for type-safe field access +- `RecordInfo` struct for read operations +- `QueryFilter` for building queries +- `CloudKitService` wrapper hiding OpenAPI complexity + +This means the generated code stays internal while users interact with the more friendly API. + + +### The Iterative Workflow with Claude + +This process of building and refining was iterative when working with Claude Code: + +1. **I draft the structure** + - Provide CloudKit domain knowledge and desired API + +2. **Claude expands** + - Fills in request/response schemas + - Generates boilerplate for similar endpoints + - Creates consistent patterns + +3. **I review for CloudKit accuracy** + - Check against Apple docs + - Add edge cases and CloudKit quirks + - Refine error responses + - Define integration and unit tests for verification + +4. **Claude validates consistency** + - Catches schema mismatches + - Suggests improvements + +5. **Iterate until complete** + +Let's take for instance, this conversation I had with Claude: + +```no-highlight +Me: "Here's the query endpoint from Apple's docs" + +Claude: *[Creates complete OpenAPI definition]* +"Here's a complete OpenAPI definition with request/response schemas" + +Me: "Add `resultsLimit` validation and `continuationMarker` for pagination" + +Claude: *[Updates definition with pagination support]* +"Updated, and I noticed the `zoneID` should be optional" +``` + +> youtube https://youtu.be/gH3QnVHsUAc + +By providing my own experience with great Swift APIs and Claude's ability at applying patterns, I quickly build a library that's friendly to use. + +#### Building MistKit from Scratch with Claude Code + +With Claude Code, I could easily create an openapi document based on the Apple's documentation. With my guidance and understanding with the REST API and good Swift design, I could guide Claude through issues like: + +* Field Value with the oneOf pattern and handling the ASSETID quirk) +* completed authentication modeling with three security schemes + +This will make it much easier to continue future features with MistKit and enabling me to create some server-side application for my apps. + + +## What's Next + +After three months of collaboration with Claude (**representing significant acceleration over manual development**), I had: +- ✅ 10,476 lines of generated, type-safe Swift code +- ✅ Three authentication methods working seamlessly +- ✅ CustomFieldValue handling CloudKit's polymorphic types +- ✅ Clean public API hiding OpenAPI complexity +- ✅ 161 tests across 47 test files + +The OpenAPI spec was complete. The generated client compiled. The abstraction layer was elegant. Unit tests passed. + +**How Claude Code Accelerated Development:** +- **Documentation Translation**: Converting Apple's prose documentation to a precise OpenAPI spec would have taken weeks manually. Claude handled the bulk of this in days, with me providing CloudKit domain expertise and corrections. +- **Boilerplate Generation**: The 10,476 lines of generated Swift code from swift-openapi-generator saved months of hand-writing networking code, request/response types, and JSON handling. +- **Pattern Application**: Once I established patterns (like `CustomFieldValue` for polymorphic types), Claude consistently applied them across the codebase. +- **Iteration Speed**: When authentication approaches needed refactoring, Claude could update dozens of files in minutes vs. hours of manual editing. + +What would have likely taken 6-12 months of solo development was compressed into 3 months of _side-project_ collaboration, with Claude handling repetitive tasks while I focused on architecture, CloudKit-specific quirks, and real-world testing. + +However I really needed to put it the test in my actual uses. In the next post, I'll talk about find flaws in MistKit by actually consuming my library with help from Claude Code. I'll be building a couple of command line tools for easily uploading data for [Bushel](https://getbushel.app) and a future RSS Reader to the public database. By doing this I'll understand [Claude's limitation, benefits and how to workaround those.](/tutorials/rebuilding-mistkit-claude-code-part-2/) diff --git a/docs/cloudkit-guide/articles/rebuilding-mistkit-claude-code-part-2.md b/docs/cloudkit-guide/articles/rebuilding-mistkit-claude-code-part-2.md new file mode 100644 index 00000000..e00d9878 --- /dev/null +++ b/docs/cloudkit-guide/articles/rebuilding-mistkit-claude-code-part-2.md @@ -0,0 +1,195 @@ +--- +title: Rebuilding MistKit with Claude Code - Real-World Lessons and Collaboration Patterns (Part 2) +date: 2025-12-10 00:00 +description: After building MistKit's type-safe CloudKit client, we put it to the test with real applications. Discover what happened when theory met practice—the unexpected discoveries, hard-earned lessons, and collaboration patterns that emerged from 428 Claude Code sessions over three months. +featuredImage: /media/tutorials/rebuilding-mistkit-claude-code/mistkit-rebuild-part1-hero.webp +subscriptionCTA: Want to learn more about AI-assisted Swift development and modern API design patterns? Sign up for our newsletter to get notified about the rest of the Modern Swift Patterns series and future tutorials on building production-ready Swift applications. +--- + +In [Part 1](https://brightdigit.com/tutorials/rebuilding-mistkit-claude-code-part-1/), I showed how [Claude Code](https://claude.ai/claude-code) and [swift-openapi-generator](https://github.com/apple/swift-openapi-generator) transformed [CloudKit's REST documentation](https://developer.apple.com/documentation/cloudkitjs/cloudkit/cloudkit_web_services) into a type-safe Swift client. We had 161 unit tests which passed, but would it actually work in the real world? + +📚 **[View Documentation](https://swiftpackageindex.com/brightdigit/MistKit/documentation)** | 🐙 **[GitHub Repository](https://github.com/brightdigit/MistKit)** + +- [Real-World Proof](#real-world-proof) + - [The Celestra and Bushel Examples](#the-celestra-and-bushel-examples) + - [Integration Testing Through Real Applications](#integration-testing-through-real-applications) +- [Lessons Learned](#lessons-learned) + - [Unit Test Generation](#unit-test-generation) + - [Human Guided Architecture](#human-guided-architecture) + - [Grabby AI](#grabby-ai) + - [Context Management](#context-management) + - [Human + AI Code Reviews](#human--ai-code-reviews) +- [Multiplier, not a Replacement](#multiplier-not-a-replacement) + + +## Real-World Proof + +Would MistKit's abstractions actually work when building an application? I had 2 real-world applications for MistKit to try it out: + +- an RSS aggregator syncing thousands of articles to CloudKit using [SyndiKit](https://github.com/brightdigit/SyndiKit) for an app codenamed **[Celestra](https://celestr.app)** +- For **[Bushel](https://getbushel.app)**, I wanted to track restore images and various metadata for macOS and developer software versions. + + + +### The Celestra and Bushel Examples + +Tests validate correctness, but real applications validate design. MistKit needed to prove it could power actual software and not just pass unit tests. Enter two real-world applications—**[the Celestra app](https://celestr.app)** (an RSS reader) and **[the Bushel app](https://getbushel.app)** (a macOS virtualization tool)—each powered by MistKit-driven CLI backends that populate CloudKit public databases. These CLI tools, running on scheduled cloud infrastructure, proved MistKit works in production. + +The architecture for both follows the same pattern: +- **Consumer apps** ([the Celestra app](https://celestr.app), [the Bushel app](https://getbushel.app)) - iOS/macOS apps that read from CloudKit +- **CLI tools** - Built with MistKit, run on cloud infrastructure (cron jobs, cloud functions, scheduled tasks) +- **CloudKit public database** - Central data layer connecting CLI tools to apps + +This pattern enables: +- **Automated updates**: CLI tools run on schedules without user devices being online +- **Separation of concerns**: Data population (CLI) vs data consumption (app) +- **Scalability**: Cloud infrastructure handles data aggregation, apps stay lightweight + +#### Celestra: Automated RSS Feed Sync for a Reader App + +The [Celestra app](https://celestr.app) is an RSS reader in development for iOS and macOS. To keep content fresh without requiring the app to be open, I built a [CLI tool with MistKit](https://github.com/brightdigit/MistKit/tree/main/Examples/Celestra) that runs on scheduled cloud infrastructure. The CLI tool runs periodically (cron job, cloud function, scheduled task) to fetch RSS feeds and sync them to CloudKit's public database, making fresh content available to all users instantly—even when their devices are offline. + +This architecture enables push notifications on updated articles without the app running, and MistKit's batch operations can efficiently handle hundreds of content updates. The [CLI tool example](https://github.com/brightdigit/MistKit/tree/main/Examples/Celestra) demonstrates key MistKit patterns: + +**Query filtering** - Find feeds that need updating: +```swift +// Query filtering - find stale feeds +QueryFilter.lessThan("lastAttempted", .date(cutoff)) +QueryFilter.greaterThanOrEquals("usageCount", .int64(minPop)) +``` + +**Batch operations** - Efficiently sync hundreds of articles: +```swift +// Batch operations +let operations = articles.map { article in + RecordOperation.create( + recordType: "Article", + recordName: article.guid, + fields: article.toCloudKitFields() + ) +} +service.modifyRecords(operations, atomic: false) +``` + +#### Bushel: Powering a macOS VM App with CloudKit + +The [Bushel app](https://getbushel.app) is a macOS virtualization tool for developers. It currently allows pluggable _hubs_ to get a list of restore images, their download URLs, and their status. However, since the data is universal, I wanted a comprehensive, queryable central database of macOS restore images and various metadata about operating system versions and developer tools. Therefore I wanted a [CLI tool with MistKit](https://github.com/brightdigit/MistKit/tree/main/Examples/Bushel) that runs on scheduled cloud infrastructure (cron jobs, cloud functions, scheduled tasks) to populate a CloudKit public database with various metadata about macOS versions and their restore images. + +This architecture provides: +- **Public Database**: Worldwide access to version history without embedding static JSON in the app +- **Automated Updates**: CLI tool syncs latest info on restore images, Xcode, and Swift versions +- **Queryable**: [Bushel app](https://getbushel.app) can easily query for restore images such as _macOS 15.2_ +- **Scalable**: CLI tool aggregates data from various sources automatically +- **Deduplication**: buildNumber-based deduplication ensures clean data + +The [CLI tool example](https://github.com/brightdigit/MistKit/tree/main/Examples/Bushel) demonstrates advanced MistKit patterns: + +```swift +// Protocol-based record conversion +protocol CloudKitRecord { + static var cloudKitRecordType: String { get } + func toCloudKitFields() -> [String: FieldValue] +} + +// Relationship handling +fields["minimumMacOS"] = .reference( + FieldValue.Reference(recordName: restoreImageRecordName) +) +``` + +--- + +> transistor https://share.transistor.fm/s/ffcb9fc1 + +Both CLI tool examples serve as copy-paste starting points for new MistKit projects. + + +Watching MistKit power real applications was satisfying—I could see the generated code actually work in production. The CLI tools successfully synced RSS articles (Celestra) and tracked complex version relationships (Bushel). But more importantly, these real-world applications revealed critical issues that no unit test could catch. + + +### Integration Testing Through Real Applications + +Building real applications exposed issues no unit test could catch. Here's what Celestra and Bushel revealed: + +- **Batch Operation Limits**: CloudKit enforces 200-operation-per-request limit (not documented clearly) therefore we added chunking logic. +- **Boolean Field Handling**: CloudKit has no native boolean type but we wanted the developer to safely use Swift Boolean types so we created a safe and easy way to use the standardized INT64 representation. + +I was able to verify that the API design and critical pieces like Server-to-Server authentication were working as intended. These real-world tests validated MistKit's foundation. + + + +## Lessons Learned + +There were few things which surprised me as far as what Claude Code was good and not good at and where [collaboration worked best](https://brightdigit.com/tutorials/syntaxkit-swift-code-generation/). Claude excelled at tasks I thought would be hard (comprehensive test generation). It struggled with things I assumed would be easy (knowing which APIs exist). Through iteration, a collaboration pattern emerged. Here's what I learned. + +### Unit Test Generation + +Test generation proved to be Claude Code's greatest strength, and it's ideal for LLMs because it leverages pattern recognition from vast training data. Claude has seen thousands of Swift test files and can apply those structural patterns to new domains. The task requires extrapolation from examples, not novel reasoning—Claude recognizes "if testing STRING type, also test empty string, Unicode, and nil" because that pattern appears frequently in its training corpus. A typical interaction looked like this: + +```no-highlight +Me: "Generate tests for all CustomFieldValue types" + +Claude: *[Creates tests covering all 10 field types with edge cases]* +"I've generated comprehensive tests covering: + - All 10 field types (STRING, INT64, DOUBLE, TIMESTAMP, BYTES, REFERENCE, ASSET, ASSETID, LOCATION, LIST) + - Edge cases: empty lists, nil values, malformed data + - Encoding/decoding round-trips + - Nested LIST types + - Invalid type/value combinations + + I also added tests for the ASSET vs ASSETID quirk" +``` + +It was able to produce 161 tests across 47 files including edge cases I hadn't considered. The only quirk I found was that it favored [XCTest](https://developer.apple.com/documentation/xctest) over [Swift Testing](https://developer.apple.com/documentation/testing) at first. This makes sense since there's probably more training material in XCTest. I've primarily switched to Swift Testing for my new work. If you are in the same place then be sure to make a note of that in your `CLAUDE.md` when you start your project. + +### Human Guided Architecture + +While Claude excelled at pattern-based tasks, architectural decisions consistently required human judgment. At various points, Claude would steer the architecture in strange directions that didn't seem correct. The issue is that its training is best for smaller contexts and code examples, which isn't enough for holistic system design. Be confident in steering Claude in the right direction—this is where developer expertise matters most. The risk is drift if the pattern isn't perfectly specified, but for well-defined transformations, LLMs excel. Luckily, Claude does a fairly good job at refactoring when corrected, and its context window (200K tokens in Sonnet 4.5) allows it to see multiple files simultaneously and apply consistent transformations across the codebase. + +### Grabby AI + +These limitations manifested in predictable patterns throughout the project. As we were implementing the CLI tools for Bushel and Celestra, Claude would often try to implement features using the direct [OpenAPI](https://www.openapis.org/) code as opposed to the abstracted API we had built: + +```swift +// WRONG: Internal type reference +let operation = Components.Schemas.RecordOperation( + recordType: "RestoreImage", + fields: fields +) +``` + +Even going so far as to make those methods and properties `public`. Often referred to as power-grabbing, it would go outside its designated boundary, even though I would tell it often not to use those APIs. It's important to set those constraints clearly within the context window and review the code intentionally. All mistakes share common traits—Claude follows patterns from training data or generated code literally without questioning ergonomics or existence. The fix is always the same: explicit guidance in prompts and immediate verification of suggestions. + +### Context Management + +Managing these challenges required strategic context management. One of the biggest challenges working with Claude Code is managing its knowledge cutoffs and lack of familiarity with newer or niche APIs. In the world of Swift, Claude's training often predates [Swift Testing](https://developer.apple.com/documentation/testing) or [swift-openapi-generator](https://github.com/apple/swift-openapi-generator) specifics. This is where providing documentation upfront in `.claude/docs/` helps. With tools like [Sosumi.ai](https://sosumi.ai) for Apple API exploration and [llm.codes](https://llm.codes) I can provide documentation like: +- `testing-enablinganddisabling.md` (126KB) - Swift Testing patterns +- `webservices.md` (289KB) - CloudKit Web Services REST API reference +- `cloudkitjs.md` (188KB) - CloudKit operation patterns and data types +- `swift-openapi-generator.md` (235KB) - Code generation configuration + +> youtube https://youtu.be/gH3QnVHsUAc + +At the root of this is the `CLAUDE.md` file which acts as a table of contents, telling Claude where to look for specific information. Claude doesn't need to memorize everything—it needs to know where to look. + +### Human + AI Code Reviews + +Whatever your AI writes should be understood by you fairly well. Don't skip this step. This is especially important in the context of [humane code](https://brightdigit.com/articles/humane-code/)—code that is empathetic to future developers who need to understand and maintain it. AI-generated code still needs to communicate clearly with the humans who will work with it later. + +> transistor https://share.transistor.fm/s/99f236b1 + +These patterns and practices reflect a deeper truth about AI-assisted development: Claude Code is a force multiplier, not a replacement for developer judgment. I provided architectural vision; Claude generated comprehensive implementations. I identified edge cases from domain knowledge; Claude translated them into exhaustive test suites. I steered strategic decisions; Claude handled mechanical transformations at scale. Together, we built something neither could have built alone—a production-ready CloudKit client that balances type safety with developer ergonomics. + + +## Multiplier, not a Replacement + +These lessons crystallized into a philosophy: **AI is a force multiplier, not a replacement**. Claude generated thousands of lines of code, but I architected what those lines should accomplish. It drafted comprehensive tests, but I knew which edge cases mattered. It refactored at scale, but I chose the patterns worth preserving. Where I lacked expertise translating CloudKit's REST API into an OpenAPI spec, Claude filled those gaps. + +The proof came from real-world application. Building **Celestra** and **Bushel** validated MistKit's design beyond what unit tests could achieve. The CLI tools exposed batch operation limits, revealed boolean field handling quirks, and confirmed that Server-to-Server authentication worked in production. These discoveries transformed MistKit from a technically correct library into a production-ready tool. + +Both CLI examples are now open source as starting points for new projects: +- [Bushel CLI Example](https://github.com/brightdigit/MistKit/tree/main/Examples/Bushel) - Demonstrates complex CloudKit relationships and batch operations powering the [Bushel app](https://getbushel.app) +- [Celestra CLI Example](https://github.com/brightdigit/MistKit/tree/main/Examples/Celestra) - Demonstrates public database patterns and automated sync for the [Celestra app](https://celestr.app) + +Through 428 sessions across three months, Claude Code and I built MistKit v1.0 Alpha—a type-safe CloudKit client that proves AI-assisted development can deliver production-quality Swift libraries when human judgment guides the process. + diff --git a/docs/why-mistkit.md b/docs/why-mistkit.md new file mode 100644 index 00000000..96a36a8e --- /dev/null +++ b/docs/why-mistkit.md @@ -0,0 +1,60 @@ +Why do I need MistKit? + +Apple's CloudKit framework only runs on Apple platforms. MistKit wraps the CloudKit Web Services REST API so that server-side Swift, Linux services, and command-line tools can participate in the same CloudKit ecosystem as your iOS and macOS apps. + +### Public Database as a Managed Content Catalog + +The most common pattern: a server-side job manages a CloudKit public database that Apple devices query. + +**macOS VM restore image catalog (Bushel)** +A Mac management tool stores available macOS VM restore images as CloudKit records. A server-side service adds new images and updates metadata as Apple releases them. Client apps query the public database to discover what's available to download — no custom API needed. + +**RSS/feed aggregation (Celestra)** +A server-side service fetches RSS or Atom feeds on a schedule, parses the entries, and writes them as CloudKit records. Apple devices query the public database to get aggregated content, with CloudKit handling sync and delivery. + +**Software version catalogs** +Track available Xcode versions, simulator runtimes, or SDK releases. A server-side job writes structured records into CloudKit; developer tools on-device query the public database to show what's available. + +**App asset distribution** +A creative app (fonts, templates, themes, presets) stores downloadable asset packs as CloudKit records. A server-side tool manages the catalog — adding packs, updating metadata, deprecating old ones — without requiring an app update. + +**Feature flags / remote config** +Store feature flag configurations as CloudKit records. A server-side admin tool writes flag values; Apple devices read them from the public database without needing a dedicated service. + +**MDM configuration catalogs** +Configuration profiles, scripts, or policy templates stored in CloudKit public database. A web-based admin console writes and updates them server-side; managed Macs fetch and apply them. + +### Private Database: Acting on Behalf of a User + +When a user authenticates once and the server stores their web auth token, the server can read and write their CloudKit private database asynchronously — without the user being present. This is the same model as storing OAuth tokens for Gmail or Dropbox access. + +**Wearable / peripheral device linking (Heartwitch)** +A user authenticates in the iOS app, and the server stores their web auth token. The server then connects their Apple Watch data to an external service (e.g., a live streaming platform) continuously — reading records the Watch writes to CloudKit and bridging them to the third-party API without requiring active user interaction each time. + +**Wearable data pipelines** +An app writes activity or sensor data from an Apple Watch or other device to the user's CloudKit private database. A server reads those records and pushes them to a fitness platform, research database, or coaching service. + +**Always-on device presence** +A user's devices write location or status records to their private database. A server monitors those records and triggers actions in another system — fleet tracking, family safety apps, or delivery coordination. + +**Two-way sync with external services** +A server reads a user's CloudKit private records and syncs them to genuinely external platforms (Todoist, Notion, Google Calendar, Obsidian) — and writes changes back — acting as a persistent background sync bridge between CloudKit and the rest of the user's toolchain. + +**Server-side processing** +A user uploads a photo or document to their private database. A server fetches it, runs processing (OCR, image resizing, transcoding, AI tagging), then writes the results back as new records. + +### Web App ↔ Apple Device Bridge + +**Web portal for a CloudKit-backed app** +A user signs into a web app with their Apple ID. The server exchanges that web auth token for CloudKit access and reads/writes the user's private database — giving them a browser-based view of their iOS app data without requiring the app to be open. + +**Webhook → CloudKit writer** +An external service (Stripe, GitHub, a form submission) triggers a webhook. A server-side Swift handler writes the result directly into a CloudKit record, which instantly syncs to the user's Apple devices. + +### Data Aggregation + +**Anonymized telemetry** +Devices write anonymized usage events to CloudKit. A server-side job reads those records via `/records/changes`, aggregates them, and stores results elsewhere for analysis — without building a custom ingestion API. + +**Crowdsourced data** +Apps contribute data points (WiFi maps, accessibility ratings, transit times) to a CloudKit public database. A server aggregates, deduplicates, and enriches the records, then writes cleaned data back — acting as a background data steward.