This repo is no longer maintained

This project is no longer actively maintained, and was never finished. Its README is an aspiration for the end goal of the project rather than an accurate description of its current behavior. I've left it here for posterity and to serve as potential inspiration or raw materials for others who may wish to pursue a similar project.

Mist

Mist combines the best parts of Realm & CloudKit to bring you the simplest possible way to build modern user-data-driven applications.

Persist Records Locally:

• Define your models using real Swift classes
• Store records on device in four lines of code
• Query for records quickly and synchronously
• Keep queried result sets up-to-date automatically (no refetching needed)
• Read and write records from any thread

Keep Local Records Securely Synced with Cloud:

• Keep records synchronized with a secure server with zero extra code (no more parsing JSON!)
• Optionally store your records so securely that even you the developer can't see them (only the user can)

Enjoy Automatic User Management:

• User is automatically logged in at all times (backed by their iCloud user)
• Mist handles iCloud logouts and user switching automatically and transparently
• No account creation, login, logout, password reset, or third party integration code to write

Share Records Effortlessly:

• Just set "share" relationship on Record you want to share and then save; Mist does the rest
• User can share Records with any other Users in their Contacts (no Contacts framework permission required)
• User can share Records with anyone on Earth that can open a link (just share the link provided)
• Current app Users receive invite via push and optional in-app UI, can accept or decline
• New app Users taken to App Store to install, then can accept or decline from app

To start using Mist, jump to Usage, or to learn more about how Mist is implemented, see Mist's Architecture Explained.

Requirements

• iOS 10.0+ / macOS 10.12+ / tvOS 10.0+ / watchOS 3.0+
• Xcode 8.3+
• Swift 3.0+

Communication

• If you find a bug, open an issue.
• If you have a feature request, open an issue.
• If you want to contribute, submit a pull request.

Installation

Before installing and using Mist, ensure that your application is configured to use CloudKit by following Apple's QuickStart instructions.

Cocoapods

1. Install CocoaPods 1.0.0 or later.
2. Run pod repo update to make CocoaPods aware of the latest available version of Mist.
3. In your Podfile, add use_frameworks! and add pod 'Mist' to your main and test targets.
4. Paste the following at the bottom of your Podfile:

post_install do |installer|
  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      config.build_settings['SWIFT_VERSION'] = '3.0'
    end
  end
end

5. From the command line, run pod install.
6. Use the .xcworkspace file generated by CocoaPods to work on your project.

Carthage

Want Carthage installation support? Open a pull request.

Manually

Want manual installation support? Open a pull request.

Embedded Framework

Want embedded framework installation support? Open a pull request.

---

Usage

Configuring AppDelegate

To use Mist, start by configuring your AppDelegate appropriately:

import UIKit
import Mist

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {

    var window: UIWindow?

    func applicationDidBecomeActive(_ application: UIApplication) {
        
	// We don't catch any errors here, but you definitely should!
	// See [doc section] below for more info
        try! Mist.setUp()
	
    }
    
    func application(_ application: UIApplication,
                     didReceiveRemoteNotification userInfo: [AnyHashable : Any],
                     fetchCompletionHandler completionHandler: @escaping (UIBackgroundFetchResult) -> Void) {
        
        do {
	    
	    // Mist will automatically handle notifications sent by CloudKit
	    try Mist.handleNotification(withUserInfo: userInfo)
	    
	} catch {
	
	    // Handle your app's other notifications as appropriate here
	    
	}
        
    }
    
}

Defining App Schema

Next, you'll want to define you app's schema. With Mist, you define your schema using regular Swift classes. Each Record Type you want to store in your app should have its own subclass of the abstract class Record.

Because Mist is backed by Realm, your model classes need to follow all of Realm's rules for model classes.

Let's say we're building a simple Todo app, which we'll call TinyTask. TinyTask lets Users create Todo Lists & Todos. Todo Lists can be private or be shared with other Users. Todos can be assigned to Users, and can have associated Attachments. Here are the model classes we need to create:

Todo

import Mist

class Todo : Record {
    
    
    // MARK: - Properties
    // Per Realm's rules, all properties have to be dynamic vars.
    // They can be optionals; we just don't need them to be here.
    
    dynamic var title: String = ""
    dynamic var description: String = ""
    dynamic var isCompleted: Bool = false
    
    
    // MARK: - Relationships
    
    // Per Realm's rules, to-one relationships must be optional dynamic vars.
    dynamic var todoList: TodoList?
    dynamic var assignee: User?
    
    // For to-many inverses to to-one relationships, Realm has
    // a LinkingObjects class which automatically stays updated
    // to reflect all the objects that have this object as a parent.
    let attachments = LinkingObjects(fromType: Attachment.self, property: "todo")
    
}

TodoList

class TodoList : Record {

    
    // MARK: - Properties
    
    dynamic var title: String = ""
    dynamic var isArchived: Bool = false
    
    
    // MARK: - Relationships
    
    let todos = LinkingObjects(fromType: Todo.self, property: "todoList")
    
}

Attachment

class Attachment : Record {
    
    
    // MARK: - Properties
    
    dynamic var title: String = ""
    
    
    // MARK: - Relationships
    
    dynamic var todo: Todo?
    
    // Mist has an Asset class that's equivalent to CloudKit's CKAsset, except that
    // Mist automatically persists the assets locally so they're always available.
    dynamic var attachedFile: Asset?
    
}

User

Because every CloudKit Container has a Users Record Type, Mist defines a subclass for it out of the box:

public class User : Record {}

Since CloudKit's Users Record Type has no properties by default, neither does this subclass. If you wish to add properties, you can do so through a class extension:

public extension User {
    
    
    // MARK: - Properties
    
    dynamic var firstName: String = ""
    dynamic var lastName: String = ""
    
}

Create Records

Once you've created your Record subclasses, you'll want to use them to create some Records. Let's say you're going to run some errands by yourself:

// Learn more about Mist.currentUser in [section goes here]
let me = Mist.currentUser

let errands = TodoList()
errands.title = "Errands"

let pickUpDryCleaning = Todo()
pickUpDryCleaning.title = "Pick up dry cleaning"
pickUpDryCleaning.assignee = me
pickUpDryCleaning.todoList = errands

let buyStamps = Todo()
buyStamps.title = "Buy stamps"
buyStamps.assignee = me
buyStamps.todoList = errands

let buyGroceries = Todo()
buyGroceries.title = "Buy groceries"
buyGroceries.assignee = me
buyGroceries.todoList = errands

let groceryList = Attachment()
groceryList.title = "Grocery List"
let groceryListTextFile = Asset()
groceryListTextFile.fileURL = ... // URL to local file on device
groceryList.asset = groceryListTextFile
buyGroceries.attachment = groceryList

Save & Retrieve Records

Now let's save these Records. Just like with CloudKit, every Record in Mist must be saved in a Database, and there are three types:

1. Public
   • This is where you store Records you want everyone to be able to see and edit.
2. Private
   • This is where you store Records that you want only the current User to be able to see and edit.
3. Shared
   • If another User shares some of their private Records with the current User, those shared Records appear here.

Much more detail is available in the Mist's Architecture Explained section below. Each one of these Database Types has a corresponding class: PublicDatabase, PrivateDatabase, and SharedDatabase. All of these inherit from the abstract class Database, and all Database instances are backed by Realm instances, which means that Databases have the same syntax and rules for interaction as Realm instances do.

This means that:

1. Database instances (and the Records related to them) are thread-locked.
   • Once you create an instance of one of the concrete Database subclasses, you can only interact with it on the thread where it was created. This also goes for Records you saved to that Database instance or retrieved from it.
2. All instances of a particular Database type (e.g. PrivateDatabase) stay synced with the same data.
   • This means that whenever you need to access or modify the Records for a Database type, you can just create an instance of that type and do the access/modification right there, and you can be sure that a) you'll be working with up-to-date data, and b) any changes you make will be propogated to the other instances of that Database type.
3. You can subscribe to be notified when data changes for a particular Database type.
   • Because everything's backed by Realm, you can use Realm's notification callbacks to ensure you're updated when a change occurs in another instance of the Database.

Saving Records

let errands = ... // As created above

// We don't catch errors here, but you should!
// See [README section] for details
let privateDb = try! PrivateDatabase()
try! privateDb.write {
    privateDb.add(errands)
}

See Realm's documentation for more.

Retrieving Records

// We don't catch errors here, but you should!
// See [README section] for details
let privateDb = try! PrivateDatabase()
let allTodos = privateDb.objects(Todo.self)
let incompleteTodos = allTodos.filter("isCompleted == false")
print("Here are the incomplete Todos: \(incompleteTodos)")

See Realm's documentation for more.

Modify Records

Once you've created your list of Errands, you'll want to check them off as you complete them.

// As created above
let errands = ...
let pickUpDryCleaning = ...
let privateDb = ...

try! privateDb.write {
    pickUpDryCleaning.isCompleted = true
}

You also realize that your roommate already got stamps, so you don't need that Todo at all:

// As created above
let errands = ...
let buyStamps = ...
let privateDb = ...

try! privateDb.write {
    privateDb.delete(buyStamps)
}

Note that all modifications (including deletion) to a Record must occur inside the write transaction for its Database.

Get Notified when Records Change

Realm (and thus Mist) provides three levels of notification: Database-level, Collection-level, and Object-level.

Database-Level Notifications

Database-level notifications fire any time any changes are committed to any of the instances of that Database type.

let privateDb = try PrivateDatabase()
let token = privateDb.addNotificationBlock {

    print("Something changed in the Private Database!")
    
}

// Some time later...
token.stop()

See Realm's documentation for more, but note that unlike Realm's addNotificationBlock function, Database's addNotificationBlock has no parameters because they aren't needed.

Collection-Level Notifications

Collection-level notifications are identical to what Realm provides, so check out their explanation.

Object-Level Notifications

Object-level notifications are identical to what Realm provides, so check out their explanation.

Sync Records with Cloud

One of the best features of Mist is the synchronization is completely automatic! Whenever you save Records (aka whenever a Database's write transaction completes), Mist automatically does the following:

1. Computes a change set for what you just saved to that Database.
2. Efficiently pulls the latest content for that Database from CloudKit.
3. Updates the change set in 1. based on the content delivered in 2. if necessary.
   • For instance, if you just saved an update to a Record that was actually deleted on CloudKit, then your update to that Record will be removed from the change set and will not be sent to CloudKit.
4. Saves the latest changes pulled from CloudKit to the local Database.
5. Pushes the changes represented by the changeset up to CloudKit.

In addition, CloudKit notifies Mist via silent data push whenever content changes in the cloud, and Mist responds to that push by pull the latest data for that Database from CloudKit and storing it in the local Database. This means that the local Database is always up to date with the server (assuming you have an internet connection).

Configuration

Configuration info goes here.

Callbacks

Callbacks info goes here.

---

Mist's Architecture Explained

Local Persistence

Storing Data

In order to understand the rationale for Mist's approach to data storage, let's remind ourselves of how CloudKit stores things.

How CloudKit Stores Data

As described in the CloudKit documentation, every CloudKit-enabled application typically has one CloudKit Container (CKContainer), and every Container has exactly one Public Database (CKDatabase), N Private Databases, and N Shared Databases, where N is the number of User Records (CKRecord) in the Container.

(Graphic Goes Here)

Therefore, all Users share the same Public Database, but each User has her own Private Database and her own Shared Database. Anyone (whether or not she is authenticated) can read from the Public Database, but to read from the Private or Shared Databases or to write to any of the three Databases, the User must be authenticated.

How Mist Stores Data

(Graphic Goes Here)

Because a given device can only have one authenticated User at a time, Mist represents this single-User view of the Container via its local cache. The local cache contains one Public Storage Scope, one Private Storage Scope, and one Shared Storage Scope. The Public Storage Scope exists independently of whether a User is authenticated; the other two Scopes (Private and Shared) are tied to the currently authenticated User, meaning that each instance of Mist has U Private Scopes and U Shared Scopes, where U is the number of Users that have ever been authenticated on a particular Device. Mist allows you to interact with the Public Scope, and the Private and Shared Scopes for the current User if one exists.

Interacting with Data

When using CloudKit directly, you interact with the data like so:

1. Create an Operation (CKOperation) that describes the action you want to perform (searching for records, creating/modifying records, deleting records, etc.),
2. Set up asynchronous callback closures that handle each result of the operation and then the completion of the operation, and
3. Add the operation to the Database on which you want the action to be performed.

Mist takes a similar, but more compact and straightforward approach:

1. Call the relevant static function (Mist.fetch, Mist.find, Mist.add, or Mist.remove),
2. Provide the relevant parameter (what you want to fetch/find, or the Records you want to create/modify/delete)
3. Specify where you want to find it (the StorageScope (.public, .private, or .shared)).

All of this is done in a single line as parameters to the static function, and all results are handled in a single callback block.

Here are some compare-and-contrast examples.

Creating some new Todos & Saving Them

CloudKit

let takeOutGarbageID = CKRecordID(recordName: UUID().uuidString)
let takeOutGarbage = CKRecord(recordType: "Todo", recordID: takeOutGarbageID)
takeOutGarbage["title"] = NSString(string: "Take out garbage") // CKRecordValue requires that we use NSString, not String

let walkTheDogID = CKRecordID(recordName: UUID().uuidString)
let walkTheDog = CKRecord(recordType: "Todo", recordID: walkTheDogID)
walkTheDog["title"] = NSString(string: "Walk the dog")

let container = CKContainer.default()
let publicDb = container.publicCloudDatabase

let modifyRecordsOp = CKModifyRecordsOperation(recordsToSave: [takeOutGarbage, walkTheDog], recordIDsToDelete: nil)
modifyRecordsOp.modifyRecordsCompletionBlock = { (modifiedRecords, deletedRecordIDs, error) in
    
    guard error == nil else {
        fatalError("An error occurred while saving the Todo: \(error)")
    }
    
    print("Todos saved successfully")
    
}

publicDb.add(modifyRecordsOp)

Mist

let takeOutGarbage = Todo()
takeOutGarbage.title = "Take out garbage" // Mist allows us to use the String directly, not NSString like above

let walkTheDog = Todo()
walkTheDog.title = "Walk the dog"

let todos: Set<Todo> = [takeOutGarbage, walkTheDog]

Mist.add(todos, to: .public) { (result, syncSummary) in

    guard result.succeeded == true else {
        fatalError("Local save failed due to error: \(result.error)")
    }
    
    guard syncSummary.succeeded == true else {
        fatalError("CloudKit sync failed: \(syncSummary)")
    }
    
    print("Todos saved successfully")
    
}

Deleting some existing Todos

CloudKit

// Fetch existing Todos
let takeOutGarbage = ...
let walkTheDog = ...

let recordIDsToDelete = [takeOutGarbage.recordID, walkTheDog.recordID]

let container = CKContainer.default()
let publicDb = container.publicCloudDatabase

let modifyRecordsOp = CKModifyRecordsOperation(recordsToSave: nil, recordIDsToDelete: recordIDsToDelete)
modifyRecordsOp.modifyRecordsCompletionBlock = { (modifiedRecords, deletedRecordIDs, error) in
    
    guard error == nil else {
        fatalError("An error occurred while saving the Todo: \(error)")
    }
    
    print("Todos deleted successfully")
    
}

publicDb.add(modifyRecordsOp)

Mist

// Fetch existing Todos
let takeOutGarbage = ...
let walkTheDog = ...

let todos: Set<Todo> = [takeOutGarbage, walkTheDog]

Mist.remove(todos, from: .public) { (result, syncSummary) in

    guard result.succeeded == true else {
        fatalError("Local save failed due to error: \(result.error)")
    }
    
    guard syncSummary.succeeded == true else {
        fatalError("CloudKit sync failed: \(syncSummary)")
    }
    
    print("Todos deleted successfully")
    
}

Fetching Todos You Haven't Yet Completed

CloudKit

 let container = CKContainer.default()
 let publicDb = container.publicCloudDatabase
 
 var todosINeedToDo: Set<CKRecord> = []
 var queryCursor: CKQueryCursor? = nil
 
 let queryPredicate = NSPredicate(format: "completed == false")
 let query = CKQuery(recordType: "Todo", predicate: queryPredicate)
 
 func performQuery() {
     
     let queryOperation = CKQueryOperation(query: query)
     queryOperation.cursor = queryCursor
     queryOperation.recordFetchedBlock = { todosINeedToDo.insert($0) }
     queryOperation.queryCompletionBlock = { (cursor, error) in
         
         guard error == nil else {
             fatalError("Error while querying CloudKit: \(error)")
         }
         
         if let cursor = cursor {
             
             queryCursor = cursor
             performQuery()
             
         } else {
             
             print("Here are the todos you still need to complete: \(todosINeedToDo)")
             
         }
         
     }
     
     publicDb.add(queryOperation)
     
 }
 
 performQuery()

Mist

Mist.find(recordsOfType: Todo, where: { $0.completed == false }, within: .public) { (recordOperationResult, todosIHaveToDo) in
    
    guard recordOperationResult.succeeded == true else {
        fatalError("Find operation failed due to error: \(recordOperation.error)")
    }
    
    print("Here are the Todos you still have to do: \(todosIHaveToDo)")
    
}

Or even simpler:

Todo.find(where: { $0.completed == false }, within: .public) { (recordOperationResult, todosIHaveToDo) in
    
    guard recordOperationResult.succeeded == true else {
        fatalError("Find operation failed due to error: \(recordOperation.error)")
    }
    
    print("Here are the Todos you still have to do: \(todosIHaveToDo)")
    
}

Typed Records with Real Relationships

The Problems CloudKit Creates

CloudKit is all about storing Records, so it's no surprise that CKRecord lies at the core of its implementation. Like CloudKit as a whole, CKRecord is intentionally designed to be flexible, and so it acts as a highly mutable container for content: CKRecord's property names, its identifier (CKRecordID), and even its type (recordType) are all defined as or ultimately backed by Strings, and all properties of CKRecord are optional. While this certainly makes things flexible, it has a very significant downside -- it's all too easy to make careless typos when developing with CKRecord, and since these typos occur in raw Strings, they'll never be caught by the compiler and thus will likely result in subtle and hard-to-notice runtime errors. It also means that you may end up writing lots of code to ensure that certain properties always have a value, and that that value is of a certain Type, since CloudKit won't enforce that for you.

CloudKit also supports a just-in-time schema in development, meaning that you can create your app's schema simply by submitting objects to CloudKit that match it. Again, this makes things very flexible, but has the downside that a simple typo (e.g. listing a CKRecord's recordType as "Todos" in one part of your code, but as "Todo" in another) can cause you to have a different schema than you intended, and leave you with data distributed in bizarre ways across your Container. And besides, most developers settle on a schema for the app quite early on in development, and then don't change it unless they're already making other major changes to their codebase (e.g. as part of making a major new version of their app).

Finally, while CloudKit allows you to relate objects to one another, they require that these relationships are represented by an intermediary object, the CKReference. This means that if you have a highly interrelated data model, you end up doing tons of CloudKit fetches to get an object, and then its related objects, and those objects' related objects, and so on.

The Solution Mist Provides

Mist seeks to solves all these problems through its abstract class Record, and through conventions about how it's used. Record wraps CKRecord (meaning it has a CKRecord instance as a private property), and enforces best practices around how to create and manipulate CKRecords, as described below.

Structure and Conventions for Using Record

CKRecord is intended to be used directly, and cannot be subclassed. This means that you're forced to interact with every CKRecord instance using tons of raw Strings (for property name and recordType values) and to write tons of repetitive code to ensure that a certain property's value is of the type you expect, among other disadvantages.

By contrast, Record is an abstract class and therefore must be subclassed to be used. Every Record subclass follows a couple of simple conventions, and by following these conventions, you get all the advantages you would get with a traditional Swift class. Every Record subclass must:

1. Implement an init function that calls Record's init(recordTypeName:String) function, and passes in the name of the Record Type from your CloudKit Container that this subclass represents
2. Implement all "raw" properties, relationships, and related Assets as computed properties, calling Records respective backing functions within each property's get & set pseudo-functions

Here's the rationale for each of these rules.

The init function

By requiring that every subclass of Record call Records init function and provide a recordTypeName, Mist removes the need to provide typeName every time you create a Record instance, while still allowing you to decouple the name of the subclass from the name of its respective Record Type in CloudKit. This is particularly useful if you want to follow the typical database vs. ORM convention where database tables (equivalent to Record Types) have plural names (Users, Todos, etc.), while ORM Models (equivalent to Record subclasses) have singular names (User, Todo, etc.). Whether you choose to follow this convention is up to you.

Using computed properties

By implementing "raw" properties, relationships, and Assets as computed properties, Mist allows you to get and set the values of your properties in a way that's ultimately compatible with the CKRecord that stores them, while also giving you all the advantages of a proper Swift property. In particular, these properties:

• Have names that Xcode can auto-complete, and can check for correctness at compile time
• Have types that Swift can enforce in both directions, so that you can only set values on that property that make sense, and so that you don't have to check the type of a property's value every time you want use it
• Have explicit nullability, so that if you know a particular property will always have a value (for instance, a boolean flag), you can set it up as such and then you never have to check to see whether it's nil

And in the case of relationships and Assets, you get additional advantages. With relationships, you're able to directly relate two Records together (something you cannot do with CKRecord), and then when you fetch the parent Record, you'll get the related Record automatically without another fetch by default. With assets, you're able to access the asset immediately, since Mist guarantees that it remains cached on the device as long as its respective Record is locally cached.

Other Advantages of Using Record

Identifiers

CloudKit requires that the unique identifiers for CKRecords (instances of CKRecordID) must be unique to each Record within a given Object Type. However, CloudKit does nothing to enforce this, since the recordName property of CKRecordID can be any String, and thus could be repeated across CKRecord instances.

By contrast, Mist's Record has an id property, which is an instance of RecordID (a typealias of String) and is read-only; at initialization time, it automatically gets set to a globally-unique value using NSUUID.

Record Zones

Although it's not very well documented, proper use of Record Zones is critical to enabling efficient synchronization of objects between CloudKit and the local device. In particular, custom record zones cannot be used at all in the public database, but they must be used in the private and shared databases in order to be able to get efficient change sets for CloudKit updates.

Mist and Record work together to ensure that these best practices are followed. In the Private and Shared scopes, root records (Record instances that have no parent Record) get their own Record Zone, and all the children coming off of that Record are put in that same Record Zone (an implicit requirement of CloudKit). In the public scope, everything is put in the default Record Zone (since CloudKit doesn't allow custom Record Zones in the Public Scope) and alternative approaches are used to get efficient data synchronization.

Automatic Synchronization

Synchroniziation is typically a three-step process: pulling down records from the server, reconciling them with what's on the device, and pushing new records back up to the server. This reconciliation process can sometimes be complex, but Mist takes a simple approach:

1. Deletes win
2. Latest edit wins

---

FAQs

FAQs go here.

Current Limitations

• No Asset support

Open Radars

Open Radars go here.

License

Mist is released under the MIT license. See LICENSE for details.