Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Chat App Data Model

Chris Anderson edited this page · 7 revisions
Clone this wiki locally

API and feature documentation for the Sync Function starts here

Couchbase Lite was designed to simplify data management for multi-user interactive mobile applications. This includes everything from onsite insurance claims, building maintenance, and flight planning apps for airline pilots, to social gaming, festival schedules, and life-logging apps. Perhaps the simplest example of a multi-user interactive application is group messaging.

This article is a tour of the CouchChat application for Couchbase Lite. We'll focus on the native iPhone version, but also take a quick look at the HTML5 PhoneGap version of it as well. CouchChat is a basic group messaging app with photos. Users can create "chat rooms" and invite other users to it. Once in a room, users can send messages and pictures, and see all the other messages and pictures in that room. The native and PhoneGap versions of the application interact with the same dataset, so it doesn't matter which platform users have.

Native Chat Room PhoneGap Chat Room

If you want to run it yourself, follow the CouchChat-iOS README or these instructions on building a PhoneGap container and CouchChat-PhoneGap. This article should be helpful even if you aren't hacking along at home, but it will be more fun if you at least clone a repo and check out the code.

The Data Model

Since we are looking at the app from a data standpoint, we are gonna ignore the UI stuff, and focus on the model classes and their JSON representation in the database. We'll also look at the data flow as it is syncs via the server.

There are three kinds of documents stored by the application: user profiles, chat rooms, and chat messages. Users can belong to many chat rooms, either as a member or an owner. Users also have a profile document where they can specify a nickname. Once a user is in a chat room, they can create messages that belong to that room.

User Profiles

In our app, the user profile documents serve two purposes. The first is as part of the end-user experience. Profile documents for all users are synchronized down to all users's devices, so that when you create a new chat room, you can invite existing users to it by choosing from a list based on profile documents. In a more realistic application, we might limit the set of people who can read a user's profile document. For example purposes we'll pretend everyone is friends with everyone. In real life this would be fine until the invitations to chat rooms from people you don't know starts to create unacceptable amounts of noise in the system. (Having a spam problem generally means you're getting something right.)

Invite to Room

The second purpose of user profile documents is to bootstrap the user in the system. The unique identifier of the profile doc is based on the email address the user authenticated with, ensuring that each user has only one profile. We'll see later when we look at sync routing how the profile document kicks off the user's initial data access.

Here's an example profile. Note that the id will be unique across the entire application. Also note that "type" is not a special database field, it's application data like all the rest. We are just using type = profile in this case to mark the document for special validation rules:

{
  _id: "profile:jchris@gmail.com"
  email: "jchris@gmail.com"
  nick: "jchris"
  type: "profile"
}

There are two rules for profile documents, we'll show how they are enforced later. First, the id must correspond with the email address. Second, you can only update your own profile document. The "nick" field and a photograph, if we decided to add it, are be purely part of the presentation, and don't play a role in a data routing logic.

Chat Rooms

Users can create chat rooms and add other users to the room. This results in the creation of a room document, which looks like this:

{
  _id: "F2194E83-049A-4ADD-9F6A-79AB8FBA5F3C"
  channel_id: "F2194E83-049A-4ADD-9F6A-79AB8FBA5F3C"
  owners: ["jchris@gmail.com"],
  members:["jens@couchbase.com","traun@couchbase.com"],
  title: "Make it Happen!"
  type: "room"
}

The room document is used in the mobile apps to display a list of chat rooms you can interact with. It is created by the owner on their device, and when it syncs to the cloud, it is then shared with all the other members of the room. Now the room document is on their devices, so they see it in their list of rooms, and can click to enter the room, and read and write chat messages.

List of Rooms

There are some backend validation and routing rules associated with room documents as well. I'll summarize them here and then we'll look in more detail when we're reading server side code. Basically, only an owner can modify the title or the list of members. So when you create a room, you've always got to list yourself as an owner. If you add other owners, one of them would be able to remove you from the list of owners, but mere members don't have that capability.

On the backend, the room documents grant access to the room to members and owners. In our app each room gets a distinct channel_id which is placed on the chat message documents to identify them with the room. So when the room document has granted a member access, they can now see all messages with that channel_id -- all messages in the room.

Chat Messages

Ahh, finally the juicy bits. The chat message document type is simple. When you type something into a room, the mobile app creates a new document with a timestamp and a channel_id for the room you are in. If you take a photograph, it is shared as an attachment to a chat document. Here's the PhoneGap code for attaching a photograph.

Here is an example chat document:

{
  _id: "DDB00A7C-936F-498A-876F-4C6F5E5756DE"
  author: "grampz@gmail.com"
  channel_id: "F2194E83-049A-4ADD-9F6A-79AB8FBA5F3C"
  created_at: "2013-04-11T11:34:39.876Z"
  markdown: "Hello"
  type: "chat"
}

Note that it has a channel_id which only users who are members or owners of the room can access. So this means when you sync from the server, you'll only get message for rooms you are a member of.

There is one validation rule for chat messages -- you can't sign a message as someone other than who you really are. This prevents messages from being forged.

Data Routing, Validation, and Access Control

Above, I described a few rules for data validation. In short: profiles can only be updated by users who own them, only room owners can add new members or owners to rooms, and the author field on a chat message must match the actual author's id.

There are also some data routing rules: owners and members of a room can see all the chat messages in it. Everyone can see all profiles in the system.

And I mentioned something about profile documents "bootstrapping" a new user's view into the system.

The Sync Function

One of the coolest things about Couchbase Mobile and the Sync Gateway, is how it allows you do simplify your backend code. Of course, real world apps will be more complex than our chat example, but we think the data access and routing model we've come up with should work even for complex apps.

Sync functions are written in JavaScript, and do three things. First of all, they can validate updates, rejecting documents that don't meet certain criteria. Secondly, they can tag individual documents with channels that they belong to. In the example app, the channel_id field we described above, is used by the sync function to derive actual channel names. Thirdly, they can grant access to a channel to individual users, or groups of users.

The reason this is all done with a function, instead of with a hard-coded schema on the documents themselves, is flexibility. By giving developers a place to adapt to real-world data, and map it to these three kinds of results, we think we've come up with a system that will get the job done for a wide variety of multi-user interactive data applications. We're also proud of it, because at least with the apps we've tried, it offers a 10x reduction in lines of code to describe backend logic. Ten times less code means less places for bugs to hide, and overall quicker development. We know that mobile app developers want to spend their time thinking about code that runs on mobile devices -- the backend is just there to support that. So hopefully the sync function is all the backend you need. We also have APIs for apps that need more flexibility in data routing and access control, but we hope you can get the job done with a short sync function. Here's (a cleaned-up version of) the sync function from CouchChat:

You don't have to grok it all at once, we'll break it up and take it piece by piece in a second.

function(doc, oldDoc, userCtx, secObj) {
  if (doc.channel_id) {
    if (doc.type == "chat") {
      requireUser(doc.author);
    }
    // doc belongs to a channel, sync it on the channel
    channel("ch-"+doc.channel_id);
    // this document describes a channel
    if (doc.channel_id == doc._id) {
      // magic document, treat it carefully
      if (oldDoc) {
        requireUser(oldDoc.owners);
      }
      if (!Array.isArray(doc.owners)) {
        throw({forbidden : "owners must be an array"})
      }
      // grants access to the channel to all members and owners
      var members = [];
      if (Array.isArray(doc.members)) {
        members = doc.members;
      }
      var them = doc.owners.concat(members);
      access(them, "ch-"+doc._id);
    }
  }
  if (doc.type == "profile") {
    channel("profiles");
    var user = doc._id.substring(doc._id.indexOf(":")+1);
    requireUser(user);
    access(user, "profiles");
  }
}

Let's break it down into those three jobs I mentioned earlier: validation, data routing, and access grants.

Validation

Look through the function source for where it calls throw. That's the validation. If the function throws an error, the document change is not written to Couchbase Server nor is is synced with any clients. The client that is trying to send the document to the Sync Gateway will receive an error (which is usually just dropped into the application log.)

The first validation call makes sure that chat documents have a correct author attribute:

    if (doc.type == "chat") {
      requireUser(doc.author);
    }

One of the arguments to the function is a user context, which has a name that we check against the document field. We also see this on the chat room documents, but they can have multiple owners, so instead we check to see if the current user's name is in that list:

  if (doc.channel_id == doc._id) {
    if (oldDoc) {
      requireUser(oldDoc.owners)
    }
    ...
  }

The last validation check is for profile documents. Here we just ensure that the document id matches the user name (wouldn't want someone else updating your profile.)

  if (doc.type == "profile") {
    var user = doc._id.substring(doc._id.indexOf(":")+1);
    requireUser(user)

Routing Data to Channels

We've talked about channels a bit without defining them. In essence, a channel is a unit of synchronization that can contain multiple documents and be accessible to many users and groups. So in the chat app, we have channels for each chat room, but also a channel for user profiles. (I left it out of this document, but if you look at the actual implementation, you'll see there's also a channel for each user, which syncs the chat room documents for the rooms they have access to. This is needed because of a limitation in the implementation that we plan to fix, so maybe by the time you read this it's no longer necessary.)

Chat Room Channels

We sync any document with a channel_id field to a channel corresponding to that id:

  if (doc.channel_id) {
    ...
    // doc belongs to a channel, sync it on the channel
    channel("ch-"+doc.channel_id);

Note that we don't check to see if the user has access to read from that channel. We could, because the list of channels the user can access is part of the userCtx object. Some apps may want to allow users to write to channels they can't read from, so if you want to restrict write access to certain channels, you'll need to add a validation check that the channel_id is valid for the user who is writing.

Also note that we prefix our channel names with ch-. This gives us flexibility down the line if we want to add more kinds of documents with channel ids that should sync differently. It also prevents a wily user from saving a document with a channel_id of a reserved channel like profiles and sending it all other users.

The call to channel routes the current document to a channel. So in this case, we route all documents with a channel_id field to the respective channel. This means both chat messages and room definition documents.

Profile Channel

There is only one other call to channel in the sync function. It is for the "profiles" channel. As mentioned earlier, we send all profiles to every user, so they can be used in the creation of new chat rooms. A more real world app might create instead many profile channels. So there'd be a channel like "jchris-friends" which would include just the people who've indicated that I can invite them to chat rooms. We could store the list of acceptable inviters on the profile document itself to make this easy.

If it's complicated...

For more complex systems, you might end up with a background process to vet new chat room invitations. In a nutshell, this would work as a two stage process. First, you'd have the invitations to chat rooms only able to be created in a "pending" state. Imagine the current schema we have with members and owners, perhaps you'd also have a "pending-members" field. And the validation routine would allow chat room owners to add other users to "pending-members" but not to modify the "members" field. The only user able to modify the "members" list would be a "friend bot" -- that is, a daemon running on the server. It would be set up to consume a channel only of room documents that have a non-empty "pending-members" list. It could then cross check the pending members against other information (maybe make a query to a relational database or an LDAP system) before moving them from the "pending-members" list to the "members" list.

This approach of using a background bot for more complex scenarios is likely one you'll want to use in anything but the most simple apps. This article doesn't cover it in detail, so if you have questions about this approach, ask them on the mailing list.

Something went wrong with that request. Please try again.