I just sent an email to our open source mobile community sharing some exciting developments we’ve made in the last months. I also want to share it with the blog, because I’m sure many of our database users also have mobile challenges. I believe sync is fundamental to connected devices, and sync is also an important bridge between the traditional database use cases, and the networked world. The JSON document model is well-suited to sync patterns, so I hope the approach we are sharing today will add another reason for people to try NoSQL.

Our mobile team has been building out a new adapter to sync Couchbase Server with mobile devices. We’ve received early feedback from some of our users (thanks!) We’ve also been writing example apps and putting it through its paces, and we really like it. We like it so much that we’re planning to build the future of Couchbase for Mobile around it.

(Veterans may recognize the name ‘Couchbase Mobile’ as our old mobile database from 2011. We’re now repurposing the name as an umbrella for our overarching effort, including the server-side components as well as the native mobile database.)

Previously, our Syncpoint technology tried to address was the management of independent databases for each sync channel. This was a bit of a kludge, due to inherent limitations in the Apache CouchDB filtered replication and security model.

Couchbase for Mobile takes a new approach, using the incremental index capability of Couchbase Server 2.0 for synchronization, so your data lives in scalable storage and channels are lightweight. Documents can also belong to multiple channels, and be visible to multiple users, without making redundant copies on the server.

We’ve talked to a few of you about this new lightweight channel model, and the feedback has been overwhelmingly positive. Channel membership is determined by a channel map function. Here’s an example channel mapper which would allow documents to be tagged with owners and members.

function(doc) {
if (doc.owner_id) {
sync(“owner-“+doc.owner_id);
}
if (doc.members) {
for (var i = 0; i < doc.members.length; ++i)
sync(“member-“ + doc.members[i]);
}
}

The other half of the model is the Sync Gateway authorization API for declaring which channels each user can access. Essentially, each user account on the Sync Gateway includes a list of channels the user has access to. The user can only read documents tagged with one of those channels, and replication will automatically be filtered to those channels.

Based on the feedback we’ve received, this model should be composable in lots of different ways. For instance in the example wiki and chat applications, each chat room or different wiki corresponds to a channel. Each chat room has many messages, and each wiki may have multiple pages. So in these apps, a user will typically be a member or owner of a handful of channels.

In other apps, each user may only sync from a single channel, with a name corresponding to their user name. Or you might have a channel for large photos, and another channel for smaller thumbnails, and sync the large photo channel only when connected via WiFi.

There’s still a lot left to do before Couchbase for Mobile is ready for prime time. We hope to have our an early version out in just a few months. This will focus on proving out the channel model. For later versions, we’ll focus on performance and scalability, as well as adding any missing features identified by the early users.

As part of ratcheting up our focus on mobile, we’ve decided to rename TouchDB to Couchbase Lite. This is not a technical change. We’ll continue to use the same backwards compatible CouchSync protocol, so you can rest assured we aren’t breaking compatibility with existing sync backends. And we are excited to be building what will become the best-in-class integrated synchronization solution.

If you want to get started, we’ve got a jumping-off point here.

I can’t wait to see what you build!

Author

Posted by J. Chris Anderson, Co-Founder and Mobile Architect, Couchbase

5 Comments

  1. How does designating a channel differ from a mobile client performing a filtered replication that synchronizes with a subset of the server documents?

    1. J Chris Anderson February 2, 2013 at 3:35 pm

      The main difference is that it is index driven. Filtered replication without channel indexes requires the filter to run on potentially millions of irrelevant documents for each document that matches. Channel indexes mean you can get acceptable performance even with big data.

      1. BOOM! This is exactly what I\’ve been looking for then. Will Lite DBs be able to utilize this for replicating between Lite databases (soon or in the future)?

        1. J Chris Anderson February 5, 2013 at 1:52 pm

          Currently we don\’t have indexed channels on the mobile device, as the data sets most folks have there are small enough that regular filters are ok.

  2. Thanks for the article Chris, it helps a lot!

Leave a reply