Migrate To PouchDB

[gr2m]

hoodie.js is currently using LocalStorage for local persistence. It's slow, synchronous, and several people advised me not to use it for anything beyond a cookie functionality.

On the other side, there is http://pouchdb.com/. It works with indexdDB, but there is also a webSQL adapter. What kept me from using PouchDB is the lack of support for IE < 10.

But, I just found out that the most simple fallback is to use PouchDB, is to use it directly with the CouchDB endpoint. So when a user saves a document, the update gets directly sent to the Couch, instead of storing all documents locally in the browser and then asynchronously replicate with the Couch in the backend.

Also, when thinking about browser support for hoodie: IE < 9 does not support CORS, which is a requirement for the current default setup of a hoodie app.

It's definitely something we should consider.

[gr2m]

maybe later.

[janl]

Another angle is that PouchDB would allow us to handle conflicts locally, which is nice.

That would then in turn also mean that we need an API for that.

A few options are: choose a default conflict resolution scheme when you start using a store: hoodie.store.open(name, {conflict_strategy:'server'}), the options are client where client versions win, server where server versions win, timestamps where the “latest” changes win (if one can guarantee that local and remote times are reasonably close to each other or manual with a function:

function(version_a, version_b) {
  // custom logic
  // return doc that is a, b or a custom merge of the two.
}

---

In addition, PouchDB’s use of IndexedDB is important for use in Firefox OS which discourages localStorage use.

[timblack1]

It seems to me the relative maturity, or maintainability, of Hoodie's vs. PouchDB's client-server sync code (which I haven't attempted to compare myself) would be a factor in deciding to pick one or the other.

[janl]

@timblack1 good call.

A bit of historical context: when hoodie.store’s sync engine was first built, PouchDB was around, but not yet anywhere mature, so we (@gr2m in fact) wrote a very limited version (no attachments, no local conflict handling, no views etc.) of a synching CouchDB client using /_changes and /_bulk_docs?new_edits=true.

The current Hoodie client is fairly mature, but so is PouchDB, so I think now is a good time as any to look at switching because of the benefits of the larger and generally more preferable underlying data stores.

A cop out would be another PouchDB backend for localStorage for REALLY old browsers, but I’d not make that a priority at the moment.

[fineline]

Hi, just weighing up Hood.ie as well as PouchDB myself at the moment, thought I'd let you know that from a potential adopter POV, using pouch under the hood rather than localStorage would give me more confidence. I agree localStorage is more for cookie style usage, and I've run up against limitations using it in the past and switched to WebSQL on that project. I saw a recent hood.ie blog post mentioning this as a known issue and indicating hood.ie will aim to switch to pouchdb in the midterm. I'll stay tuned and circle back for another evaluation of hood.ie at that point. Good luck with this interesting framework.

[matthew-andrews]

Does using Hoodie need to tie you to using CouchDB?

TL;DR: IndexedDB falls back to WebSQL via a shim. PouchDB falls back to Couch via the network.

It would take more work but I believe it would be much more flexible if people were to architect their applications like this:-

Assuming a non-IndexedDB browser (iOS, Safari - and of course the browsers that lie about their ability to support IndexedDB - e.g. Blackberry 10*)

• Native browser APIs (in this case WebSQL)
• IndexedDB Polyfill for WebSQL (https://github.com/axemclion/IndexedDBShim / https://github.com/jensarps/IDBWrapper) on top of native APIs.
• Perhaps a lighter IndexedDB only fork of PouchDB sitting on top of the IndexedDB polyfill.

As a community moving forward I think it would be far more future proof to encourage people to standardise on a IndexedDB shim (for example @axemclion's) and then when we want to do more custom stuff (e.g. data sync) in our abstractions on top of that we would only need to worry about implementing the IndexedDB adapter and then once browsers finally drop WebSQL we would need only to throw away our polyfill.

* Try one on this page: http://users.telenet.be/kristofdegrave/test/

[janl]

Data sync is core to how Hoodie operates. I’d be happy to consider alternatives once they arise, but given that sync is really tough problem and PouchDB and CouchDB are the only two compatible client/server projects for this at this point, I think moving to PouchDB is a very good idea :)

[matthew-andrews]

I'm not suggesting not using PouchDB for data sync - just splitting out the two problems being solved (data sync and unifying browser APIs into separate modules to maximise their use in other projects).

[janl]

Pouch already does both of those things with a rather nicely separation, I’m not sure I understand the benefits of what you are proposing.

[matthew-andrews]

I've taken a bit more time to try to understand what hoodie's goals are a bit and I think I misunderstood slightly :). Also it's a shame that Coach / Pouch are the only databases that work in this way. It looks like they have got good native support for blob data which would should cover #151 too.

[andru]

I'm working on an application for which IndexDB is going to be necessary beyond prototype stage. I switched from PouchDB to Hoodie a few weeks ago (because I love the goal of Hoodie!) with the intention to create an adaptor for Hoodie to use PouchDB as it's store.
Is there any active work on this I can contribute to? Any existing docs or thoughts I could use as a jumping off point?

[gr2m]

There is no active work on it from our side yet, but still on our roadmap. If you want to get on it, please go ahead, ping me when you have any questions. I implemented the current solution on top of localStorage

[janl]

I just came across localForage. I’m not sure yet if it is of any use to us, I just wanted to throw it in the round and didn’t know where else to put it :)

[kswedberg]

@janl: While you're checking out local storage shims, you might want to take a look at this one as well: https://github.com/WebReflection/db

[svnlto]

depends on #96

[nolanlawson]

Just in case you folks weren't aware, we're actually pretty close to adding localStorage support to PouchDB: https://github.com/daleharvey/pouchdb/issues/44. It looks like it might even make it into the next or next+1 version.

[jo]

Yeah!
We also did some significant replicator performance improvements.

[svnlto]

/cc @caolan

[janl]

Excellent news, thanks! :)

[caolan]

@jo nice :) Replication performance has stopped me from being able to use pouchdb in the past, so I'm looking forward to trying it out again on some slow connections and reasonably sized data sets :)

[jo]

I know ;)
Its still not as performant as I desire but the first steps are done.

[JohnBerg60]

I've been following hood.ie for some time now, and yesterday I stumbled over PouchDb. I've build some infrastructure (remote node and couch server). This took some effort, as pouchdb needs >1.3 couch, and Debian installs 1.7. But after compiling everything myself, it worked.
I build my first app on it. Of course its a todo app, with angular and pouchdb.

The point that I am missing, is what the real difference is between hood.ie and pouchdb. Is it just the use of localstorage vs IndexedDb?
Or is it that hood.ie has some methods to let users quickly signup?
What am I missing here?

[gr2m]

PouchDB is a library that provides the CouchDB API in the browser. Hoodie is a backend that happens to use CouchDB for storage, and has a simple to use Frontend API for common backend operations and offline-sync. We've rebuild a subset of the CouchDB API ourselves way before there was PouchDB. But now that it's there (and it's great) we want to replace our own localStorage based offline data store with Pouch. Makes sense?

Note that this is discussion to use PouchDB in the frontend (hoodie.js), what happens on the backend is out of scope of this discussion.

[davelab6]

@nolanlawson posted http://pouchdb.com/2014/07/25/pouchdb-levels-up.html a few days ago, which is excellent news and addresses the concern of @matthew-andrews and others that Pouch was tightly coupled to Couch.

[gr2m]

@davelab6 yeah we've read the post with big interest, it's fantastic news indeed :)

[gr2m]

Long time coming ... it's happening:
https://github.com/hoodiehq/wip-hoodie-store-on-pouchdb

[andru]

<3 x 1000

I'm back working on a project which could really benefit from this so I'll try to pick up any issues I can help with.

[gr2m]

@andru I've prepared tons of very small issues, might take only 10-20min to fix some of them, and I think they are all well described. Just assign to yourself and hack away :)
https://github.com/hoodiehq/wip-hoodie-store-on-pouchdb/issues

[gr2m]

I've setup a first patched Hoodie App that is using PouchDB internally:
https://github.com/gr2m/hoodie-pouchdb-test/

There is one issue with bundling hoodie.js that is causing trouble at the moment, if anyone could look into it, that'd be great:
gr2m/hoodie-pouchdb-test#1

[vicentedealencar]

+1

[andreabedini]

loving this

[michielbdejong]

Wow, cool!

[gr2m]

Hey everyone, we are doing great progress on the migration to PouchDB. Now is a great time to get involved — let's get it done! Here's a great overview of open issues, including some starter ones:
http://espy.github.io/ubersicht/?showOpen=true&showClosed=false&showCommented=true&showUncommented=true&showIssues=true&showPullRequests=true&last24Hours=false&repos=hapi-couchdb-store,pouchdb-hoodie-api,pouchdb-hoodie-store,pouchdb-hoodie-sync&labels=help%20wanted&milestones=null&usernames=null#hoodiehq

Happy to help you get going, ping me for any questions

[gr2m]

Done with new Hoodie: https://github.com/hoodiehq/my-first-hoodie/tree/something-tracker#local-setup-to-test-new-hoodie

Still work in progress, but it’s working \o/

[Largo]

Edit: Thanks for delivering it now :) 👍 👍 👍

After fixing an npm error, I managed to get something-tracker to work. Is that still the recommended way to work with pouchdb in hoodie? I don't know much about npm.

I'm about to start a project and Hoodie would be great overall, but I can't use it for my app, when I have to worry about localstorage limits.

PS: If you're worried about backwards compatibility, how about a plugin. hoodie install pouchdb would be easy