[nano] Transaction support?

[jhs]

Have you seen the library for CouchDB transactions I released yesterday, Txn?

The idea is, instead of having a document and trying to store it, you have an id and you first fetch, then see what it looks like, make some changes, and finally store again. (With automatic retry features.) However, the goal is that your code looks roughly the same.

Do you think it would make sense for Nano to expose this functionality? If so, what do you think the API would look like?

[dscape]

Yup, definitely. I have some ideas on supporting transactions over REST. Let's talk on IRC and make up a draft for nano/Txn :)

[perezd]

I'd love to be in on this as well.

[jhs]

I think we had best discuss it here, in public. I'll start off with brainstorming of ideas.

I guess the first concern is that Txn might be a little bit conceptually difficult for new users, for two reasons:

1. It requires you to write code like libraries: receive a callback and call it exactly once, with optional error objects (which you must never throw)
2. Txn requires two callbacks which can be a bit odd-looking in the source code

Next, my concern is conceptual misunderstanding.

1. Primarily, instead of having a document and doing something with it, you have an ID and you react to whatever the content happens to be. That is how Txn can retry several times, it re-runs your function, which may have completely different behavior from run to run (as the doc changes)
2. On the other hand, a major exception that I use a ton is hinting the doc contents to Txn, to skipping the first fetch. Obviously this is really nice for a _changes workflow
   1. Query _changes?include_docs=true
   2. When you get an event, you already know the doc and thus _id and _rev
   3. Try a "claim the job" transaction
      • txn({max_tries:1, after:Math.random(), doc:change.doc}, ...) With max_tries=1, this is basically a normal doc update after a random delay. (The random delay means you can run many identical copies of this code.)
      • The operation function just sets doc.pending = true; doc.expires_at = new Date + 5000
      • If the transaction fails, ignore it. Somebody else got it and you will see it again soon.
      • If the transaction succeeds, run another transaction to do the real job.

That is pretty cool and I am using something like this in some places, however:

• That seems a bit complicated, perhaps it could be simplified and implemented inside Txn or Nano.
• It is not a general job queue, If the worker crashes, it will never update the doc again, but you have already ignored it. So that job will be incomplete forever (or until you start a new worker from scratch). This is exactly why I moved to CQS for jobs. Still, some _changes tools would probably help the community.

The final note is that I am still not a regular Nano user (still waiting on that next big couch client project)

[dscape]

I didn't get the time to review Txn yet but these are so far my overall thoughts on transactions:

1. There's no space in nano for transactional code. So importing a library like Txn would be ideal. nano should be clean and feel like a introduction to the CouchDB API via code. Keeping things minimal and working with each other is awesome and desirable.
2. CouchDB supports transactional insert of a set of documents (bulk upload). However it's possible that Couch accepts some updates and not others, so these should be possible to rollback. (am I wrong here?)
3. CouchDB allows bulk deletes but with the some restrictions.
4. nano should allow doing this while keeping ACID semantics, if possible
5. It is desirable that you can mix inserts and deletes in a single statement.
6. It is desirable that you can do property level updates on CouchDB. This is something like I envision on implementing transactions over REST. Responsibility would have to be on the client side cause server does not deal with this.

function update_user(gh) {
  db.view( ''users", "by_github_account", {key: gh.id})
       .increment('version', 1)
       .replace('github_login', gh.login)
       .replace('github', gh.user)
       .delete('linkedin.deletemeplease')
       .rename('meeutp','meetup')
       .push('anarray', 1) // should also include security stuff
       .save(
         function save_cb(e) {
           if(e) {
             console.log("Couldn't update ");
             return;
           }
           console.log(gh.id + " updated");
           return;
       });
};    

This should look a lot better than this but would be cool to be able to do something like this.

I'll revisit the issue after I check out Txn later this week :)