Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Bucket shadowing not propagating deletes #285

Closed
jnordberg opened this Issue · 10 comments

4 participants

@jnordberg

When removing a document from the bucket that is being shadowed the sync bucket is not updated. New documents and updates works fine.

@jessliu

@snej or @andreibaranouski do we have a test for this? Can we reproduce?

@jnordberg do you have any other information that you can share, such as how the document was removed, how the sync function was set up, etc.?

@jnordberg

The document was removed from the bucket by a remove call via coucbhaselib, changes never propagate. From what i can tell sync_gateway isn't even getting a message that something happened.

sync fn

function(doc, oldDoc) {

  if (doc._deleted) {
    requireUser(oldDoc.owner);
    return;
  }

  function isValidUsername(name) {
    return (typeof name == 'string' && /^[0-9a-z]+$/.test(name));
  }

  if (!isValidUsername(doc.owner)) {
    throw ({forbidden: "invalid document owner"});
  }

  if (!(Array.isArray(doc.members) && doc.members.every(isValidUsername))) {
    throw ({forbidden: "invalid document members"});
  }

  if (oldDoc) {
    // validate changes
    if (doc.owner != oldDoc.owner) {
      // only admins can change the owner of a document
      requireRole('admin');
    }

    // only doc.members and admins can modify the document
    if (!haveUser(oldDoc.owner) && !haveUser(oldDoc.members) && !haveRole('admin')) {
      throw ({forbidden: 'no access'});
    }
  } else {
    // validate new document
    // only allow users to create documents as themselves
    requireUser(doc.owner);
  }

  channel(doc.owner, doc.members);
}
@snej
Owner

Try turning on "CRUD" and "Shadow+" logging. You should get some logs regarding the deletion. Please post them here.

@jessliu

@jnordberg is this issue still persisting after turning on CRUD and/or Shadow+? Also, can you try using the latest stable branch? There is a possible fix for this.

@jessliu jessliu added this to the G.A. milestone
@jnordberg

We changed our setup so we didn't need the bucket shadowing, i'll see if i can reproduce just by keeping a shadow running on our new setup

@jnordberg

Same thing... If i delete a document via the sync gateway the document in the bucket gets deleted, if i do it the other way around and delete it from the shadow bucket it still appears in the gateway and the logs are silent.

Also i noticed now that turning it on for an existing bucket did not shadow all the existing document, just stuff that was created/updated after shadowing was turned on.

@jessliu

I see, thank you. Instead of bucket shadowing what are you using instead?

@jnordberg

We are using the gateway as db for our backend as well, over the admin interface, never talking directly to the couchbase server

@snej
Owner

Our unit tests are infallible so there's no way this could be happening :wink: but I'll humor you and test this manually today using a real Couchbase Server bucket.

@snej snej closed this in 9e71ab4
@snej
Owner

Fixed. For some reason Couchbase Server sends Tap deletion messages with a nonzero expiration field, even for non-expiring documents. The shadower code was ignoring all events with a nonzero expiration. Made it only ignore them when it's an insertion/update event, not deletion. Verified that deletion from a real Couchbase Server bucket is propagated to the gateway database.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.