Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_bulkDocs always includes _conflicts=[] --> error from couchbase sync gateway #5793

Closed
hermitdemschoenenleben opened this issue Oct 22, 2016 · 11 comments

Comments

@hermitdemschoenenleben
Copy link

commented Oct 22, 2016

Issue

When inserting new documents into a database and then synchronizing it with couchbase sync gateway, this error occurs:

2016-10-22T21:41:19.973+02:00 BulkDocs: Doc "okifiergzuedih9i19k9" --> 400 user defined top level properties beginning with '' are not allowed in document body (400 user defined top level properties beginning with '' are not allowed in document body)

The reason for this is that the inserted documents have a property _conflicts=[] set, which couchbase sync gateway does not allow. For testing I removed these properties using a wrapper for XMLHttpRequest and then it worked.

I suppose this was introduced with a new version of PouchDB, but I'm not able to test it right now because I can't get older versions of PouchDB to work with rollup.

Info

  • PouchDB 6.0.7
  • Environment: ionic app in the browser
  • Platform: Chrome
  • Server: Couchbase sync gateway 1.3
@willholley

This comment has been minimized.

Copy link
Member

commented Oct 23, 2016

this is likely related to 8f0d7d2. I'll need to investigate further but it's curious that CSG throws errors that CouchDB doesn't. It suggests a compatibility issue in CSG.

@willholley willholley self-assigned this Oct 23, 2016

willholley added a commit that referenced this issue Oct 24, 2016
(#5793) - test _conflicts in allDocs
allDocs should not contain a _conflicts array
when no conflicts exist. This is the behaviour in CouchDB but
not in other implementations.
willholley added a commit that referenced this issue Oct 24, 2016
(#5793) - do not return empty _conflicts array
Previously, PouchDB would include an empty conflicts array in the
response to a document fetch with conflicts=true specified. This patch
adjusts it to match the CouchDB behaviour, omitting the _conflicts
field altogether if there are no conflicts for a given document.
willholley added a commit that referenced this issue Oct 24, 2016
(#5793) - do not POST _conflicts during sync
Couchbase Sync Gateway throws errors when a document within a bulk_docs
request includes a _conflicts field. Strictly speaking, this is likely
a CSG error - it should behave the same as CouchDB - but this is a
pragmatic fix to keep PouchDB compatible.
@nolanlawson

This comment has been minimized.

Copy link
Member

commented Oct 25, 2016

Please see #5795 (comment) ; CSG support is very buggy and I wouldn't really recommend it. PouchDB's test suite only has 100% support for CouchDB 1.x, CouchDB 2.x, and PouchDB Server. Anecdotally Cloudant also seems to work well because it's largely identical to CouchDB 2.x.

@daleharvey

This comment has been minimized.

Copy link
Member

commented Oct 29, 2016

Will leave this open since there is a fix in progress

willholley added a commit that referenced this issue Oct 29, 2016
(#5793) - test _conflicts in allDocs
allDocs should not contain a _conflicts array
when no conflicts exist. This is the behaviour in CouchDB but
not in other implementations.
willholley added a commit that referenced this issue Oct 29, 2016
(#5793) - do not return empty _conflicts array
Previously, PouchDB would include an empty conflicts array in the
response to a document fetch with conflicts=true specified. This patch
adjusts it to match the CouchDB behaviour, omitting the _conflicts
field altogether if there are no conflicts for a given document.
willholley added a commit that referenced this issue Oct 29, 2016
(#5793) - do not POST _conflicts during sync
Couchbase Sync Gateway throws errors when a document within a bulk_docs
request includes a _conflicts field. Strictly speaking, this is likely
a CSG error - it should behave the same as CouchDB - but this is a
pragmatic fix to keep PouchDB compatible.
willholley added a commit that referenced this issue Oct 31, 2016
(#5793) - do not POST _conflicts during sync
Couchbase Sync Gateway throws errors when a document within a bulk_docs
request includes a _conflicts field. Strictly speaking, this is likely
a CSG error - it should behave the same as CouchDB - but this is a
pragmatic fix to keep PouchDB compatible.
daleharvey added a commit that referenced this issue Nov 1, 2016
(#5793) - fixes for sync against Couchbase Sync Gateway
* (#5793) - test _conflicts: [] is rejected by bulk_docs

* (#5793) - test _conflicts in allDocs

allDocs should not contain a _conflicts array
when no conflicts exist. This is the behaviour in CouchDB but
not in other implementations.

* (#5793) - do not return empty _conflicts array

Previously, PouchDB would include an empty conflicts array in the
response to a document fetch with conflicts=true specified. This patch
adjusts it to match the CouchDB behaviour, omitting the _conflicts
field altogether if there are no conflicts for a given document.

* (#5793) - do not POST _conflicts during sync

Couchbase Sync Gateway throws errors when a document within a bulk_docs
request includes a _conflicts field. Strictly speaking, this is likely
a CSG error - it should behave the same as CouchDB - but this is a
pragmatic fix to keep PouchDB compatible.
@daleharvey

This comment has been minimized.

Copy link
Member

commented Nov 1, 2016

Fixed in ef1db28

@daleharvey daleharvey closed this Nov 1, 2016

@snej

This comment has been minimized.

Copy link

commented Dec 10, 2016

Hi, Couchbase developer here (though I work on Couchbase Lite, not Sync Gateway)

the inserted documents have a property _conflicts=[] set, which couchbase sync gateway does not allow.

I'm not aware of this property; I don't think it's one of the allowed "magic" meta-properties in the versions of CouchDB that we based our API support on. Where is this property documented?

@nolanlawson

This comment has been minimized.

Copy link
Member

commented Dec 11, 2016

@snej If you fetch a doc with ?conflicts=true you'll get the _conflicts object: http://docs.couchdb.org/en/2.0.0/api/document/common.html?highlight=conflicts

@willholley

This comment has been minimized.

Copy link
Member

commented Dec 12, 2016

I think the seemingly undocumented behaviour is that CouchDB allows all special fields
to exist in document bodies at write time - it just ignores any fields that are meaningless in that context.

@snej

This comment has been minimized.

Copy link

commented Dec 12, 2016

Ah, I did not know that.
But why would the replicator try to push a document with a _conflicts property? That property only makes sense when received.

@willholley

This comment has been minimized.

Copy link
Member

commented Dec 12, 2016

@snej that was due to a bug in PouchDB, fixed whilst addressing this issue.

@willholley

This comment has been minimized.

Copy link
Member

commented Dec 12, 2016

@snej for more context, PouchDB uses _all_docs as an optimisation to fetch documents with only one revision (I think CBL has a similar optimisation?). However, documents could have conflicts at the root - i.e. multiple 1- revisions, which we weren't replicating. Using _bulk_get instead removes the need for this optimisation/fixes the problem, but until that's widely deployed, we pass conflicts=true to _all_docs and, if a document has conflicts, fall back to individual document fetches. Unfortunately, we weren't stripping out the _conflicts field from the _all_docs results so it leaked into the corresponding _bulk_docs call.

@snej

This comment has been minimized.

Copy link

commented Dec 12, 2016

@willholley Yes, Couchbase Lite has the same optimization for use with CouchDB. Sync Gateway supports _bulk_get, so you can use it there, as Couchbase Lite does.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.