Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fleece error when querying array property #787

Closed
timritzer opened this issue May 21, 2019 · 9 comments

Comments

@timritzer
Copy link

commented May 21, 2019

Can send relevant DB to someone to test, but would rather not post publicly.

Enterprise client

Recently upgraded to 2.5.1 from 2.1.0. Our demo database (pre-staged db that ships with app) now errors on our only complicated array query. Completely deleting the data and staging fresh with 2.5.1 appears to work, but has very scary implications for our customers.

Tried debugging through code, but fails deep within SQLite calls I am assuming when it hands off to Fleece. Data in DB appears fine, and can be retrieved without issue if not using an array filter (just the doc type as a filter returns fine)

Error as logged: (Max Verbosity)

2019-05-21 11:28:58.192493-0500 Clinical Solution[1835:6120215] CouchbaseLite Query Info: {Query#1} Compiling JSON query: {"WHERE":["AND",["=",[".type"],"FormTemplateAssociation"],["ANY","form_template_orgs",[".search.form_template_orgs"],["IN",["?form_template_orgs"],["[]",1]]]],"WHAT":[[".content"],[".keyid"],["._attachments"],["._id"]]}
2019-05-21 11:28:58.192917-0500 Clinical Solution[1835:6120215] CouchbaseLite Query Info: {Query#1} Compiled as SELECT fl_result(fl_value(_doc.body, 'content')), fl_result(fl_value(_doc.body, 'keyid')), fl_result(fl_value(_doc.body, '_attachments')), fl_result(_doc.key) FROM kv_default AS _doc WHERE (fl_value(_doc.body, 'type') = 'FormTemplateAssociation' AND (EXISTS (SELECT 1 FROM fl_each(_doc.body, 'search.form_template_orgs') AS _form_template_orgs WHERE _form_template_orgs.value IN (1)))) AND (_doc.flags & 1 = 0)
2019-05-21 11:28:58.194350-0500 Clinical Solution[1835:6120215] CouchbaseLite Database WARNING: Invalid Fleece data in SQLite table
2019-05-21 11:28:58.194690-0500 Clinical Solution[1835:6120215] CouchbaseLite Database ERROR: SQLite error (code 20): statement aborts at 17: [SELECT fl_result(fl_value(_doc.body, 'content')), fl_result(fl_value(_doc.body, 'keyid')), fl_result(fl_value(_doc.body, '_attachments')), fl_result(_doc.key) FROM kv_default AS _doc WHERE (fl_value(_doc.body, 'type') = 'FormTemplateAssociation' AND (EXISTS (SELECT 1 FROM fl_each(_doc.body, 'search.form_template_orgs') AS _form_template_orgs WHERE _form_template_orgs.value IN (1)))) AND (_doc.flags & 1 = 0)] datatype mismatch
2019-05-21 11:28:58.194894-0500 Clinical Solution[1835:6120215] CouchbaseLite Database ERROR: datatype mismatch (20/20)
2019-05-21 11:28:58.195338-0500 Clinical Solution[1835:6120215] CouchbaseLite Database ERROR: datatype mismatch (20/20)
2019-05-21 11:28:58.197706-0500 Clinical Solution[1835:6120215] CouchbaseLite Query ERROR: CBLQuery failed: 3/20


  • Version: CBLite 2.5.1
  • Client OS: iOS 12.2
  • Server: N/A
@snej

This comment has been minimized.

Copy link
Member

commented May 21, 2019

If you're a Couchbase customer, you should go through our support channels instead of Github. That will prioritize your issue, and you'll also be able to send data confidentially.

@snej

This comment has been minimized.

Copy link
Member

commented May 21, 2019

...and it does look like we'll need to see the database file itself to debug this.

@snej snej transferred this issue from couchbase/couchbase-lite-ios May 21, 2019

@timritzer

This comment has been minimized.

Copy link
Author

commented May 21, 2019

Created a ticket there as well. Thanks!

@JFlath JFlath added the ffc label May 21, 2019

@snej

This comment has been minimized.

Copy link
Member

commented May 22, 2019

Customer support issue: CBSE-6844. I'm on it.

@snej snej self-assigned this May 22, 2019

@snej snej added this to the Cobalt milestone May 22, 2019

@snej

This comment has been minimized.

Copy link
Member

commented May 22, 2019

This is a repeat of issue #589 -- when I fixed that I forgot that there are two places in the query code where Fleece data is read from the SQLite blob, so I didn't fix the other one, which is part of handling ANY/EVERY queries. I just need to apply the same fix there.

@snej snej closed this in e0a0c24 May 22, 2019

@snej

This comment has been minimized.

Copy link
Member

commented May 22, 2019

Fixed. I verified that the query shown in the logs above runs without error on the database (sent privately). It returns zero result rows, but hopefully that's correct (if not, let me know ASAP.)

@snej

This comment has been minimized.

Copy link
Member

commented May 22, 2019

This bug may affect ANY/EVERY queries on any database created in 2.0/2.1 ... we should consider this fix for a maintenance release.

@timritzer

This comment has been minimized.

Copy link
Author

commented May 23, 2019

Try 16 instead of 1 for the Orgs filter. I was eliminating variables to test the effect. The query as written should indeed return 0. With 16 it should return about 20 rows

{"WHERE":["AND",["=",[".type"],"FormTemplateAssociation"],["ANY","form_template_orgs",[".search.form_template_orgs"],["IN",["?form_template_orgs"],["[]",16]]]],"WHAT":[[".content"],[".keyid"],["._attachments"],["._id"]]}

jayahariv added a commit that referenced this issue Jun 7, 2019

Fixed “datatype mismatch” error in ANY/EVERY query
SQLiteFleeceEach was failing to parse Fleece data at an odd address.
This was already worked around in the fix for #589 but that only
covered regular queries, not the special ‘virtual table’ used by ANY
and EVERY queries. This time I made sure that there are no more places
where this bug could lurk.

Fixes #787

Copied from: e0a0c24

jayahariv added a commit that referenced this issue Jun 7, 2019

Fixed “datatype mismatch” error in ANY/EVERY query (#796)
SQLiteFleeceEach was failing to parse Fleece data at an odd address.
This was already worked around in the fix for #589 but that only
covered regular queries, not the special ‘virtual table’ used by ANY
and EVERY queries. This time I made sure that there are no more places
where this bug could lurk.

Fixes #787

Copied from: e0a0c24

@borrrden borrrden modified the milestones: Cobalt, 2.5.2 Jun 8, 2019

@borrrden

This comment has been minimized.

Copy link
Member

commented Jun 8, 2019

Just FYI I have also verified this and with 16 instead of 1 it does return approximately the number of rows indicated. Here is what I have found (✖ for crash ◎ for success)

Opened With Created with 2.1 Created with 2.5
Pre 89f28da
89f28da +
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.