Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql_select, sql_explain fails with json contains queries #1133

Closed
minusf opened this issue Jan 23, 2019 · 3 comments · Fixed by #1229
Closed

sql_select, sql_explain fails with json contains queries #1133

minusf opened this issue Jan 23, 2019 · 3 comments · Fixed by #1229

Comments

@minusf
Copy link

minusf commented Jan 23, 2019

Consider this simple query (data is a JSONField):

ApiUser.objects.filter(data__contains={"is_member": True})

The sql panel will show this query correctly:

SELECT ... FROM "api_apiuser" WHERE "api_apiuser"."data" @> '{"is_member": true}' LIMIT 21

But Sel and Expl fail with an exception:

2019-01-23 16:13:40,664 DEBUG: utils.py:111: (0.003) SELECT ... FROM "api_apiuser" WHERE "api_apiuser"."data" @> '''{"is_member": true}'''  LIMIT 21; args=['\'{"is_member": true}\'']
django.db.utils.DataError: invalid input syntax for type json
LINE 1: ... FROM "api_apiuser" WHERE "api_apiuser"."data" @> '''{"is_me...
                                                             ^
DETAIL:  Token "'" is invalid.
CONTEXT:  JSON data, line 1: '...

cursor.execute() fails because of the triple quotation marks(?).

(Pdb++) sql
'SELECT ... FROM "api_apiuser" WHERE "api_apiuser"."data" @> %s  LIMIT 21'
(Pdb++) params
['\'{"is_member": true}\'']
@tim-schilling
Copy link
Contributor

Was this with postgres?

@minusf
Copy link
Author

minusf commented Nov 24, 2019

yes

@tim-schilling
Copy link
Contributor

What's happening here is that when we decode a JsonAdapter param, the adapter returns the param as a sql quoted string. I think what we want to do is prevent that from happening and simply have it return the JSON string.

Running the following:

Model.objects.filter(json_field__contains={'foo': 'bar'})

With panels.sql.tracking.NormalCursorWrapper._decode below

from django.contrib.postgres.fields.jsonb import JsonAdapter

def _decode(self, param):
    if isinstance(param, JsonAdapter):
        print([param.dumps(param.adapted), str(param)])
        return param.dumps(param.adapted)

Prints the following:

['{"foo": "bar"}', '\'{"foo": "bar"}\'']

The str call is effectively what we return today from _decode for this parameter. The problem is that the rendered form that's used to request the Select or Explain data now has a string that contains the sql wrapping ' characters. Instead we just want a string of '{"foo": "bar"}' to be the parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants