Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cassandra / ScyllaDB date fields can't serialize into JSON #4970

Open
susodapop opened this issue Jun 12, 2020 · 3 comments
Open

Cassandra / ScyllaDB date fields can't serialize into JSON #4970

susodapop opened this issue Jun 12, 2020 · 3 comments
Labels

Comments

@susodapop
Copy link
Contributor

susodapop commented Jun 12, 2020

Issue Summary

The CassandraJSONEncoder doesn't support encoding Python Date() objects:

class CassandraJSONEncoder(JSONEncoder):
def default(self, o):
if isinstance(o, sortedset):
return list(o)
return super(CassandraJSONEncoder, self).default(o)

Steps to Reproduce

  1. Run SELECT date_field FROM table against Cassandra or ScyllaDB
  2. Observe the error: "Error running query: Object of type Date is not JSON serializable"

Technical details:

  • Redash Version: v9 Alpha
  • Browser/OS: Firefox / Safari / Chrome
  • How did you install Redash: SaaS
@susodapop
Copy link
Contributor Author

What's interesting is that Redash's JSONEncoder does support dates (see line 90).

class JSONEncoder(simplejson.JSONEncoder):
"""Adapter for `simplejson.dumps`."""
def default(self, o):
# Some SQLAlchemy collections are lazy.
if isinstance(o, Query):
result = list(o)
elif isinstance(o, decimal.Decimal):
result = float(o)
elif isinstance(o, (datetime.timedelta, uuid.UUID)):
result = str(o)
# See "Date Time String Format" in the ECMA-262 specification.
elif isinstance(o, datetime.datetime):
result = o.isoformat()
if o.microsecond:
result = result[:23] + result[26:]
if result.endswith("+00:00"):
result = result[:-6] + "Z"
elif isinstance(o, datetime.date):
result = o.isoformat()
elif isinstance(o, datetime.time):
if o.utcoffset() is not None:
raise ValueError("JSON can't represent timezone-aware times.")
result = o.isoformat()
if o.microsecond:
result = result[:12]
elif isinstance(o, memoryview):
result = binascii.hexlify(o).decode()
elif isinstance(o, bytes):
result = binascii.hexlify(o).decode()
else:
result = super(JSONEncoder, self).default(o)
return result

@susodapop
Copy link
Contributor Author

Just a note for other searchers: you can work around this by casting to TIMESTAMP like so:

SELECT CAST(datefield AS TIMESTAMP) "new_date_alias" FROM table

@tim5go
Copy link
Contributor

tim5go commented Sep 21, 2020

@susodapop
https://github.com/datastax/python-driver/blob/master/cassandra/util.py
Screenshot 2020-09-21 at 12 29 50 PM
From the above code snippet, I believe the cassandra python driver casts a date_field into a cassandra custom Date object cassandra.util.Date (rather than datetime.date).

That's why the CassandraJSONEncoder is throwing error when it encounters date_field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants