Query result with columns that contain Japanese causes Python encoding error.

[araiman]

Issue Summary

• When a query result by Treasuredata has a column that contains Japanese characters, "QueryExecutionError(ascii codec can't encode characters)" occurs.

Steps to Reproduce

1. Run query like select 1 AS "こんにちは" from table

• This error doesn't happen when some values of the query result has Japanese.

Technical details:

• Redash Version: v8(https://github.com/getredash/redash/releases/tag/v8.0.0) / v7(https://github.com/getredash/redash/releases/tag/v7.0.0)
• Browser/OS:Chrome(Version 79.0.3945.130) MacOS(Mojave)
• How did you install Redash:docker

How to solve

• upgrade td_client from 0.8.0 to 0.14.0

Detail

• This error happens on this line. https://github.com/getredash/redash/blob/master/redash/query_runner/treasuredata.py#L95
• Repository of the library.
  https://github.com/treasure-data/td-client-python
• Error message.

[2020-02-02 23:19:41,251][PID:12][WARNING][ForkPoolWorker-1] Unexpected error while running query:
Traceback (most recent call last):
  File "/app/redash/tasks/queries.py", line 361, in run
   data, error = query_runner.run_query(annotated_query, self.user)
  File "/app/redash/query_runner/treasuredata.py", line 100, in run_query
   cursor.execute(query)
  File "/usr/local/lib/python2.7/site-packages/tdclient/cursor.py", line 50, in execute
   self._do_execute()
  File "/usr/local/lib/python2.7/site-packages/tdclient/cursor.py", line 77, in _do_execute
   return self._do_execute()
  File "/usr/local/lib/python2.7/site-packages/tdclient/cursor.py", line 77, in _do_execute
   return self._do_execute()
  File "/usr/local/lib/python2.7/site-packages/tdclient/cursor.py", line 68, in _do_execute
   job = self._api.show_job(self._executed)
  File "/usr/local/lib/python2.7/site-packages/tdclient/job_api.py", line 106, in show_job
   if js.get("hive_result_schema") is not None and 0 < len(str(js["hive_result_schema"])):
UnicodeEncodeError: 'ascii' codec can't encode characters in position 57-60: ordinal not in range(128)
[2020-02-02 23:19:41,251][PID:12][INFO][ForkPoolWorker-1] task_name=redash.tasks.execute_query task_id=27f482cd-ee6b-472f-8878-86cd7e425150 task=execute_query query_hash=8c87b3d333848901b81a7aabd7cb3504 data_length=None error=['ascii' codec can't encode characters in position 57-60: ordinal not in range(128)]
[2020-02-02 23:19:41,254][PID:12][ERROR][ForkPoolWorker-1] Task redash.tasks.execute_query[27f482cd-ee6b-472f-8878-86cd7e425150] raised unexpected: QueryExecutionError(u"'ascii' codec can't encode characters in position 57-60: ordinal not in range(128)",)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/celery/app/trace.py", line 385, in trace_task
   R = retval = fun(*args, **kwargs)
  File "/app/redash/worker.py", line 84, in __call__
   return TaskBase.__call__(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/celery/app/trace.py", line 648, in __protected_call__
   return self.run(*args, **kwargs)
  File "/app/redash/tasks/queries.py", line 436, in execute_query
   scheduled_query).run()
  File "/app/redash/tasks/queries.py", line 382, in run
   raise result
QueryExecutionError: 'ascii' codec can't encode characters in position 57-60: ordinal not in range(128)

[ariarijp]

@araiman Thank you for your contribution!
I'll check it out later.

[ariarijp]

note: Is it similar to treasure-data/td-client-python#27 ?

[ariarijp]

Hi again @araiman

Unfortunately I couldn't reproduce that error because I guess my TD account is gone.
However, I suppose td-client will be upgrade to 1.0.0 in next version of Redash.

If you want to solve that error in Redash v8, you should upgrade td-client to 0.14.0 manually or you can build your own Docker image.

Thanks,

[araiman]

Grateful to your examination. @ariarijp
Got it.I’ll upgrade the version of the library by myself.

I’m looking forward to the next release.