SNOW-182387: UTF-8 decoding exception when querying table in SNOWFLAKE_SAMPLE_DATA (InterfaceError: 252005: Failed to convert current row)

[jazzido]

1. Python version: 3.7.3
2. Issue manifests in Darwin and Linux

import snowflake.connector
c = snowflake.connector.connect(..., database='SNOWFLAKE_SAMPLE_DATA')

r = c.cursor().execute("""
    SELECT tpcds_sf100tcl.customer.c_birth_country
    FROM tpcds_sf100tcl.customer WHERE c_birth_country LIKE 'R%'
    LIMIT 25 OFFSET 50
    """)
list(r)

Exception (likely caused by the value RÉUNION):

arrow_result.pyx in snowflake.connector.arrow_result.ArrowResult.__next__()

arrow_iterator.pyx in snowflake.connector.arrow_iterator.PyArrowIterator.__next__()

~/.local/share/virtualenvs/coeff-app-LEFAeCib/lib/python3.7/site-packages/snowflake/connector/errors.py in errorhandler_wrapper(connection, cursor, error_class, error_value)
    121         if cursor is not None:
    122             cursor.messages.append((error_class, error_value))
--> 123             cursor.errorhandler(connection, cursor, error_class, error_value)
    124             return
    125         elif connection is not None:

~/.local/share/virtualenvs/coeff-app-LEFAeCib/lib/python3.7/site-packages/snowflake/connector/errors.py in default_errorhandler(connection, cursor, error_class, error_value)
     86             sqlstate=error_value.get('sqlstate'),
     87             sfqid=error_value.get('sfqid'),
---> 88             done_format_msg=error_value.get('done_format_msg'))
     89
     90     @staticmethod

InterfaceError: 252005: Failed to convert current row, cause: 'utf-8' codec can't decode byte 0xc9 in position 1: invalid continuation byte

Expected result should be, of course, a correctly decoded string.

[rc-solon]

1. Python version: 3.7.3
2. Issue manifests in Darwin and Linux

import snowflake.connector
c = snowflake.connector.connect(..., database='SNOWFLAKE_SAMPLE_DATA')

r = c.cursor().execute("""
    SELECT tpcds_sf100tcl.customer.c_birth_country
    FROM tpcds_sf100tcl.customer WHERE c_birth_country LIKE 'R%'
    LIMIT 25 OFFSET 50
    """)
list(r)

Exception (likely caused by the value RÉUNION):

arrow_result.pyx in snowflake.connector.arrow_result.ArrowResult.__next__()

arrow_iterator.pyx in snowflake.connector.arrow_iterator.PyArrowIterator.__next__()

~/.local/share/virtualenvs/coeff-app-LEFAeCib/lib/python3.7/site-packages/snowflake/connector/errors.py in errorhandler_wrapper(connection, cursor, error_class, error_value)
    121         if cursor is not None:
    122             cursor.messages.append((error_class, error_value))
--> 123             cursor.errorhandler(connection, cursor, error_class, error_value)
    124             return
    125         elif connection is not None:

~/.local/share/virtualenvs/coeff-app-LEFAeCib/lib/python3.7/site-packages/snowflake/connector/errors.py in default_errorhandler(connection, cursor, error_class, error_value)
     86             sqlstate=error_value.get('sqlstate'),
     87             sfqid=error_value.get('sfqid'),
---> 88             done_format_msg=error_value.get('done_format_msg'))
     89
     90     @staticmethod

InterfaceError: 252005: Failed to convert current row, cause: 'utf-8' codec can't decode byte 0xc9 in position 1: invalid continuation byte

Expected result should be, of course, a correctly decoded string.

Hello, did you already find a solution for the endoding problem using SnowFlake? I would be very interested! thanks

[sfc-gh-achandrasekaran]

Folks this issue has been fixed for a few weeks now. Could you please verify? @rc-solon @jazzido

[Saritakarki]

I am still facing the same error
Failed to convert current row, cause: 'utf-8' codec can't decode byte 0xed in position 24: invalid continuation byte.
I updated to the latest version 3.0.3
Also the old version 2.7.0 gave same issue.

[sfc-gh-mkeller]

That makes me believe that you have some corrupt UTF-8 data in your table. The ability to not verify UTF-8 encoding was obsoleted last year. If your table is older that could be lingering and reencoding your data might be your only way to stop this error from happening in the future.