CyMySQL cannot detect encoding of database

[roniemartinez]

I recently encountered failure to insert unicode characters to a table even though the database and tables are already in unicode.

Here are the current conditions:

1. Same tests works on PyMySQL but not on CyMySQL.
2. Tests on CyMySQL only works when explicitly specifying charset='utf8mb4'

EDIT: This is in Python 3 which should automatically use unicode.

[nakagami]

Please show me sample code and that Traceback or any error messages.

[roniemartinez]

Here is the error code, unfortunately, I cannot show everything:

That is just a normal error code when inserting an emoji into a table. However, the database and the tables are already set up in utf8mb4 (charset) and utf8mb4_unicode_ci (collation).

This is the CyMySQL code that does not work:

connection = cymysql.connect(host=host, port=port, user=user, passwd=password, db=db)

However, it's equivalent PyMySQL code works by just changing the module

connection = pymysql.connect(host=host, port=port, user=user, passwd=password, db=db)

By explicitly specifying charset, CyMySQL works.

connection = cymysql.connect(host=host, port=port, user=user, passwd=password, db=db, charset='utf8mb4')

This is not the intended behavior of CyMySQL (based on source) for Python 3 as it should default to using unicode.

[nakagami]

Thanks, I understand.

Can you test with master HEAD branch on Github ?

[roniemartinez]

INSERT query works, however, it fails when SELECT is used to cross-check

E       AssertionError: assert '\U0001f40d' == '\xf0\\x9f\\x90\\x8d'
E         - \U0001f40d
E         ? ^
E         + \xf0\x9f\x90\x8d
E         ? ^^^^

Here are the original and SELECTed data

🐍
�

Explicitly specifying charset='utf8mb4' will output the correct emoji. I believe, the decoding side was not set.

[nakagami]

thanks
I will support utf8mb4 charset.

[nakagami]

version 0.9.14 released
Please check it

[roniemartinez]

@nakagami Verified! Thank you!