Does not seem to handle blobs larger than 8M.

[jenstroeger]

I've got a files table with a MEDIUMBLOB column, which is set up to hold up to 12M large blobs. Here is a test program which consistently fails for me

import cymysql
conn = cymysql.connect(
    host='localhost',
    user='user', passwd='',
    db='test_db',
    charset='utf8',
    )
cur = conn.cursor()
for i in range(1,14):
    blob = b'x' * i * 1048576
    print('writing', i, 'len', len(blob))
    cur.execute(
        'insert into files (id, created, updated, content, name) VALUES (%s, %s, %s, %s, %s)',
        ['manual-test-'+str(i), None, None, blob, None]
        )
    conn.commit() # Fails with or without.
conn.close()

Running this program results in the following:

writing 1 len 1048576
writing 2 len 2097152
writing 3 len 3145728
writing 4 len 4194304
writing 5 len 5242880
writing 6 len 6291456
writing 7 len 7340032
writing 8 len 8388608
Traceback (most recent call last):
  File "/.../lib/python3.4/site-packages/cymysql/cursors.py", line 128, in execute
    self._query(query)
  File "/.../lib/python3.4/site-packages/cymysql/cursors.py", line 234, in _query
    conn.query(q)
  File "/.../lib/python3.4/site-packages/cymysql/connections.py", line 340, in query
    self._execute_command(COM_QUERY, sql)
  File "/.../lib/python3.4/site-packages/cymysql/connections.py", line 440, in _execute_command
    self._send_command(command, sql)
  File "/.../lib/python3.4/site-packages/cymysql/connections.py", line 436, in _send_command
    self.socket.sendall(prelude + sql)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dbtest.py", line 21, in <module>
    ['manual-test-'+str(i), None, None, blob, None]
  File "/.../lib/python3.4/site-packages/cymysql/cursors.py", line 133, in execute
    self.errorhandler(exc, value)
  File "/.../lib/python3.4/site-packages/cymysql/cursors.py", line 35, in errorhandler
    self.connection.errorhandler(self, errorclass, errorvalue)
  File "/.../lib/python3.4/site-packages/cymysql/connections.py", line 116, in errorhandler
    raise Error(errorclass, errorvalue)
cymysql.err.Error: (<class 'BrokenPipeError'>, BrokenPipeError(32, 'Broken pipe'))

I've spent a lot of time trying to configure my DB thinking this is a MySQL setup problem, and I ended up with the following my.cnf:

max_allowed_packet=1G
innodb_buffer_pool_size = 256M
innodb_log_file_size = 256M
bulk_insert_buffer_size = 16M

This should be sufficient.

However, the MySQL documentation states that both server and client have to support large packet sizes (see documentation link). Above configuration is server side only, with cymysql being the client's driver.

Looking at the PyMySQL implementation of the connection setup (source code) I see a default parameter max_allowed_packet=16*1024*1024 which is missing from the CyMySQL implementation (source code). So I switched to import pymysql and everything works as expected!

I think that this issue needs to be investigated, and probably fixed by CyMySQL.

[nakagami]

Thanks.
I recognize as a driver's problem.

[nakagami]

Umm, My modification does not work well....
Can the command packet send chunks one by one ? Query packet length has limitation, I think.

[jenstroeger]

I'm not sure I understand your question, could you please elaborate?

[nakagami]

I think COM_QUERY can send long query string,
But max_allowed_packet branch has error.
master...nakagami:max_allowed_packet

[nakagami]

Closes as a specification limitation