Send huge SQL with --compress=true fails

[djshow832]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

# create a huge SQL
$ mysql -h127.1 -uroot -P4000
mysql> select concat('select ', repeat('a', 16*1024*1024)) into outfile '/tmp/data.txt';

$ mysql -h127.1 -uroot -P4000 --compress=true < /tmp/data.txt;
WARNING: --compress is deprecated and will be removed in a future version. Use --compression-algorithms instead.
ERROR 2013 (HY000) at line 1: Lost connection to MySQL server during query

2. What did you expect to see? (Required)

success

3. What did you see instead (Required)

Lost connection and TiDB logs unexpected EOF.

4. What is your TiDB version? (Required)

tidb: master

$ mysql --version
mysql  Ver 8.1.0 for macos12.6 on x86_64 (Homebrew)

[djshow832]

/label sig/sql-infra

[ti-chi-bot]

@djshow832: The label(s) sig/sql-infra cannot be applied. These labels are supported: fuzz/sqlancer, challenge-program, compatibility-breaker, first-time-contributor, contribution, require-LGT3, good first issue, correctness, duplicate, proposal, security, ok-to-test, needs-more-info, needs-cherry-pick-release-5.3, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.0, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.2, needs-cherry-pick-release-6.3, needs-cherry-pick-release-6.4, needs-cherry-pick-release-6.5, needs-cherry-pick-release-6.6, needs-cherry-pick-release-7.0, needs-cherry-pick-release-7.1, needs-cherry-pick-release-7.2, needs-cherry-pick-release-7.3, affects-5.3, affects-5.4, affects-6.0, affects-6.1, affects-6.2, affects-6.3, affects-6.4, affects-6.5, affects-6.6, affects-7.0, affects-7.1, affects-7.2, affects-7.3, may-affects-5.3, may-affects-5.4, may-affects-6.0, may-affects-6.1, may-affects-6.2, may-affects-6.3, may-affects-6.4, may-affects-6.5, may-affects-6.6, may-affects-7.0, may-affects-7.1, may-affects-7.2, may-affects-7.3.

In response to this:

/label sig/sql-infra

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

[djshow832]

/sig sql-infra

[jebter]

/severity major

[djshow832]

Reason:
A packet larger than 16M is split into multiple compressed packets. TiDB just reads the compressed header, allocates the memory, and assumes all the incoming data(even in new packets) are uncompressed:

tidb/server/packetio.go

Line 121 in 72cf5df

r, err = zlib.NewReader(p.bufReadConn)

However, when the zlib reader reads the next compressed packet, it also takes the compressed header as uncompressed data.
It should read data of compressed length and then uncompress the next packet.

[dveeden]

Looks like this needs SET GLOBAL max_allowed_packet=17*1024*1024; to allow reproducing this.

[djshow832]

Looks like this needs SET GLOBAL max_allowed_packet=17*1024*1024; to allow reproducing this.

They work the same to me. Even if the max_allowed_packet stays 64M (default value), it is the same.

BTW, I tried --compression-algorithms=zstd and --compression-algorithms=zlib, they both fail.

[dveeden]

Looks like this needs SET GLOBAL max_allowed_packet=17*1024*1024; to allow reproducing this.

They work the same to me. Even if the max_allowed_packet stays 64M (default value), it is the same.

I tried again and now it works fine with both a tiup playground with a nightly version and with ./tidb-server (unistore). I guess I had modified max_allowed_packet in the past...