[BUG] mydumper data transmited l doesn't match.

[str]

Describe the bug
When creating a csv+stream dump , I get the following error message:

mydumper --database $DBNAME --stream --csv | pbzip2 --read -m6144 -f > ./$DBNAME-$DATE.csv.mydumper.bz2

** (mydumper:18085): CRITICAL **: 08:02:45.982: Data transmited for export-20220518-080245/mydb-schema-create.sql doesn't match. File size: 0 Transmited: 166

** (mydumper:18085): CRITICAL **: 08:02:46.058: Data transmited for export-20220518-080245/mydb.categories-schema.sql doesn't match. File size: 0 Transmited: 952

** (mydumper:18085): CRITICAL **: 08:02:46.062: Data transmited for export-20220518-080245/mydb.cif_main-schema.sql doesn't match. File size: 0 Transmited: 753

** (mydumper:18085): CRITICAL **: 08:02:46.064: Data transmited for export-20220518-080245/mydb.cif_part_1-schema.sql doesn't match. File size: 0 Transmited: 1132

... and so on for each table

The dump does contain both structure+data. When trying to import the file I get the following error message:

$ pbzip2 -cd kaizensy_kaizendb-20220518.csv.mydumper.bz2 | myloader --stream --innodb-optimize-keys -B mydestinationDB
free(): unaligned chunk detected in tcache 2
Aborted

I noticed that the files with data deine the origin DB name,

``
LOAD DATA LOCAL INFILE '$original_db_name.users.00000.dat' ...

So I tested loading the dump without defining a destination DB. Myloader does create the DB and the tables, but fails to load the data.

**To Reproduce**
Command executed:
* mydumper --database $DBNAME --stream --csv | pbzip2 --read -m6144 -f > ./$DBNAME-$DATE.csv.mydumper.bz2
*  pbzip2 -cd mydump-20220518.csv.mydumper.bz2 | myloader --stream --innodb-optimize-keys 

What mydumper and myloader version has been used?
mydumper_0.12.3-2

**Environment (please complete the following information):**
 - OS version: Ubuntu 22.04
 - MyDumper version: mydumper_0.12.3-2

[str]

I've just tested without the --csv when creating the dump and got the same error messsages

[str]

No, when loading the csv dump, I get the following error:

tar -xf export-20220518-083709.tar.bz2  -I pbzip2
myloader -d export-20220518-083709

** (myloader:364968): CRITICAL **: 16:43:22.157: Error occurs between lines: 4 and 5 on file mydb.kc_users_modules.00000.sql: Loading local data is disabled; this must be enabled on both the client and server sides

** (myloader:364968): CRITICAL **: 16:43:22.157: Error occurs between lines: 4 and 6 on file mydb.gm_cif_cat3_page1.00000.sql: Loading local data is disabled; this must be enabled on both the client and server sides

** (myloader:364968): CRITICAL **: 16:43:22.157: Error occurs between lines: 4 and 5 on file mydb.gm_group_positiongroups_taskOngoing.00000.sql: Loading local data is disabled; this must be enabled on both the client and server sides

** (myloader:364968): CRITICAL **: 16:43:22.157: Thread 4 issue restoring mydb.gm_cif_cat3_page1.00000.sql: 

** (myloader:364968): CRITICAL **: 16:43:22.157: Error occurs between lines: 4 and 5 on file mydb.gm_docs_folders_backup.00000.sql: Loading local data is disabled; this must be enabled on both the client and server sides

** (myloader:364968): CRITICAL **: 16:43:22.157: Thread 1 issue restoring mydb.kc_users_modules.00000.sql: 

** (myloader:364968): CRITICAL **: 16:43:22.157: Thread 2 issue restoring mydb.gm_group_positiongroups_taskOngoing.00000.sql: 

** (myloader:364968): CRITICAL **: 16:43:22.157: Thread 3 issue restoring mydb.gm_docs_folders_backup.00000.sql: 

... and so on

[davidducos]

Can you enable the load local on server side? https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_local_infile

[davidducos]

@str You can ignore the data transmisión errors, I forgot to delete that message, file size calculation wasn't working on some scenarios

[str]

OK, I enadbled the local load with running

SET GLOBAL local_infile = true;
-- and confirmed the flag was set with
SHOW GLOBAL VARIABLES LIKE 'local_infile';

It looks like it worked, as the data was loaded BUT some CSV data was broken, the quotes were not escaped as I see the following error:

myloader -B  mydestdb  --innodb-optimize-keys -d export-20220518-083709

** (myloader:371096): CRITICAL **: 17:09:09.986: Error occurs between lines: 4 and 5 on file mydb.gm_dynforms2_answers.00000.sql: Invalid utf8mb3 character string: '"1. Do you feel comfortable to ask your case manager questions a'

** (myloader:371096): CRITICAL **: 17:09:09.987: Thread 2 issue restoring mydb.gm_dynforms2_answers.00000.sql: 

** (myloader:371096): CRITICAL **: 17:09:16.693: Error occurs between lines: 4 and 6 on file mydb.gm_compliance_calendar.00000.sql: Invalid utf8mb3 character string: '"<div class="entry">['

** (myloader:371096): CRITICAL **: 17:09:16.693: Thread 4 issue restoring mydb.gm_compliance_calendar.00000.sql: 

** (myloader:371096): CRITICAL **: 17:09:29.651: Error occurs between lines: 4 and 6 on file mydb.gm_docs_files.00000.sql: Invalid utf8mb3 character string: '"<div class="entry">['

** (myloader:371096): CRITICAL **: 17:09:29.651: Thread 2 issue restoring mydb.gm_docs_files.00000.sql: 


free(): double free detected in tcache 2
Aborted

This is the line that breaks the import

2480486,57231,1852019,0,"1. Do you feel comfortable to ask your case manager questions about your service?                                             Yes\r\n2. Are you happy with the contact you have with your case managers?                                                                   Yes\r\n3. How could the service be improved?                                                                                                                     N/A\r\n4. Are staff kind caring & respective?                                                                                                                        Yes\r\n5.Overall are you happy with the service provided by Rumbalara Aboriginal Co-operative Ltd.                               Yes",""

And if I try to open the file in vscode I see the following error message:

[davidducos]

Hi @str,
I tried to reproduce the issue, but I couldn't. Is it possible for you to share the files that are failing? or the files with a test case. And finally, what DB version are you using?