The failure to restore - myloader errors and missing data.

[lkotzian-boku]

Describe the bug
There is a sequence of events where mydumper was executed first, with following options:

> mydumper --rows 1000000:10000000 -F 100 -T paymo_prod.billing_cdr --compress --threads 7 --no-locks --outputdir paymo_prod/billing_cdr -L paymo_prod.billing_cdr.mydumper.log -v 3

The mydumper completed with success. Then the original table was renamed:

mysql> rename paymo_prod.billing_cdr to paymo_prod.billing_cdr_org;

And then myloader started and finished:

> myloader -B paymo_prod -t 7 -d paymo_prod/billing_cdr -L paymo_prod.billing_cdr.myloader.log -v 3

The tail of myloader log carry these messages:

2024-02-06 10:33:49 [DEBUG] - Thread 16: ending
2024-02-06 10:33:49 [DEBUG] - Thread 17: ending
2024-02-06 10:33:49 [INFO] - Errors found:

• Tablespace: 0
• Schema: 0
• Data: 1
• View: 0
• Sequence: 0
• Index: 0
• Trigger: 0
• Constraint: 0
• Post: 0
  Retries: 1

and more error messages were found in the log:

2024-02-06 06:22:22 [WARNING] - Thread 7: Error restoring 1: Column count doesn't match value count at row 5
2024-02-06 06:22:22 [ERROR] - Thread 7: Error restoring: Column count doesn't match value count at row 5
2024-02-06 06:22:22 [ERROR] - Error occurs between lines: 125487 and 126717 on file paymo_prod.billing_cdr.00005.00323.sql.gz: Column count doesn't match value count at row 5
2024-02-06 06:22:30 [ERROR] - Thread 7: issue restoring paymo_prod.billing_cdr.00005.00323.sql.gz:
2024-02-06 10:33:49 [INFO] - Errors found:

As an experiment, I was able to extract the lines from paymo_prod.billing_cdr.00005.00323.sql.gz, and re-insert the data.
However after some spot checking (and comparing billing_cdr against the billing_cdr_org) I found that billing_cdr is missing data.

I've executed the same sequence before against the same table, but with different number of threads and rows' specification without problems.

To Reproduce
Command executed:

• mydumper --rows 1000000:10000000 -F 100 -T paymo_prod.billing_cdr --compress --threads 7 --no-locks --outputdir paymo_prod/billing_cdr -L paymo_prod.billing_cdr.mydumper.log -v 3
• myloader -B paymo_prod -t 7 -d paymo_prod/billing_cdr -L paymo_prod.billing_cdr.myloader.log -v 3

What mydumper and myloader version has been used?
mydumper v0.15.1-3, built against MySQL 5.7.42-46 with SSL support
myloader v0.15.1-3, built against MySQL 5.7.42-46 with SSL support

Expected behavior
I expected that new (billing_cdr) and original (billing_cdr_org) table would match.

Log
If applicable, add the --verbose 3 / -v 3 and --logfile / -L to the execution of the command and upload the file.

Backup
If applicable, add the list of content of the database path, an ls -l will be enough

How to repeat
If applicable, add the minimal table structure and data that we need to reproduce the issue.
Or upload a core dump for cases where mydumper or myloader is crashing.

Environment (please complete the following information):

• OS version: CentOS Linux release 7.7.1908 (Core)
• MyDumper version: myloader v0.15.1-3, built against MySQL 5.7.42-46 with SSL support

Additional context
Add any other context about the problem here.

[davidducos]

Hi @lkotzian-boku
We need a reproducible test case or the content of the file that is failing to understand where might be the issue.

[midenok]

Also 0.15.1 is very old version. There was many bugfixes since then.

[lkotzian-boku]

I was not able to replicate the error.
After switching to version v0.16.1-3, I've repeated the tests. The mydumper worked, however the myloader failed, with those messages:

gzip: stdout: Broken pipe gzip: stdout: Broken pipe gzip: stdout: Broken pipe

The log file output did not contain any warning or error messages.

The loader command was executed as:
myloader --user=... --ask-password -B test -t 12 -d test/test1 -L test.test1.myloader.log -v 3

The last entries in the log file:
2024-04-12 23:17:36 [INFO] - Thread 4: restoring test.test1 part 4017 of 4917 from test.test1.01659.00005.sql.gz. Progress 3045 of 4917. Tables 0 of 1 completed 2024-04-12 23:17:39 [INFO] - Thread 7: restoring test.test1 part 4029 of 4917 from test.test1.01659.00006.sql.gz. Progress 3046 of 4917. Tables 0 of 1 completed

[davidducos]

You need to run the same version of mydumper and myloader.