[BUG] "real_db_name" isn't found for schema SQLs

[Razican]

Describe the bug
I'm getting the following critical error:

** (myloader:3509885): CRITICAL **: 08:45:41.129: It was not possible to process file: int1_16011.a8n_pipeline_command_s7e-schema.sql (2) because real_db_name isn't found. Restore without schema-create files is not supported

It says that a restore without schema-create files is not supported, but the file it mentions is the schema file itself, generated by mydumper (same version as myloader), so might be related to something else.

To Reproduce
The mydumper command I'm using:

mydumper -S /var/mysql/mysql.sock -B int1_16011 --events --less-locking -o /db_backups/int1_16011 --trx-consistency-only --threads 8 --statement-size 5000000 --chunk-filesize 500 --build-empty-files

And the myloader command I'm using:

myloader -S /var/mysql/mysql.sock -B int1_16011 -d /db_backups/int1_16011_replicator --threads 8 -e

Using mydumper & myloader:

myloader 0.12.3-2, built against MySQL 5.7.37-40

Expected behavior
I would expect the load to work without any critical error that stops the load.

Log
If applicable, add the --verbose 3 / -v 3 and --logfile / -L to the execution of the command and upload the file:

2022-05-16 08:56:16 [INFO] - Server version reported as: 5.5.5-10.2.30-5-MariaDB-enterprise-log
2022-05-16 08:56:16 [DEBUG] - Thread 1: Starting import
2022-05-16 08:56:16 [DEBUG] - Thread 2: Starting import
2022-05-16 08:56:16 [DEBUG] - Thread 3: Starting import
2022-05-16 08:56:16 [DEBUG] - Thread 4: Starting import
2022-05-16 08:56:16 [DEBUG] - Thread 5: Starting import
2022-05-16 08:56:16 [DEBUG] - Thread 6: Starting import
2022-05-16 08:56:16 [DEBUG] - Thread 7: Starting import
2022-05-16 08:56:16 [DEBUG] - Thread 8: Starting import
2022-05-16 08:56:16 [DEBUG] - Adding database: int1_16011 -> int1_16011
2022-05-16 08:56:16 [ERROR] - It was not possible to process file: int1_16011.a8n_p6e_filter_table_column-schema.sql (2) because real_db_name isn't found. Restore without schema-create files is not supported

Additional context
These commands were working fine in mydumper/myloader 0.9 and 0.10, we are testing to upgrade to 0.12, but we just got this error with the same commands. Checking the usage file, I could not find that something changed in the commands.

[Razican]

As a bit more context, I followed the comments in #613, so I got to this myloader command:

myloader -S /var/mysql/mysql.sock -B int1_16011 -d /db_backups/int1_16011_replicator --threads 8 -s int1_16011 -e

The DB is created, surprisingly (also without the -s parameter), but I sill get the critical error.

The int1_16011-schema-create.sql exists.

[Razican]

Finally, I was able to make it work. It seems that if I use the -B parameter, it will fail (it doesn't matter if I put the -s parameter or not. If I don't use the -B parameter, then it works, and it doesn't matter if I put the -s parameter or not. So this might be something to be checked.

[davidducos]

I was not able to reproduce your issue, if you can develop a reproducible test case, please share and I will be reviewing it, meanwhile I will set this as invalid. Feel free to share more info or close it.

[sebastienbarre]

I can see that error as well (MacOs).

I "accidentally" upgraded to 0.12.3-2 the other day and kept seeing the same errors:

myloader(2238,0x102364580) malloc: *** error for object 0x600002dfc180: pointer being freed was not allocated
myloader(2238,0x102364580) malloc: *** set a breakpoint in malloc_error_break to debug

I just upgraded to 0.12.3-3 to hopefully fix it as I urgently needed to load some backup, and now pretty much all I see is the same error:

** (myloader:5446): CRITICAL **: 19:14:47.966: It was not possible to process file: kwik.contracts_employees-schema.sql.gz (2) because real_db_name isn't found. Restore without schema-create files is not supported

Nothing else in my usage has changed, I've used the same scripts for eons. My call is essentially myloader --database=$db --user=kwik --queries-per-transaction=50000 --compress-protocol --threads=10 --verbose=3 --overwrite-tables --password=$pass --directory=$dir where $db, $pass, and $dir are parameters to my shell script.

The file kwik.contracts_employees-schema.sql.gz exists, though I don't know what the (2) refers to.

The kwik-schema-create.sql.gz file is there too (the database is named kwik, as you probably guessed.
It contains:

CREATE DATABASE /*!32312 IF NOT EXISTS*/ `kwik` /*!40100 DEFAULT CHARACTER SET latin1 */;

Note that all the table files are prefixed with kwik., whereas the schema-screate file is prefixed with kwik-. This has always been the case as far as I remember, and I have a lot of backups on my drive to confirm that, but I figured I would mention it in case you changed the naming convention in the past week...

It's very unfortunate that homebrew doesn't allow people to easily revert to previous versions (but see next comment).
Efforts to build a 0.11 version from cloned source failed pretty miserably (using the same commands as seen when running brew reinstall mydumper --build-from-source --verbose).

cmake . -DCMAKE_INSTALL_PREFIX=/opt/homebrew/Cellar/mydumper/0.12.3-3 -DCMAKE_INSTALL_LIBDIR=lib -DCMAKE_BUILD_TYPE=Release -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_VERBOSE_MAKEFILE=ON -Wno-dev -DBUILD_TESTING=OFF -DCMAKE_OSX_SYSROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -DMYSQL_CONFIG_PREFER_PATH=/opt/homebrew/opt/mysql-client/bin -DMYSQL_LIBRARIES=/opt/homebrew/opt/mysql-client/lib/libmysqlclient.dylib -DCMAKE_DISABLE_FIND_PACKAGE_ZLIB=1 -DZLIB_INCLUDE_DIRS=/usr/include -DZLIB_LIBRARIES=-lz
...
 58%] Building C object CMakeFiles/myloader.dir/myloader.c.o
mydumper/myloader.c:272:50: error: format specifies type 'long' but the argument has type 'GTimeSpan' (aka 'long long') [-Werror,-Wformat]
  return g_strdup_printf("%ld %02ld:%02ld:%02ld",days,hours,minutes,seconds);
...

[sebastienbarre]

OK for MacOS/homebrew users finding this, I got it working by reverting to the latest 0.11. Something was introduced in 0.12 that is breaking on MacOS.

The steps are:

• unlink your current mydumper using: brew unlink mydumper
• manually download the mydumper.rb formula you wish to install instead. For example, head to homebrew-core/mydumper.rb at HEAD · Homebrew/homebrew-core, then click on the History button, and look for the commit that introduced the version you want (in my case, 0.11.5-2) then click on the "View at this point in history" button:

• Click on the "Raw" button to see the raw mydumper.rb file, and download it to your drive (this file).
• Build this formula from source using brew install --build-from-source ~/Downloads/mydumper.rb
• Test the version:

$ mydumper --version
mydumper 0.11.5, built against MySQL 8.0.29

• Prevent brew from updating your version in the future (something I should have done ages ago): brew pin mydumper
• Enjoy mydumper working again :)

[davidducos]

Hi @sebastienbarre, sorry to hear that you hit with this bug. Unfortunately, I'm not able to reproduce it, I need at least a reproducible test case, as several things changed since 0.11.5 and I have an idea where it could be but it is really difficult to find without a test case.

[YogSottot]

I have the same issue

Describe the bug
If target and source db have the same name, then if we use --database option in myloader we have this error.
If i remove --database in myloader command, then it is work.

To Reproduce
Command executed:
in mysql-console:

• create database sitemanager;
• use sitemanager;
• create table tutorials_tbl(
  tutorial_id INT NOT NULL AUTO_INCREMENT,
  tutorial_title VARCHAR(100) NOT NULL,
  tutorial_author VARCHAR(40) NOT NULL,
  submission_date DATE,
  PRIMARY KEY ( tutorial_id )
  );

Then in bash-console:

• mydumper --threads 2 --compress --less-locking --use-savepoints --database sitemanager --outputdir /opt/backup/mydumper
• myloader --verbose 3 --database sitemanager --directory /opt/backup/mydumper --overwrite-tables --threads 2

What mydumper and myloader version has been used?
mydumper 0.12.3-3, built against MySQL 5.7.37-40
myloader 0.12.3-3, built against MySQL 5.7.37-40

Expected behavior
Restore dump without error.

Log

myloader  --verbose 3 --database sitemanager --directory /opt/backup/mydumper --overwrite-tables --threads 2
** Message: 09:26:12.840: Server version reported as: 5.7.37-40

** (myloader:23677): CRITICAL **: 09:26:12.847: It was not possible to process file: sitemanager.tutorials_tbl-schema.sql.zst (2) because real_db_name isn't found. Restore without schema-create files is not supported

How to repeat
Create new empty db with test table:

create database sitemanager;
use sitemanager;
create table tutorials_tbl(
   tutorial_id INT NOT NULL AUTO_INCREMENT,
   tutorial_title VARCHAR(100) NOT NULL,
   tutorial_author VARCHAR(40) NOT NULL,
   submission_date DATE,
   PRIMARY KEY ( tutorial_id )
);

Environment:

• OS version: CentOS Linux release 7.9.2009
• MyDumper version: 0.12.3-3, (zstd)

[davidducos]

Thanks for the test case, bug confirmed.

[sebastienbarre]

Thanks @YogSottot, you beat me to it.
Sorry @davidducos, we had a production emergency, I really needed mydumper, so I focused my efforts on finding (and sticking to) a working version. Hopefully you find the bug.

[davidducos]

@sebastienbarre sorry to hear that you had an emergency, hope that went ok.
I found the bug, It has already fixed in #708 and merged to master, it will be released in the next version. The test case helped me to understand where to find and I was able to add another test case to avoid this kind of issues in the future.
Thanks for all the help @YogSottot @Razican @sebastienbarre !!!