Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --innodb-optimize-keys=SKIP to ignore the new enabled default --innodb-optimize-keys option #1493

Open
TomMettam opened this issue May 10, 2024 · 2 comments

Comments

@TomMettam
Copy link

TomMettam commented May 10, 2024

Describe the Feature Request
Please consider automatically enabling --innodb-optimize-keys for innodb tables with more than 1 index and more than 1 million records.

Current behaviour
I attempted to import a database which includes a table with 0.3 billion records and 10 indexes.

It very quickly slowed to an absolute crawl - after 24 hours I calculated the current velocity and it would've taken a minimum of a week to complete, but in reality much more as it was becoming slower and slower.

I used --innodb-optimize-keys and disabled the "optimisation" in #987 (set UNIQUE_CHECKS back to 0), and the import completed in a few hours.

Expected behavior
Myloader should be able to identify large/complex tables and apply optimisations accordingly

Environment
Linux/AMD64/Gentoo

10.6.17-MariaDB-log

@davidducos davidducos changed the title --innodb-optimize-keys Add --auto to automatically configure mydumper and myloader May 10, 2024
@davidducos davidducos changed the title Add --auto to automatically configure mydumper and myloader Add --innodb-optimize-keys=SKIP to ignore the new enabled default --innodb-optimize-keys option May 10, 2024
@TomMettam
Copy link
Author

@davidducos I'm not sure if making it the default is a good strategy either, for smaller imports it is slower this way.

@davidducos
Copy link
Member

Even with --innodb-optimize-keys, we are not splitting the table on smaller tables.

@davidducos davidducos added this to the Release 0.16.5-1 milestone May 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants