This repository has been archived by the owner on May 31, 2022. It is now read-only.
Updates from upstream 2021-10-28 #14
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit marks MariaDB 10.6 as a supported flavor. It also fixes some test failures caused by 10.6 making "utf8" be an alias for "utf8mb3". Note that MariaDB 10.6's change in this regard is quite different than MySQL 8.0.24's similar change. MariaDB 10.6 updates this charset and its collations completely, in all cases: SHOW CREATE TABLE, information_schema.tables, information_schema.columns, etc. Meanwhile, MySQL 8.0.24 (and still current as of 8.0.26) only replaces "utf8" with "utf8mb3" in the table-level default in SHOW CREATE. However, column- level charsets still show as "utf8" in SHOW CREATE; information_schema still shows "utf8" in all spots; and the collation names have not been changed at all. See 44e227f for the related commit for MySQL, which is inherently hacky due to MySQL's handling here. Since the MySQL and MariaDB behavior have diverged, there's no common Flavor function related to this logic yet.
In MariaDB 10.6, indexes can now be marked as IGNORED. This is essentially equivalent to MySQL 8's INVISIBLE indexes, but with different syntax and information_schema.statistics representation. This commit adds introspection and diff support for IGNORED indexes, including generation of ALTER TABLE ... ALTER INDEX ... [NOT] IGNORED when appropriate.
Add MariaDB 10.6 to the list, and just use a range since the MariaDB list is getting so long.
github.com/fsouza/go-dockerclient: was v1.7.2, now v1.7.4 github.com/VividCortex/mysqlerr: was non-semver, now v1.0.0 github.com/jmoiron/sqlx: was v1.3.3, now v1.3.4
Go 1.17 came out earlier this month, meaning that 1.16 is now the oldest Go version supported by the Go authors, so that's what is used for tests.
Previously, this package only used DB.SetConnMaxLifetime to avoid issues with reusing connections that were idle longer than the session wait_timeout. Connection pools had a max lifetime of 30s, or lower if the session wait_timeout was aggressively low. Golang 1.15 introduced DB.SetConnMaxIdleTime, which is a better solution for this problem. This commit changes the connection logic to now always use a max connection reuse lifetime of 1 minute, and a max idle time of 10s or less depending on the wait_timeout. Additionally, during schema introspection, a much higher max idle connection count is applied as long as the session max conn or global max conn limits aren't too aggressive. This change is intended to help ensure efficient conn reuse during concurrent introspection queries.
CI was failing with a confusing error on this module, "missing go.sum entry for module providing package golang.org/x/sync/errgroup" even though it was present in go.sum already. This updates this module to fix.
In foreign key definitions, all flavors treat the following 3 clauses
completely equivalently:
* ON {UPDATE|DELETE} NO ACTION
* ON {UPDATE|DELETE} RESTRICT
* omitting an action clause from the FK definition entirely
However, the logic for SHOW CREATE TABLE for MySQL 8 is actually the opposite
of that for all other flavors: MySQL 8 will omit NO ACTION clauses but include
RESTRICT clauses, while all other flavors omit RESTRICT clauses but include NO
ACTION clauses.
Previously commit 9139dc8 attempted to correct this logic for MySQL 8, but it
erroneously omitted both NO ACTION and RESTRICT. As a result, tables using
RESTRICT clauses were incorrectly flagged with UnsupportedDDL=true in MySQL 8.
This commit now fixes the logic properly and ensures the integration test suite
covers this case.
In a very specific combination of conditions, some tables were erroneously marked as unsupported for diff operations: * Table created in a MySQL version prior to 8 * Table has at least one column that specifies an explicit character set, which differs from the table's default character set * The column's collation is the default collation for that character set * An in-place upgrade to MySQL 8 later occurs The SHOW CREATE TABLE output for such tables slightly differs from those with identical definitions created after MySQL 8, even though nothing in information_schema differs between the tables. This commit handles this situation so that these tables are not marked as unsupported for diff.
In MySQL 8, SHOW CREATE TABLE behaves differently than previous versions with respect to column-level charsets and collations. Most of these semantics were already handled properly by this package, but some edge cases were not, which caused some tables to erroneously be marked as unsupported for diff. In particular, edge cases exist for tables that are created prior to MySQL 8; after an in-place upgrade (non-dump-and-reload) to MySQL 8, SHOW CREATE TABLE behaves differently for these tables, vs ones initially created in MySQL 8. Recent commit 702c27e attempted to fix this problem, but missed some cases, which this commit fixes. Unit test coverage is also now included. The integration test suite does not currently support database upgrade scenarios (e.g. 5.7 -> 8.0 in the same DB/container), but manual testing further confirmed correctness.
In foreign key definitions, the database treats RESTRICT and NO ACTION identically in terms of behavior, even though they show up differently in information_schema and SHOW CREATE TABLE. This commit adjusts foreign key comparison logic to take this into account: * ForeignKey.Equivalent() treats these as equivalent now * ForeignKey.Equals() still differentiates between them * StatementModifiers.StrictForeignKeyNaming also now controls whether to be strict about these clauses (field name unchanged since it's exported, but it now handles more than just naming) This change is motivated by spurious diffs from MySQL pre-8.0 to 8.0 in-place upgrades, which often hit this RESTRICT vs NO ACTION condition.
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
dbussink
approved these changes
Oct 29, 2021
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pulling latest from
upstream/main