infoschema, sessionctx: add support for mysqldump from 8.0 #10829
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Required for compatibility with mysqldump from MySQL 8.0. Fixes #10804
|
@SunRunAway PTAL |
Codecov Report
@@ Coverage Diff @@
## master #10829 +/- ##
=========================================
Coverage 80.901% 80.901%
=========================================
Files 418 418
Lines 89450 89450
=========================================
Hits 72366 72366
Misses 11853 11853
Partials 5231 5231 |
|
Another error occured, mysqldump -h127.0.0.1 -P5000 -uroot test
-- MySQL dump 10.13 Distrib 8.0.15, for osx10.14 (x86_64)
--
-- Host: 127.0.0.1 Database: test
-- ------------------------------------------------------
-- Server version 8.0.15
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
SET NAMES utf8mb4 ;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `t`
--
DROP TABLE IF EXISTS `t`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
SET character_set_client = utf8mb4 ;
CREATE TABLE `t` (
`id` int(11) DEFAULT NULL,
`cnt` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `t`
--
LOCK TABLES `t` WRITE;
/*!40000 ALTER TABLE `t` DISABLE KEYS */;
INSERT INTO `t` VALUES (4,1),(3,2),(1,4),(2,2),(1,1),(1,5),(2,6),(2,1),(1,3),(3,4),(4,5),(3,6);
/*!40000 ALTER TABLE `t` ENABLE KEYS */;
UNLOCK TABLES;
--
-- Table structure for table `t1`
--
DROP TABLE IF EXISTS `t1`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
SET character_set_client = utf8mb4 ;
CREATE TABLE `t1` (
`id` int(11) DEFAULT NULL,
`sex` char(1) COLLATE utf8mb4_general_ci DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `t1`
--
LOCK TABLES `t1` WRITE;
/*!40000 ALTER TABLE `t1` DISABLE KEYS */;
INSERT INTO `t1` VALUES (1,'F'),(2,'F'),(3,'F'),(3,'M'),(4,'M'),(5,'M'),(6,'F');
/*!40000 ALTER TABLE `t1` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;
-- Dump completed on 2019-06-18 10:32:25
[sunrunaway:~]$ mysqldump -h127.0.0.1 -P4000 -uroot test
-- MySQL dump 10.13 Distrib 8.0.15, for osx10.14 (x86_64)
--
-- Host: 127.0.0.1 Database: test
-- ------------------------------------------------------
-- Server version 5.7.25-TiDB-None
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
SET NAMES utf8mb4 ;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `t`
--
DROP TABLE IF EXISTS `t`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
SET character_set_client = utf8mb4 ;
CREATE TABLE `t` (
`id` int(11) DEFAULT NULL,
`cnt` int(11) DEFAULT NULL,
KEY `id` (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `t`
--
LOCK TABLES `t` WRITE;
/*!40000 ALTER TABLE `t` DISABLE KEYS */;
INSERT INTO `t` VALUES (4,1),(3,2),(1,4),(2,2),(1,1),(1,5),(2,6),(2,1),(4,1),(3,2),(1,4),(2,2),(1,1),(1,5),(2,6),(2,1),(4,1),(3,2),(1,4),(2,2),(1,1),(1,5),(2,6),(2,1);
/*!40000 ALTER TABLE `t` ENABLE KEYS */;
UNLOCK TABLES;
mysqldump: Couldn't execute 'SELECT COLUMN_NAME, JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"') FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME = 'test' AND TABLE_NAME = 't';': Table 'information_schema.COLUMN_STATISTICS' doesn't exist (1146) |
|
(Editing my previous comment - I was looking at the wrong schema). I think we can fix this by creating a new empty |
morgo
changed the title
|
@SunRunAway PTAL again. Thanks :-) |
|
for problem 2:
Seems MySQL(root@localhost:information_schema) > show create view column_statistics\G
*************************** 1. row ***************************
View: COLUMN_STATISTICS
Create View: CREATE ALGORITHM=UNDEFINED DEFINER=`mysql.infoschema`@`localhost` SQL SECURITY DEFINER VIEW `COLUMN_STATISTICS` AS select `mysql`.`column_statistics`.`schema_name` AS `SCHEMA_NAME`,`mysql`.`column_statistics`.`table_name` AS `TABLE_NAME`,`mysql`.`column_statistics`.`column_name` AS `COLUMN_NAME`,`mysql`.`column_statistics`.`histogram` AS `HISTOGRAM` from `mysql`.`column_statistics` where can_access_table(`mysql`.`column_statistics`.`schema_name`,`mysql`.`column_statistics`.`table_name`)
character_set_client: utf8
collation_connection: utf8_general_ci
1 row in set (0.00 sec)
MySQL(root@localhost:information_schema) > desc column_statistics;
+-------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| SCHEMA_NAME | varchar(64) | NO | | NULL | |
| TABLE_NAME | varchar(64) | NO | | NULL | |
| COLUMN_NAME | varchar(64) | NO | | NULL | |
| HISTOGRAM | json | NO | | NULL | |
+-------------+-------------+------+-----+---------+-------+
4 rows in set (0.01 sec)An ideal fix on TiDB is to return the result from |
zz-jason
reviewed
Jun 19, 2019
|
Yes, an ideal fix is to format the tidb stats. |
In MySQL 8.0 I will take a look to see if it makes sense to map to |
|
I took a look at populating the histograms table. For some background context:
The histogram is then visible in How mysqldump uses information_schema.column_statisticsmysqldump queries for the presence of histograms on the schema+table. It only needs to know the column the histogram is on and the number of buckets: It uses this information to write the following to the dump file: This statement is a parse error in TiDB, so if the My recommendation
|
|
@zz-jason @lamxTyler @SunRunAway PTAL again |
|
/run-all-tests |
|
/run-unit-test |
alivxxx
added
status/LGT2
|
/run-all-tests |
|
/run-all-tests |
|
/run-all-tests |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Fixes #10804
What is changed and how it works?
Problem 1
The mysqldump client in MySQL 8.0 issues an SQL command to the server to stop caching data dictionary stats (a new feature of MySQL 8.0):
This is backward compatible for MySQL, since it uses the conditional version syntax to only work in 8.0 and above. At least currently in TiDB, the conditional version syntax is ignored, and thus this statement attempts to execute, resulting in an error preventing backup:
Problem 2
The mysqldump client queries the table
information_schema.column_statisticsto retrieve histogram information. This PR provides an empty table of the correct structure, which seems to satisfy the dump client.Check List
Tests
Code changes
Side effects
Related changes