Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

diff: add collation config #75

Merged
merged 6 commits into from Sep 30, 2018
Merged

Conversation

WangXiangUSTC
Copy link
Contributor

@WangXiangUSTC WangXiangUSTC commented Sep 29, 2018

for example, we execute these sqls in mysql and tidb:

CREATE TABLE `test_collation` (  `name` varchar(24) DEFAULT NULL );
insert into test_collation values("A");
insert into test_collation values("D");
insert into test_collation values("C");
insert into test_collation values("b")

show create table in mysql:

mysql> show create table test_collation \G
*************************** 1. row ***************************
       Table: test_colltion
Create Table: CREATE TABLE `test_colltion` (
  `name` varchar(24) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
1 row in set (0.00 sec)

show create table in tidb:

mysql> show create table test_collation \G
*************************** 1. row ***************************
       Table: test_colltion
Create Table: CREATE TABLE `test_collation ` (
  `name` varchar(24) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin
1 row in set (0.01 sec)

then we select the max and min value:
in mysql:

mysql> select min(name), max(name) from test_collation;
+-----------+-----------+
| min(name) | max(name) |
+-----------+-----------+
| A         | D         |
+-----------+-----------+
1 row in set (0.00 sec)

in tidb:

mysql> select min(name), max(name) from test_collation;
+-----------+-----------+
| min(name) | max(name) |
+-----------+-----------+
| A         | b         |
+-----------+-----------+
1 row in set (0.01 sec)

we can see the max(name) is not equal, but we can use collation to let the result equal, execute this sql in mysql:

select min(name collate latin1_bin), max(name collate latin1_bin) from test_collation;
+------------------------------+------------------------------+
| min(name collate latin1_bin) | max(name collate latin1_bin) |
+------------------------------+------------------------------+
| A                            | b                            |
+------------------------------+------------------------------+
1 row in set (0.00 sec)

@@ -139,7 +139,7 @@ func GetRowCount(ctx context.Context, db *sql.DB, schemaName string, tableName s
}

// GetRandomValues returns some random value of a column.
func GetRandomValues(ctx context.Context, db *sql.DB, schemaName, table, column string, num int64, min, max interface{}, limitRange string) ([]interface{}, error) {
func GetRandomValues(ctx context.Context, db *sql.DB, schemaName, table, column string, num int64, min, max interface{}, limitRange string, collation string) ([]interface{}, error) {
/*
example:
mysql> SELECT `id` FROM (SELECT `id` FROM `test`.`test` WHERE `id` > 0 AND `id` < 100 AND true ORDER BY RAND() LIMIT 3)rand_tmp ORDER BY `id`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update example

@@ -272,7 +276,7 @@ func GetCRC32Checksum(ctx context.Context, db *sql.DB, schemaName, tableName str

query := fmt.Sprintf("SELECT BIT_XOR(CAST(CRC32(CONCAT_WS(',', %s, CONCAT(%s)))AS UNSIGNED)) AS checksum FROM `%s`.`%s` WHERE %s;",
strings.Join(columnNames, ", "), strings.Join(columnIsNull, ", "), schemaName, tableName, limitRange)
log.Debugf("checksum sql: %s, args: %v", query, args)
log.Infof("checksum sql: %s, args: %v", query, args)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is info better?

@IANTHEREAL
Copy link
Collaborator

Rest LGTM

@IANTHEREAL
Copy link
Collaborator

@csuzhangxc PTAL

@IANTHEREAL
Copy link
Collaborator

LGTM

@IANTHEREAL IANTHEREAL merged commit 8eb41d8 into pingcap:master Sep 30, 2018
@WangXiangUSTC WangXiangUSTC deleted the xiang/collate branch January 8, 2019 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants