Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add load data fail log #40

Merged
merged 6 commits into from
Feb 1, 2021
Merged

add load data fail log #40

merged 6 commits into from
Feb 1, 2021

Conversation

xiongjiwei
Copy link

@xiongjiwei xiongjiwei commented Jan 19, 2021

What problem does this PR solve?

add log when loading data with errors

a new session variable tidb_load_broken_data= on (default) | off.
if tidb_load_broken_data is on, load data will try to fix broken data in the file and insert it into the table.
if tidb_load_broken_data is off, load data will ignore broken data. (PBOC use)

table schema

create table t(a int, b int);

data in file

1,2
2,3,,
1,
a,2

1,3

load data log

time="2021-01-19T14:41:41+08:00" level=info msg="load file [tidb.out], line 2: [2,3,,] has too much columns"
time="2021-01-19T14:41:41+08:00" level=info msg="load file [tidb.out], line 3: [1,] [types:1292]Truncated incorrect FLOAT value: ''"
time="2021-01-19T14:41:41+08:00" level=info msg="load file [tidb.out], line 4: [a,2] [types:1292]Truncated incorrect FLOAT value: 'a'"
time="2021-01-19T14:41:41+08:00" level=info msg="load file [tidb.out], line 5: [] is missing columns"

if tidb_load_broken_data=on

mysql root@127.0.0.1:test> select * from t;
+---+--------+
| a | b      |
+---+--------+
| 1 | 2      |
| 2 | 3      |
| 1 | 0      |
| 0 | 2      |
| 0 | <null> |
| 1 | 3      |
+---+--------+
6 rows in set

if tidb_load_broken_data=off

mysql root@127.0.0.1:test> select * from t;
+---+---+
| a | b |
+---+---+
| 1 | 2 |
| 1 | 3 |
+---+---+
2 rows in set

@xiongjiwei
Copy link
Author

@bb7133 PTAL

@bb7133
Copy link
Owner

bb7133 commented Feb 1, 2021

LGTM

@bb7133 bb7133 merged commit 9d177c1 into bb7133:4.0-itai Feb 1, 2021
@xiongjiwei xiongjiwei deleted the load-data branch October 28, 2021 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants