Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coprocessor/codec/mysql/time: Fix parse datetime from string #3589

Merged
merged 17 commits into from Sep 25, 2018

Conversation

@spongedu
Copy link
Collaborator

commented Sep 9, 2018

Signed-off-by: spongedc duchuangucas@live.com

What have you changed? (mandatory)

  1. Fix #3564

  2. Currently there's some bugs when tikv parse datetime value from string. For example, In MySQL:

mysql> select cast("20170118.111" as datetime);
+----------------------------------+
| cast("20170118.111" as datetime) |
+----------------------------------+
| 2017-01-18 11:01:00              |
+----------------------------------+
1 row in set (0.03 sec)

This pr fixes these bugs according to implementation in TiDB. Make behavior consistent in both systems.

What are the type of the changes? (mandatory)

  • Bug fix

How has this PR been tested? (mandatory)

Unittest

Does this PR affect documentation (docs/docs-cn) update? (mandatory)

Does this PR affect tidb-ansible update? (mandatory)

Refer to a related PR or issue link (optional)

Benchmark result if necessary (optional)

Add a few positive/negative examples (optional)

Signed-off-by: spongedc <duchuangucas@live.com>
@sre-bot

This comment has been minimized.

Copy link
Collaborator

commented Sep 9, 2018

Hi contributor, thanks for your PR.

This patch needs to be approved by someone of admins. They should reply with "/ok-to-test" to accept this PR for running test automatically.

@hicqu

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2018

nice work. LGTM.

Makefile Outdated
@@ -111,3 +111,6 @@ clean:

expression: format clippy
LOG_LEVEL=ERROR RUST_BACKTRACE=1 cargo test --features "${ENABLE_FEATURES}" "coprocessor::dag::expr" -- --nocapture

time: format clippy

This comment has been minimized.

Copy link
@breeswish

breeswish Sep 10, 2018

Member

We don't need to introduce this because it is rarely used.

This comment has been minimized.

Copy link
@spongedu

spongedu Sep 10, 2018

Author Collaborator

ok

let mut need_adjust = false;
let parts = Time::parse_datetime_format(s);
let mut has_hhmmss = false;

This comment has been minimized.

This comment has been minimized.

Copy link
@spongedu

spongedu Sep 10, 2018

Author Collaborator

ok, I'll fix this issue too

@spongedu

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 10, 2018

It seems that currently there also something wrong in TiDB when parsing string to datetime. I've fixed these int #7654. I'll update this pr later for the newest changes. @breeswish

spongedu and others added 5 commits Sep 10, 2018
Signed-off-by: spongedc <duchuangucas@live.com>
Signed-off-by: spongedc <duchuangucas@live.com>
@spongedu

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 11, 2018

@breeswish @hicqu Currently the implementation of GREATEST, LEAST is wrong on illegal inputs. I've fixed time related part in this pr. I'll review the rest parts in another pr.

spongedu and others added 4 commits Sep 11, 2018
Signed-off-by: spongedc <duchuangucas@live.com>
@DorianZheng DorianZheng requested a review from hicqu Sep 14, 2018
@DorianZheng

This comment has been minimized.

Copy link
Contributor

commented Sep 14, 2018

@hicqu ping

@hicqu

This comment has been minimized.

Copy link
Contributor

commented Sep 17, 2018

LGTM.

spongedu added 3 commits Sep 18, 2018
Signed-off-by: spongedc <duchuangucas@live.com>
@huachaohuang

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2018

@solotzg PTAL

@solotzg

This comment has been minimized.

Copy link
Contributor

commented Sep 22, 2018

LGTM.

@morgo

This comment has been minimized.

Copy link

commented Sep 25, 2018

I am hoping this will fix bugs like this one?

@huachaohuang

This comment has been minimized.

Copy link
Contributor

commented Sep 25, 2018

@morgo seems related, you can give it a try and let us know if it fixes your problem :)

@huachaohuang huachaohuang merged commit b03492e into tikv:master Sep 25, 2018
3 checks passed
3 checks passed
DCO All commits are signed off!
Details
ci/circleci: test Your tests passed on CircleCI!
Details
jenkins-ci-tikv/build Jenkins job succeeded.
Details
@spongedu

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 26, 2018

@morgo I test 2010-11-7 1:0:8.000000000 and it works.

@spongedu spongedu deleted the spongedu:0909 branch Sep 26, 2018
@morgo

This comment has been minimized.

Copy link

commented Sep 27, 2018

@spongedu I checked just now against the latest server tarball. tikv:

./bin/tikv-server --version
TiKV 
Release Version:   2.1.0-rc.2
Git Commit Hash:   8eddb4c5199c3ce09cdc1517989db672cd388102
Git Commit Branch: master
UTC Build Time:    2018-09-26 08:26:35
Rust Version:      rustc 1.29.0-nightly (4f3c7a472 2018-07-17)

Still getting the same error:

mysql> delete from trips where start_date < '2017-01-01 00:00:00';
ERROR 1105 (HY000): other error: unknown error StringError("[src/coprocessor/dag/executor/mod.rs:239]: unknown error StringError(\"[src/coprocessor/codec/mysql/time/mod.rs:95]: \\\'2016-11-6 1:0:8.000000000\\\' is not a valid datetime\")")
@spongedu

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 27, 2018

wired... I'll look into this. @morgo

@morgo

This comment has been minimized.

Copy link

commented Sep 27, 2018

It looks to be timezone related. Let's continue the discussion in pingcap/tidb#7768

MyonKeminta added a commit to MyonKeminta/tikv that referenced this pull request Oct 15, 2018
commit 179d540
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Oct 10 15:10:21 2018 +0800

    Fix test

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 674ff1c
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Oct 10 14:51:50 2018 +0800

    Add more comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit cd66b77
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Oct 10 13:07:59 2018 +0800

    Remove debug code

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 2baaac4
Merge: 592fcfa d0929ad
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Oct 10 12:47:21 2018 +0800

    Merge branch 'master' of https://github.com/tikv/tikv into misono/leader-collector

commit 592fcfa
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Oct 10 12:45:06 2018 +0800

    add integrated test

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit e160a0c
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Tue Oct 9 15:47:15 2018 +0800

    refine naming; impl display

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 0957b2b
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Tue Oct 9 13:49:02 2018 +0800

    Refine fault tolerance

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 0164f74
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Oct 8 20:22:15 2018 +0800

    Add unit tests

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 5050e4f
Merge: 64521d1 268dfab
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Oct 8 14:15:41 2018 +0800

    Merge branch 'misono/more-observers' into misono/leader-collector

commit 268dfab
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Oct 8 14:09:41 2018 +0800

    Address comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit f55ca6f
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Oct 8 13:37:56 2018 +0800

    Address comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit a8d5e39
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 28 14:08:05 2018 +0800

    Address comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 64521d1
Merge: 14d9316 62108dc
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 28 13:29:25 2018 +0800

    Merge branch 'misono/more-observers' into misono/leader-collector

commit 62108dc
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 28 13:25:26 2018 +0800

    Address comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 14d9316
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 28 00:18:50 2018 +0800

    Fix region collection didn't start

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit a621d77
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Thu Sep 27 21:15:45 2018 +0800

    Use region change observer

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 08c6e52
Merge: 27e5bf0 2317511
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Thu Sep 27 21:11:36 2018 +0800

    Merge branch 'misono/more-observers' into misono/leader-collector

commit 2317511
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Sep 26 15:20:55 2018 +0800

    Address comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit beba0f8
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Sep 26 14:51:25 2018 +0800

    update observer name

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 06778e7
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Sep 26 14:16:08 2018 +0800

    Add tests

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit ce786c9
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Wed Sep 26 00:29:24 2018 +0800

    Fix build err

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 042bf7d
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Tue Sep 25 23:55:23 2018 +0800

    Add unit tests

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 372e21f
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Tue Sep 25 23:31:47 2018 +0800

    Observer all changes of regions

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit d14cb63
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Sep 24 01:20:34 2018 +0800

    Add region load observer

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 27e5bf0
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Sep 24 01:40:58 2018 +0800

    Add more comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 7905288
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Sep 24 01:20:34 2018 +0800

    Track region loading on tikv starting up

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit a3d1771
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sun Sep 23 23:07:19 2018 +0800

    use two maps in region collection

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit f3711a7
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sun Sep 23 19:19:46 2018 +0800

    Add tests; fix seek region bug

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit e42d3aa
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sun Sep 23 17:23:45 2018 +0800

    implement seek region

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 05ea7ff
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sat Sep 22 15:51:12 2018 +0800

    simplify code. no need to separate the sender part

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit b1aed40
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 18:12:16 2018 +0800

    Fix build error

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 80347de
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 17:39:52 2018 +0800

    Fix typo

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 25cc08d
Merge: 1d1034e bcaa0f8
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 17:26:47 2018 +0800

    Merge branch 'master' of https://github.com/tikv/tikv into misono/leader-collector

commit 1d1034e
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 17:24:39 2018 +0800

    partially implemented

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

Date:   Tue Sep 25 23:55:23 2018 +0800

    Add unit tests

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 372e21f
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Tue Sep 25 23:31:47 2018 +0800

    Observer all changes of regions

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit b03492e
Author: Du Chuan <duchuangucas@live.com>
Date:   Tue Sep 25 22:46:56 2018 +0800

    coprocessor/codec/mysql/time: Fix parse datetime from string (tikv#3589)

    Signed-off-by: spongedc <duchuangucas@live.com>

commit fbc18a1
Author: Wenlu Wang <kingwenlu@gmail.com>
Date:   Tue Sep 25 20:00:23 2018 +0800

    expression: add year buildin function (tikv#3622)

    Signed-off-by: 王文璐 <kingwenlu@gmail.com>

commit d14cb63
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Sep 24 01:20:34 2018 +0800

    Add region load observer

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 78eff19
Author: niedhui <niedhui@gmail.com>
Date:   Tue Sep 25 10:25:30 2018 +0800

    expression: add builtin truncate_int (tikv#3532)

    Signed-off-by: niedhui <niedhui@gmail.com>

commit 8458ce5
Author: Balaji R <rbalajis25@gmail.com>
Date:   Mon Sep 24 06:59:51 2018 +0000

    raftstore: add invalid propose metrics (tikv#3620)

    Signed-off-by: பாலாஜி ஜின்னா <rbalajis25@gmail.com>

commit 27e5bf0
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Sep 24 01:40:58 2018 +0800

    Add more comments

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 7905288
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Mon Sep 24 01:20:34 2018 +0800

    Track region loading on tikv starting up

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit a3d1771
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sun Sep 23 23:07:19 2018 +0800

    use two maps in region collection

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit f3711a7
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sun Sep 23 19:19:46 2018 +0800

    Add tests; fix seek region bug

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit e42d3aa
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sun Sep 23 17:23:45 2018 +0800

    implement seek region

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit eae3f9d
Author: Weizhen Wang <hawking.rei@gmail.com>
Date:   Sun Sep 23 01:09:24 2018 +0800

    refactor Month by using new handle_invalid_time_error (tikv#3615)

    Signed-off-by: Wangweizhen <hawking.rei@gmail.com>

commit 05ea7ff
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Sat Sep 22 15:51:12 2018 +0800

    simplify code. no need to separate the sender part

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit b4198ff
Author: Qinxuan Chen <koushiro.cqx@gmail.com>
Date:   Sat Sep 22 14:09:41 2018 +0800

    coprocessor/codec/mysql/duration: Fix micro_secs of Duration (tikv#3625)

    Signed-off-by: koushiro <koushiro.cqx@gmail.com>

commit 91dab79
Author: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Date:   Sat Sep 22 11:56:49 2018 +0800

    Fix typo (tikv#3628)

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit b1aed40
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 18:12:16 2018 +0800

    Fix build error

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit 80347de
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 17:39:52 2018 +0800

    Fix typo

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit c6b5a33
Author: Lilian Lee <lilin@pingcap.com>
Date:   Fri Sep 21 17:37:19 2018 +0800

    docs/tools: add PD Recover user guide (tikv#3626)

    * docs/tools: add PD Recover user guide

    Signed-off-by: lilin90 <lilin@pingcap.com>

    * docs/tools: address the comment

    Signed-off-by: lilin90 <lilin@pingcap.com>

commit 25cc08d
Merge: 1d1034e bcaa0f8
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 17:26:47 2018 +0800

    Merge branch 'master' of https://github.com/tikv/tikv into misono/leader-collector

commit 1d1034e
Author: MyonKeminta <MyonKeminta@users.noreply.github.com>
Date:   Fri Sep 21 17:24:39 2018 +0800

    partially implemented

    Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

commit bcaa0f8
Author: Neil Shen <overvenus@gmail.com>
Date:   Fri Sep 21 14:36:10 2018 +0800

    worker: make run as a default method (tikv#3593)

    Signed-off-by: Neil Shen <overvenus@gmail.com>

commit 244b55a
Author: haoxiang <linhaoxiang32@gmail.com>
Date:   Fri Sep 21 07:11:46 2018 +0800

    expression: add builtin sha1 (tikv#3612)

    * expression: add builtin sha1

    Signed-off-by: haoxiang47 <linhaoxiang32@gmail.com>

    * update api result_str -> result

    Signed-off-by: haoxiang47 <linhaoxiang32@gmail.com>

    * revert sha1 api from result -> result_str

    Signed-off-by: haoxiang47 <linhaoxiang32@gmail.com>

commit dfdacc8
Author: Wenxuan Shi <hi@breeswish.org>
Date:   Thu Sep 20 21:50:22 2018 +0800

    Move coprocessor to read pool (tikv#3515)

    Signed-off-by: Breezewish <breezewish@pingcap.com>

commit ad1d9f3
Author: zhangjinpeng1987 <zhangjinpeng@pingcap.com>
Date:   Wed Sep 19 01:06:50 2018 +0800

    fix from_hex (tikv#2873)

    Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>

commit 2972f3e
Author: Malong <hello@sllt.me>
Date:   Mon Sep 17 22:28:07 2018 +0800

    coprocessor/dag/expr: add un-pushed builtin UDFs(Log1Arg, Log2Args) (tikv#3603)

    Signed-off-by: sllt <hello@sllt.me>

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.