expression: implement is_ipv4 and is_ipv6 function pushdown #41173

AntiTopQuark · 2023-02-07T15:44:34Z

What problem does this PR solve?

Issue Number: close #41172
related issue: pingcap/tiflash#6773
Problem Summary:
implement is_ipv4 and is_ipv6 function pushdown

What is changed and how it works?

open pushdown of is_ipv4 and is_ipv6
add some test

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Side effects

Performance regression: Consumes more CPU
Performance regression: Consumes more Memory
Breaking backward compatibility

Documentation

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

implement is_ipv4 and is_ipv6 function pushdown to tiflash

support is_ipv4 and is_ipv6 expr to pushdown to tiflash

ti-chi-bot · 2023-02-07T15:44:36Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

LittleFall
SeaRise

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

ti-chi-bot · 2023-02-07T15:44:45Z

Hi @AntiTopQuark. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

AntiTopQuark · 2023-02-12T00:01:16Z

/run-all-tests

hawkingrei · 2023-02-12T14:02:38Z

/ok-to-test

SeaRise · 2023-02-13T01:49:51Z

planner/core/integration_test.go

+	tk.MustExec("insert into t values('123.123.123.123', 'F746:C349:48E3:22F2:81E0:0EA8:E7B6:8286')")
+	tk.MustExec("insert into t values('0.0.0.0', '0000:0000:0000:0000:0000:0000:0000:0000')")
+	tk.MustExec("insert into t values('127.0.0.1', '2001:0:2851:b9f0:6d:2326:9036:f37a')")
+	tk.MustExec("insert into t values('192.168.0.0/10', 'fe80::2dc3:25a5:49a1:6002%24')")
+	tk.MustExec("insert into t values('192.168.99.22.123', '4207:A33A:58D3:F2C3:8EDC:A548:3EC7:0D00:0D00')")
+	tk.MustExec("insert into t values('999.999.999.999', '4207:A33A:58D3:F2C3:8EDC:A548::0D00')")
+	tk.MustExec("insert into t values('3.2.1.', '4207::::8EDC:A548:3EC7:0D00')")
+	tk.MustExec("insert into t values('3..2.1', '4207:::::A548:3EC7:0D00')")
+	tk.MustExec("insert into t values('...', '::::::')")
+	tk.MustExec("insert into t values('4556456', '4556456')")
+	tk.MustExec("insert into t values('ajdjioa', 'ajdjioa')")
+	tk.MustExec("insert into t values('', '')")
+	tk.MustExec("insert into t values(null,null)")


Seems useless here

SeaRise · 2023-02-13T01:50:11Z

planner/core/integration_test.go

+	tk.MustExec("insert into t values('123.123.123.123', 'F746:C349:48E3:22F2:81E0:0EA8:E7B6:8286')")
+	tk.MustExec("insert into t values('0.0.0.0', '0000:0000:0000:0000:0000:0000:0000:0000')")
+	tk.MustExec("insert into t values('127.0.0.1', '2001:0:2851:b9f0:6d:2326:9036:f37a')")
+	tk.MustExec("insert into t values('192.168.0.0/10', 'fe80::2dc3:25a5:49a1:6002%24')")
+	tk.MustExec("insert into t values('192.168.99.22.123', '4207:A33A:58D3:F2C3:8EDC:A548:3EC7:0D00:0D00')")
+	tk.MustExec("insert into t values('999.999.999.999', '4207:A33A:58D3:F2C3:8EDC:A548::0D00')")
+	tk.MustExec("insert into t values('3.2.1.', '4207::::8EDC:A548:3EC7:0D00')")
+	tk.MustExec("insert into t values('3..2.1', '4207:::::A548:3EC7:0D00')")
+	tk.MustExec("insert into t values('...', '::::::')")
+	tk.MustExec("insert into t values('4556456', '4556456')")
+	tk.MustExec("insert into t values('ajdjioa', 'ajdjioa')")
+	tk.MustExec("insert into t values('', '')")
+	tk.MustExec("insert into t values(null,null)")


planner/core/integration_test.go

AntiTopQuark · 2023-02-14T15:29:26Z

/reset

AntiTopQuark · 2023-02-15T00:34:40Z

/test all

AntiTopQuark · 2023-02-15T00:35:57Z

/cc @ywqzzy

AntiTopQuark · 2023-02-15T02:11:57Z

/retest

LittleFall · 2023-02-15T03:25:47Z

/retest

ywqzzy · 2023-02-15T05:09:58Z

/retest

guo-shaoge · 2023-02-15T07:40:36Z

/merge

ti-chi-bot · 2023-02-15T07:40:39Z

This pull request has been accepted and is ready to merge.

Commit hash: 2780503

SeaRise · 2023-02-15T07:46:11Z

/retest

guo-shaoge · 2023-02-15T08:12:12Z

Please fix case: TestIsIPv6ToTiFlash @AntiTopQuark

25665 [2023/02/15 15:56:48.047 +08:00] [WARN] [2pc.go:1738] ["schemaLeaseChecker is not set for this transaction"] [sessionID=10] [startTS=439470734192672770] [checkTS=439470734192672771]
25666     result.go:131:
25667             Error Trace:    /home/jenkins/.tidb/tmp/63a9840cd0739f2c243bb46478607469/sandbox/linux-sandbox/7660/execroot/__main__/bazel-out/k8-fastbuild/bin/planner/core/core_test_/core_test.runfiles/__main__/planner/core/result.go:131
25668                                         /home/jenkins/.tidb/tmp/63a9840cd0739f2c243bb46478607469/sandbox/linux-sandbox/7660/execroot/__main__/bazel-out/k8-fastbuild/bin/planner/core/core_test_/core_test.runfiles/__main__/planner/core/integration_test.go:8523
25669             Error:          Not equal:
25670                             expected: "[[TableReader_9 root data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [  └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]"
25671                             actual  : "[[TableReader_9 root MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [  └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]"
25672
25673                             Diff:
25674                             --- Expected
25675                             +++ Actual
25676                             @@ -1 +1 @@
25677                             -[[TableReader_9 root data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [  └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]
25678                             +[[TableReader_9 root MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [  └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]
25679             Test:           TestIsIPv6ToTiFlash
25680             Messages:       sql:explain select is_ipv6(v6) from t;, args:[]

AntiTopQuark · 2023-02-15T08:40:09Z

/reset

AntiTopQuark · 2023-02-15T09:55:10Z

/reset

guo-shaoge · 2023-02-15T09:58:30Z

Looks like case still failed: @AntiTopQuark

[checkTS=439472502008184835]
    result.go:131:
            Error Trace:    /home/jenkins/.tidb/tmp/63a9840cd0739f2c243bb46478607469/sandbox/linux-sandbox/7919/execroot/__main__/bazel-out/k8-fastbuild/bin/planner/ core/core_test_/core_test.runfiles/__main__/planner/core/result.go:131
                                        /home/jenkins/.tidb/tmp/63a9840cd0739f2c243bb46478607469/sandbox/linux-sandbox/7919/execroot/__main__/bazel-out/k8-fastbuild/ bin/planner/core/core_test_/core_test.runfiles/__main__/planner/core/integration_test.go:8523
            Error:          Not equal:
                            expected: "[[TableReader_9 root   MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [     └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]"
                            actual  : "[[TableReader_9 root MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [       └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]"

                            Diff:
                            --- Expected
                            +++ Actual
                            @@ -1 +1 @@
                            -[[TableReader_9 root   MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [               └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]                                                                    +[[TableReader_9 root MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [  └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]
            Test:           TestIsIPv6ToTiFlash
            Messages:       sql:explain select is_ipv6(v6) from t;, args:[]

guo-shaoge · 2023-02-15T09:59:16Z

You can run go test . --tags=intest -run TestIsIPv6ToTiFlash in your local env, and make sure case is ok. @AntiTopQuark

AntiTopQuark · 2023-02-15T14:51:46Z

/test check-dev2

ti-chi-bot · 2023-02-15T14:52:03Z

@AntiTopQuark: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test build
/test check-dev
/test check-dev2
/test mysql-test
/test unit-test

Use /test all to run all jobs.

In response to this:

/test check-dev2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

AntiTopQuark · 2023-02-15T14:55:14Z

You can run go test . --tags=intest -run TestIsIPv6ToTiFlash in your local env, and make sure case is ok. @AntiTopQuark

thank u very much, now it is okey.

AntiTopQuark · 2023-02-15T14:55:35Z

Looks like case still failed: @AntiTopQuark

[checkTS=439472502008184835]
    result.go:131:
            Error Trace:    /home/jenkins/.tidb/tmp/63a9840cd0739f2c243bb46478607469/sandbox/linux-sandbox/7919/execroot/__main__/bazel-out/k8-fastbuild/bin/planner/ core/core_test_/core_test.runfiles/__main__/planner/core/result.go:131
                                        /home/jenkins/.tidb/tmp/63a9840cd0739f2c243bb46478607469/sandbox/linux-sandbox/7919/execroot/__main__/bazel-out/k8-fastbuild/ bin/planner/core/core_test_/core_test.runfiles/__main__/planner/core/integration_test.go:8523
            Error:          Not equal:
                            expected: "[[TableReader_9 root   MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [     └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]"
                            actual  : "[[TableReader_9 root MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [       └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]"

                            Diff:
                            --- Expected
                            +++ Actual
                            @@ -1 +1 @@
                            -[[TableReader_9 root   MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [               └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]                                                                    +[[TableReader_9 root MppVersion: 1, data:ExchangeSender_8] [└─ExchangeSender_8 mpp[tiflash] ExchangeType: PassThrough] [  └─Projection_4 mpp[tiflash] is_ipv6(test.t.v6)->Column#4] [    └─TableFullScan_7 mpp[tiflash] keep order:false, stats:pseudo]]
            Test:           TestIsIPv6ToTiFlash
            Messages:       sql:explain select is_ipv6(v6) from t;, args:[]

thank u very much~

AntiTopQuark · 2023-02-15T14:59:51Z

it looks like that the test named "check_dev_2" is unstable .
The same problem also occurs with another PR :#41453
https://do.pingcap.net/jenkins/blue/organizations/jenkins/pingcap%2Ftidb%2Fghpr_check2/detail/ghpr_check2/14262/

AntiTopQuark · 2023-02-15T15:01:01Z

/test check-dev2

AntiTopQuark · 2023-02-15T15:05:56Z

**idc-jenkins-ci-tidb/check_dev_2 ** Pending — Jenkins job enqueued. BaseSHA:3d8b7c21f15b1d45d05169d1c81cd5e42233c894

…41173) close pingcap#41172

AntiTopQuark added 2 commits February 7, 2023 23:38

support is_ipv4 and is_ipv6 expr to tiflash

1eadca1

support is_ipv4 and is_ipv6 expr to pushdown to tiflash

add integration test for is_ipv4 and is_ipv6

c615b3e

ti-chi-bot added release-note size/L Denotes a PR that changes 100-499 lines, ignoring generated files. needs-ok-to-test labels Feb 7, 2023

AntiTopQuark added 2 commits February 8, 2023 10:09

Merge branch 'master' into master

8015742

Merge branch 'pingcap:master' into master

3a32ae4

ti-chi-bot added ok-to-test and removed needs-ok-to-test labels Feb 12, 2023

SeaRise reviewed Feb 13, 2023

View reviewed changes

AntiTopQuark added 2 commits February 13, 2023 22:59

Update expr_to_pb_test.go

aa359fe

Update integration_test.go

e3d52f4

ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 13, 2023

SeaRise approved these changes Feb 13, 2023

View reviewed changes

ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Feb 13, 2023

AntiTopQuark added 2 commits February 14, 2023 22:58

Merge branch 'master' into master

21f3125

Update integration_test.go

18147aa

Update expr_to_pb_test.go

2780503

ti-chi-bot requested a review from ywqzzy February 15, 2023 00:35

LittleFall approved these changes Feb 15, 2023

View reviewed changes

ti-chi-bot removed the status/LGT1 Indicates that a PR has LGTM 1. label Feb 15, 2023

ti-chi-bot added the status/LGT2 Indicates that a PR has LGTM 2. label Feb 15, 2023

guo-shaoge approved these changes Feb 15, 2023

View reviewed changes

ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Feb 15, 2023

Update integration_test.go

c17346b

AntiTopQuark added 2 commits February 15, 2023 17:13

Update integration_test.go

32a8952

Update integration_test.go

f617a27

Update integration_test.go

30158e2

AntiTopQuark added 2 commits February 15, 2023 19:16

Update integration_test.go

957ebd9

Merge branch 'pingcap:master' into master

9c5229a

ti-chi-bot merged commit aa3b57c into pingcap:master Feb 15, 2023

ghazalfamilyusa pushed a commit to ghazalfamilyusa/tidb that referenced this pull request Feb 15, 2023

expression: implement is_ipv4 and is_ipv6 function pushdown (pingcap#…

2de1d17

…41173) close pingcap#41172

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

expression: implement is_ipv4 and is_ipv6 function pushdown #41173

expression: implement is_ipv4 and is_ipv6 function pushdown #41173

AntiTopQuark commented Feb 7, 2023 •

edited

ti-chi-bot commented Feb 7, 2023 •

edited

ti-chi-bot commented Feb 7, 2023

AntiTopQuark commented Feb 12, 2023

hawkingrei commented Feb 12, 2023

SeaRise Feb 13, 2023

SeaRise Feb 13, 2023

AntiTopQuark commented Feb 14, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

LittleFall commented Feb 15, 2023

ywqzzy commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023

ti-chi-bot commented Feb 15, 2023

SeaRise commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023 •

edited

AntiTopQuark commented Feb 15, 2023

ti-chi-bot commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

expression: implement is_ipv4 and is_ipv6 function pushdown #41173

expression: implement is_ipv4 and is_ipv6 function pushdown #41173

Conversation

AntiTopQuark commented Feb 7, 2023 • edited

What problem does this PR solve?

What is changed and how it works?

Check List

Release note

ti-chi-bot commented Feb 7, 2023 • edited

ti-chi-bot commented Feb 7, 2023

AntiTopQuark commented Feb 12, 2023

hawkingrei commented Feb 12, 2023

SeaRise Feb 13, 2023

Choose a reason for hiding this comment

SeaRise Feb 13, 2023

Choose a reason for hiding this comment

AntiTopQuark commented Feb 14, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

LittleFall commented Feb 15, 2023

ywqzzy commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023

ti-chi-bot commented Feb 15, 2023

SeaRise commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023

guo-shaoge commented Feb 15, 2023 • edited

AntiTopQuark commented Feb 15, 2023

ti-chi-bot commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 15, 2023

AntiTopQuark commented Feb 7, 2023 •

edited

ti-chi-bot commented Feb 7, 2023 •

edited

guo-shaoge commented Feb 15, 2023 •

edited