Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support tidb chunk's data format and columnar execution in Spark #1220

Merged
merged 16 commits into from Nov 29, 2019

Conversation

@zhexuany
Copy link
Member

zhexuany commented Nov 16, 2019

What problem does this PR solve?

In this PR, we support tidb chunk encoding. We also switch spark's execution model from row-wise to columnar.

@zhexuany zhexuany force-pushed the support_tidb_arrow branch from 0f71870 to e28272a Nov 17, 2019
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 17, 2019

/run-all-tests

1 similar comment
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 18, 2019

/run-all-tests

core/pom.xml Show resolved Hide resolved
@zhexuany zhexuany changed the title Support tidb arrow Support tidb chunk's data format and columnar execution in Spark Nov 18, 2019
@zhexuany zhexuany force-pushed the support_tidb_arrow branch from e7abc21 to 03bccc0 Nov 19, 2019
@zhexuany zhexuany force-pushed the support_tidb_arrow branch 2 times, most recently from ea9603b to c7a4009 Nov 19, 2019
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 19, 2019

/run-all-tests

1 similar comment
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 19, 2019

/run-all-tests

@zhexuany zhexuany force-pushed the support_tidb_arrow branch 3 times, most recently from ead41d2 to d6dd4b6 Nov 20, 2019
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 21, 2019

/run-all-tests

@zhexuany zhexuany force-pushed the support_tidb_arrow branch 2 times, most recently from 30c687d to 2963038 Nov 22, 2019
@zhexuany zhexuany force-pushed the support_tidb_arrow branch from 1bea2a5 to 9048feb Nov 26, 2019
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 26, 2019

/run-all-tests tidb=v3.0.5

@zhexuany zhexuany force-pushed the support_tidb_arrow branch from 1ea1304 to 5c3bfb9 Nov 26, 2019
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 26, 2019

/run-all-tests tidb=v3.0.5

Copy link
Collaborator

marsishandsome left a comment

suggest to merge this PR after #1238

core/pom.xml Outdated Show resolved Hide resolved
core/pom.xml Outdated Show resolved Hide resolved
@zhexuany zhexuany force-pushed the support_tidb_arrow branch from 41fcae8 to 23092ec Nov 28, 2019
@zhexuany

This comment has been minimized.

Copy link
Member Author

zhexuany commented Nov 28, 2019

/run-all-tests mode=full

@birdstorm

This comment has been minimized.

Copy link
Member

birdstorm commented Nov 28, 2019

LGTM if all tests are passed. @zanmato1984 PTAL

Copy link
Member

birdstorm left a comment

LGTM

@ilovesoup ilovesoup merged commit 56fc5b6 into master Nov 29, 2019
3 checks passed
3 checks passed
idc-jenkins-ci-tispark/build Jenkins job succeeded.
Details
idc-jenkins-ci-tispark/integration-test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details
@zhexuany zhexuany deleted the support_tidb_arrow branch Nov 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.