Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the SQL join query partition error when deal with too small testdata #207

Merged
merged 1 commit into from Mar 31, 2018

Conversation

@jiayuasu
Copy link
Member

commented Mar 31, 2018

This PR fixed Issue #193.

Bug description: The original code cannot well handle too small test data for spatial join query. When the user provides some super small test data, (2 or 3 rows per DataFrame), the partition number logic will jump in a wrong fallback setting and eventually pick "-1" as the partition number.

Solution: use a smarter Partition Number Optimizer to select a proper partition number for both input tables of a spatial join query.

@jiayuasu jiayuasu merged commit 3cc5bb3 into DataSystemsLab:master Mar 31, 2018

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
jiayuasu added a commit that referenced this pull request Mar 31, 2018
Merge pull request #207 from jiayuasu/master
Fix the SQL join query partition error when deal with too small testdata
jiayuasu added a commit that referenced this pull request Mar 31, 2018
Merge pull request #207 from jiayuasu/master
Fix the SQL join query partition error when deal with too small testdata
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.