Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(stdlib): panic in chunking outer joins #5404

Merged
merged 1 commit into from
Mar 30, 2023
Merged

Conversation

mhilton
Copy link
Contributor

@mhilton mhilton commented Mar 29, 2023

An outer join that creates large tables (more than 3000 rows) caused a panic when splitting the resulting table into chunks. This was due to the stop parameter becoming lower than that start parameter, attempting to use nonsensical ranges. Tables larger than 2000 rows also potentially dropped rows.

The fix is to ensure that the stop parameter is set relative to the start.

Checklist

Dear Author 馃憢, the following checks should be completed (or explicitly dismissed) before merging.

  • 鉁忥笍 Write a PR description, regardless of triviality, to include the value of this PR
  • 馃敆 Reference related issues
  • 馃弮 Test cases are included to exercise the new code
  • 馃И If new packages are being introduced to stdlib, link to Working Group discussion notes and ensure it lands under experimental/
  • 馃摉 If language features are changing, ensure docs/Spec.md has been updated

Dear Reviewer(s) 馃憢, you are responsible (among others) for ensuring the completeness and quality of the above before approval.

An outer join that creates large tables (more than 3000 rows) caused
a panic when splitting the resulting table into chunks. This was
due to the stop parameter becoming lower than that start parameter,
attempting to use nonsensical ranges. Tables larger than 2000 rows
also potentially dropped rows.

The fix is to ensure that the stop parameter is set relative to the start.
@mhilton mhilton self-assigned this Mar 29, 2023
@mhilton mhilton requested a review from a team as a code owner March 29, 2023 15:55
Copy link

@wolffcm wolffcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. Nice work.

I can see why this was never tested. It would have been good to be able to vary the buffer size to be very small for tests to catch stuff like this. (I have been writing tests with batch size set to 1 for IOx and it has kept me from writing tons of bugs)

@mhilton mhilton merged commit f83a24a into master Mar 30, 2023
@jacobmarble jacobmarble deleted the mh-large-join-tables branch January 4, 2024 17:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants