Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nuke to_mem_table_with_op_type #990

Merged
merged 4 commits into from
Mar 18, 2024
Merged

Nuke to_mem_table_with_op_type #990

merged 4 commits into from
Mar 18, 2024

Conversation

gefjon
Copy link
Contributor

@gefjon gefjon commented Mar 18, 2024

Description of Changes

Rather than annotating rows with __op_type during eval_incr of selects, partition the rows before evaluation, then merge after.

API and ABI breaking changes

N/a

Expected complexity level and risk

2 - a purely interior change to eval_incr of SELECT queries, but if incorrect could lead to subtle weird behaviors. Probably worth a bot test before merging.

Testing

Describe any testing you've done, and any testing you'd like your reviewers to do,
so that you're confident that all the changes work as expected!

  • Ran incr-select benchmark, saw:
    incr-select             time:   [322.05 ns 322.44 ns 322.85 ns]
                          change: [-17.272% -16.995% -16.747%] (p = 0.00 < 0.05)
                          Performance has improved.
    Found 7 outliers among 100 measurements (7.00%)
      5 (5.00%) high mild
      2 (2.00%) high severe
    
  • @bfops run a BitCraft bot test to ensure this didn't break anything. Large world or many bots not required; just a basic sanity check.

Rather than annotating rows with `__op_type` during `eval_incr` of selects,
partition the rows before evaluation, then merge after.
row.elements.remove(pos_op_type).into_u8().unwrap_or_else(|_| {
panic!("Failed to extract `{OP_TYPE_FIELD_NAME}` from `{}`", header.table_name)
});
let row = row_ref.into_product_value();
into.push(TableOp::new(op_type, row));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect; next step (I'll fix it): make TableOp hold a RelValue

Copy link
Contributor

@Centril Centril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful!

Co-authored-by: Mazdak Farrokhzad <twingoow@gmail.com>
Signed-off-by: Phoebe Goldman <phoebe@goldman-tribe.org>
@bfops bfops added performance release-any To be landed in any release window labels Mar 18, 2024
@gefjon gefjon requested a review from bfops March 18, 2024 19:51
Copy link
Collaborator

@bfops bfops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bots are happily running around as far as I can tell, no obvious breakage.

@gefjon gefjon added this pull request to the merge queue Mar 18, 2024
Merged via the queue into master with commit a21b1bc Mar 18, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-any To be landed in any release window
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants