Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement BoundPredicateVisitor trait for ManifestFilterVisitor #367

Merged
merged 11 commits into from
Jun 13, 2024

Conversation

s-akhtar-baig
Copy link
Contributor

@s-akhtar-baig s-akhtar-baig commented May 9, 2024

GitHub issue: #350

Description: ManifestEvaluator was defined in #322. This PR implements all functions and adds most of the Python unit tests.

Testing: Added new unit tests.

Copy link
Contributor

@sdd sdd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks so much for the contribution! Just a few small issues that are straightforward to resolve. 🙌🏼

crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
@s-akhtar-baig
Copy link
Contributor Author

@sdd, thank you for reviewing the changes and providing references! I have modified my code based on your suggestions. Please take a look and let me know if I miss anything.

@s-akhtar-baig s-akhtar-baig requested a review from sdd May 10, 2024 20:42
Copy link
Contributor

@sdd sdd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're almost there! Just a couple of small stylistic changes required and then I'm happy.

Thanks again! 😁

return ROWS_MIGHT_MATCH;
}

if let Some(Literal::Primitive(lower_bound)) = &field.lower_bound {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is much cleaner! Thanks :-)

Copy link
Contributor

@marvinlanhenke marvinlanhenke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR @s-akhtar-baig which looks really good - left some minor comments. However, please check the comment about 'comparison' and the and implementation and verify its correct.

crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
Copy link
Contributor Author

@s-akhtar-baig s-akhtar-baig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marvinlanhenke @sdd, thank you for reviewing. I have modified the code accordingly and added comments for clarification. Let me know what you think.

crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @s-akhtar-baig for this great pr, it looks great! I left some questions about the confusing part. Also I think one important thing is that we should not rely one the Ord of PrimitiveLiteral.

Copy link
Collaborator

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @s-akhtar-baig for this pr, it's great! There are some changes since #378 got resolved, so please rebase main branch to fix it. Others LGTM, thanks for this effort!

crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
crates/iceberg/src/expr/visitors/manifest_evaluator.rs Outdated Show resolved Hide resolved
@s-akhtar-baig
Copy link
Contributor Author

@liurenjie1024 @marvinlanhenke @sdd, thank you for reviewing these changes and for your feedback! Greatly appreciated!

I have pushed the required changes. Please let me know if I missed anything, thanks!

Copy link
Contributor

@sdd sdd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! Thanks for persevering :-)

@sdd
Copy link
Contributor

sdd commented Jun 11, 2024

Hi @liurenjie1024 - sorry to pester you but are you able to re-review this please? It's the last major piece of the puzzle on the read side.

Copy link
Contributor

@marvinlanhenke marvinlanhenke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@s-akhtar-baig
LGTM. Thanks for your effort here, just a minor nit that is still unresolved from the last review about extracting a common helper function, but this is not a blocker.

@liurenjie1024
Copy link
Collaborator

Hi @liurenjie1024 - sorry to pester you but are you able to re-review this please? It's the last major piece of the puzzle on the read side.

Hi, @sdd Sorry for delay. I'll review it today.

Copy link
Collaborator

@liurenjie1024 liurenjie1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @s-akhtar-baig for this effort!

@liurenjie1024
Copy link
Collaborator

cc @Fokko Do you want to take a review for this? I see you assigned yourself for review.

Copy link
Contributor

@Fokko Fokko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @s-akhtar-baig for working on this, and adding all the tests 🙌 Thanks @marvinlanhenke, @sdd and @liurenjie1024 for the review!

@Fokko Fokko merged commit 070576b into apache:main Jun 13, 2024
6 checks passed
@s-akhtar-baig s-akhtar-baig deleted the impl_boundpredvisitor_traits branch June 13, 2024 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants