Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Timestamp statistics for ORC file #1147

Open
wants to merge 3 commits into
base: master
from

Conversation

@Praveen2112
Copy link
Member

commented Jul 19, 2019

This patch enables us to skip certain stripes and row groups in ORC file based on timestamp statistics.

@cla-bot cla-bot bot added the cla-signed label Jul 19, 2019

@Praveen2112 Praveen2112 force-pushed the Praveen2112:orc_timestamp_stats branch from 328be74 to 5e68956 Jul 19, 2019

@dain dain self-requested a review Jul 23, 2019

@findepi findepi removed their assignment Jul 24, 2019

@dain dain assigned Praveen2112 and unassigned dain Jul 27, 2019

@Praveen2112 Praveen2112 force-pushed the Praveen2112:orc_timestamp_stats branch 2 times, most recently from 1c8ce11 to f9ec55f Jul 28, 2019

@dain dain assigned dain and unassigned Praveen2112 Aug 1, 2019

@Praveen2112 Praveen2112 force-pushed the Praveen2112:orc_timestamp_stats branch from f9ec55f to f27bcc9 Aug 2, 2019

@Praveen2112 Praveen2112 force-pushed the Praveen2112:orc_timestamp_stats branch from f27bcc9 to ecabb4e Aug 2, 2019

@dain
Copy link
Member

left a comment

This looks good, but I just noticed we are missing a test. The new timestamp predicate should be integrated into: https://github.com/prestosql/presto/blob/master/presto-orc/src/test/java/io/prestosql/orc/TestingOrcPredicate.java
I believe this would have caught the previous errors.

I also think we might be missing some additions to AbstractStatisticsBuilderTest. Search for uses of io.prestosql.orc.metadata.statistics.ColumnStatistics#getDateStatistics in the codebase.

@dain dain assigned Praveen2112 and unassigned dain Aug 5, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
You can’t perform that action at this time.