-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[C++][Parquet] Support read BloomFilter in reader #15164
Comments
@mapleFU |
No, I would like to support it during China's Spring festival. And some bug fix might be added later this week |
pitrou
added a commit
that referenced
this issue
Feb 1, 2023
#33776) The original Parquet Bloom Filter spec, added in 2018 (PARQUET-41), was based on the Murmur3 hash function. The spec was later changed to use the XXH64 hash function from xxHash (PARQUET-1609); however, Parquet C++ wasn't updated for this and kept implementing the original spec. This PR switches the bloom filter implementation to the current version of the Bloom Filter spec (as of apache/parquet-format@3fb10e0). Conformance is tested using a dedicated test file in the parquet-testing repository (`bloom_filter.xxhash.bin`). Lead-authored-by: mwish <maplewish117@gmail.com> Co-authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org>
Issue resolved by pull request 33776 |
sjperkins
pushed a commit
to sjperkins/arrow
that referenced
this issue
Feb 10, 2023
…er spec (apache#33776) The original Parquet Bloom Filter spec, added in 2018 (PARQUET-41), was based on the Murmur3 hash function. The spec was later changed to use the XXH64 hash function from xxHash (PARQUET-1609); however, Parquet C++ wasn't updated for this and kept implementing the original spec. This PR switches the bloom filter implementation to the current version of the Bloom Filter spec (as of apache/parquet-format@3fb10e0). Conformance is tested using a dedicated test file in the parquet-testing repository (`bloom_filter.xxhash.bin`). Lead-authored-by: mwish <maplewish117@gmail.com> Co-authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org>
gringasalpastor
pushed a commit
to gringasalpastor/arrow
that referenced
this issue
Feb 17, 2023
…er spec (apache#33776) The original Parquet Bloom Filter spec, added in 2018 (PARQUET-41), was based on the Murmur3 hash function. The spec was later changed to use the XXH64 hash function from xxHash (PARQUET-1609); however, Parquet C++ wasn't updated for this and kept implementing the original spec. This PR switches the bloom filter implementation to the current version of the Bloom Filter spec (as of apache/parquet-format@3fb10e0). Conformance is tested using a dedicated test file in the parquet-testing repository (`bloom_filter.xxhash.bin`). Lead-authored-by: mwish <maplewish117@gmail.com> Co-authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org>
fatemehp
pushed a commit
to fatemehp/arrow
that referenced
this issue
Feb 24, 2023
…er spec (apache#33776) The original Parquet Bloom Filter spec, added in 2018 (PARQUET-41), was based on the Murmur3 hash function. The spec was later changed to use the XXH64 hash function from xxHash (PARQUET-1609); however, Parquet C++ wasn't updated for this and kept implementing the original spec. This PR switches the bloom filter implementation to the current version of the Bloom Filter spec (as of apache/parquet-format@3fb10e0). Conformance is tested using a dedicated test file in the parquet-testing repository (`bloom_filter.xxhash.bin`). Lead-authored-by: mwish <maplewish117@gmail.com> Co-authored-by: Antoine Pitrou <antoine@python.org> Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the enhancement requested
Currently, our C++ parquet support a
bloom_filter
class, but it breaks some standard. And, parquet C++ doesn't support reading bloom filter. I'd like to support reading bloom filter. I may split the task into these steps:Component(s)
C++, Parquet
The text was updated successfully, but these errors were encountered: