Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More benchmarks for creating bloom filters #672

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

anish749
Copy link
Contributor

In some use cases in Scio we create large Bloom filters with around 5 ~ 50M entries in some batch data pipelines. We were using BloomFilterAggregator API for most of our uses cases, and it felt like a good idea to bench mark the aggregator for comparison.

@CLAassistant
Copy link

CLAassistant commented Feb 21, 2019

CLA assistant check
All committers have signed the CLA.

@anish749 anish749 changed the title more benchmarks for creating bloom filters via aggregators More benchmarks for creating bloom filters Feb 21, 2019
@codecov-io
Copy link

Codecov Report

Merging #672 into develop will increase coverage by 0.06%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #672      +/-   ##
===========================================
+ Coverage    89.45%   89.52%   +0.06%     
===========================================
  Files          113      113              
  Lines         8944     8944              
  Branches       490      490              
===========================================
+ Hits          8001     8007       +6     
+ Misses         943      937       -6
Impacted Files Coverage Δ
...main/scala/com/twitter/algebird/monad/Reader.scala 50% <0%> (-5.56%) ⬇️
.../main/scala/com/twitter/algebird/BloomFilter.scala 94.69% <0%> (+0.44%) ⬆️
.../main/scala/com/twitter/algebird/HyperLogLog.scala 92.99% <0%> (+0.77%) ⬆️
...src/main/scala/com/twitter/algebird/Interval.scala 82.6% <0%> (+0.86%) ⬆️
...c/main/scala/com/twitter/algebird/MapAlgebra.scala 75.67% <0%> (+0.9%) ⬆️
...in/scala/com/twitter/algebird/scalacheck/Gen.scala 100% <0%> (+9.09%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 07b185f...8d028cd. Read the comment docs.

@johnynek
Copy link
Collaborator

johnynek commented May 7, 2020

by the way, I'm happy to merge this, but you never signed the CLA. Could you do that?

@anish749
Copy link
Contributor Author

anish749 commented May 7, 2020

Hello, I am extremely sorry I missed the CLA 🤦, I initially thought I already did it.
I tried doing it again, the comment got updated now, but the check still seems to be yellow.

@regadas
Copy link
Collaborator

regadas commented May 7, 2020

Yeah I think we add a similar case before where the status got stuck in the Pending phase ... couldn't find the PR where that happened though

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants