Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[INFRA] Update hibf and chopper #395

Merged
merged 3 commits into from
Oct 27, 2023
Merged

[INFRA] Update hibf and chopper #395

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1fd7a98
[INFRA] Update hibf
eseiler Oct 27, 2023
da7d2b5
[INFRA] Update chopper
eseiler Oct 27, 2023
52c8dd9
[TEST] Adapt for new relaxed_fpr
eseiler Oct 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/chopper
Open in desktop
Submodule chopper updated 18 files
+0 −12 include/chopper/layout/hibf_statistics.hpp
+1 −1 lib/hibf
+2 −1 src/layout/determine_best_number_of_technical_bins.cpp
+16 −17 src/layout/hibf_statistics.cpp
+11 −1 src/set_up_parser.cpp
+54 −0 src/util/display_layout/compute_ibf_size.hpp
+222 −117 src/util/display_layout/general.cpp
+33 −0 src/util/display_layout/process_file.cpp
+4 −0 src/util/display_layout/shared.hpp
+6 −36 src/util/display_layout/sizes.cpp
+23 −20 test/api/config_test.cpp
+11 −0 test/api/display_layout/CMakeLists.txt
+93 −0 test/api/display_layout/compute_ibf_size_test.cpp
+5 −3 test/api/layout/execute_layout_test.cpp
+24 −19 test/api/layout/execute_with_estimation_test.cpp
+2 −1 test/api/layout/hibf_statistics_test.cpp
+3 −10 test/benchmark/benchmark_data/query_cost/bin_influence_benchmark.cpp
+7 −5 test/cli/cli_chopper_pipeline_test.cpp
2 changes: 1 addition & 1 deletion lib/hibf
Open in desktop
Submodule hibf updated 36 files
+5 −1 .clang-tidy
+1 −1 README.md
+6 −1 include/hibf/build/bin_size_in_bits.hpp
+54 −21 include/hibf/config.hpp
+2 −0 include/hibf/contrib/robin_hood.hpp
+232 −0 include/hibf/contrib/std/chunk_by_view.hpp
+161 −0 include/hibf/contrib/std/detail/movable_box.hpp
+1 −1 include/hibf/hierarchical_interleaved_bloom_filter.hpp
+24 −24 include/hibf/interleaved_bloom_filter.hpp
+5 −0 include/hibf/layout/graph.hpp
+70 −9 include/hibf/layout/hierarchical_binning.hpp
+27 −0 include/hibf/misc/divide_and_ceil.hpp
+2 −2 include/hibf/misc/insert_iterator.hpp
+18 −8 src/build/construct_ibf.cpp
+2 −1 src/build/insert_into_ibf.cpp
+9 −2 src/config.cpp
+10 −6 src/hierarchical_interleaved_bloom_filter.cpp
+2 −1 src/interleaved_bloom_filter.cpp
+2 −2 src/layout/compute_layout.cpp
+10 −10 src/layout/graph.cpp
+25 −32 src/layout/hierarchical_binning.cpp
+9 −4 src/layout/simple_binning.cpp
+5 −4 test/performance/example/example_benchmark.cpp
+3 −2 test/performance/hibf/hierarchical_interleaved_bloom_filter_benchmark.cpp
+2 −2 test/performance/ibf/binning_bitvector_benchmark.cpp
+2 −1 test/performance/ibf/interleaved_bloom_filter_benchmark.cpp
+2 −2 test/snippet/hibf/hibf_construction.cpp
+1 −1 test/snippet/readme.cpp
+5 −0 test/unit/hibf/build/CMakeLists.txt
+28 −0 test/unit/hibf/build/bin_size_in_bits_test.cpp
+34 −13 test/unit/hibf/config_test.cpp
+5 −4 test/unit/hibf/hierarchical_interleaved_bloom_filter_test.cpp
+12 −1 test/unit/hibf/layout/graph_test.cpp
+6 −6 test/unit/hibf/layout/hierarchical_binning_test.cpp
+2 −1 test/unit/hibf/sketch/hyperloglog_test.cpp
+2 −1 util/fpr_correction_check.cpp
14 changes: 7 additions & 7 deletions test/data/datasources.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -124,37 +124,37 @@ declare_internal_datasource (FILE 128bins23window.hibf
)
declare_internal_datasource (FILE three_levels.hibf
URL ${CMAKE_CURRENT_LIST_DIR}/hibf/three_levels.hibf
URL_HASH SHA256=d04b045cf1db05a5ebc53b26e1bd2aaec430742310698dd4605990ab962fa439
URL_HASH SHA256=0690cd6e40de654d662092bd9efa793838e70aab445674235d399661f875572c
)

declare_internal_datasource (FILE 1bins.pack
URL ${CMAKE_CURRENT_LIST_DIR}/hibf/1bins.pack
URL_HASH SHA256=fb6a5acd10a419f735961242019d4d24e97fded230602db4de4355da819666e8
URL_HASH SHA256=5130c56fd5ba7a392c335fbeea38e16fdef0d895215a2dfa0815d203dcc62158
CONFIGURE true
)
declare_internal_datasource (FILE 64bins.pack
URL ${CMAKE_CURRENT_LIST_DIR}/hibf/64bins.pack
URL_HASH SHA256=e42fed94fd9a83e731bd1262ee77158b6cab91ffaeb8041c3d2765c2635ee7b2
URL_HASH SHA256=37bd8f436fc0b5d805aa0e65baab8b9ab75c7b902fda2afe604aacf367676a55
CONFIGURE true
)
declare_internal_datasource (FILE 128bins.pack
URL ${CMAKE_CURRENT_LIST_DIR}/hibf/128bins.pack
URL_HASH SHA256=eaf90b5214301dc9858807ba90a2f9977d98502607c53137ab11539492242e84
URL_HASH SHA256=425467230374a1a50bb464279ef883d3d1bd14df3f81449ba49ec3c9f87ff28a
CONFIGURE true
)
declare_internal_datasource (FILE three_levels.pack
URL ${CMAKE_CURRENT_LIST_DIR}/hibf/three_levels.pack
URL_HASH SHA256=5ebe3f469774533b6d66df690a1076406240fbb36a73730c1ea3299bbe89e7ae
URL_HASH SHA256=d0247914a3050cfb6e4c52f9b84ac50b7806d8d75c7586c96d4e184c66499494
CONFIGURE true
)
declare_internal_datasource (FILE test.layout
URL ${CMAKE_CURRENT_LIST_DIR}/test.layout
URL_HASH SHA256=fc8e2038062a67be39e79fbd8bfab3f6aaeb042189f8e8c9df2e3860ad05f94c
URL_HASH SHA256=0e9344550ca8de68fe51430b46100f39e8806e590bab7908479f87cd54f9586d
CONFIGURE true
)
declare_internal_datasource (FILE test_nocfg.layout
URL ${CMAKE_CURRENT_LIST_DIR}/test_nocfg.layout
URL_HASH SHA256=b3fe9494fca6a21265144a0501de7bbc3f40e38239a66b9d7e39703a390968a2
URL_HASH SHA256=df071bb98ab4dd06d052fc105e1cc65cb34b581dcc6240ca2fb85b39a0181306
CONFIGURE true
)

Expand Down
3 changes: 2 additions & 1 deletion test/data/hibf/128bins.pack
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,8 @@
@ "version": 1,
@ "number_of_user_bins": 128,
@ "number_of_hash_functions": 2,
@ "maximum_false_positive_rate": 0.05,
@ "maximum_fpr": 0.05,
@ "relaxed_fpr": 0.05,
@ "threads": 1,
@ "sketch_bits": 12,
@ "tmax": 64,
Expand Down
3 changes: 2 additions & 1 deletion test/data/hibf/1bins.pack
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@
@ "version": 1,
@ "number_of_user_bins": 1,
@ "number_of_hash_functions": 2,
@ "maximum_false_positive_rate": 0.05,
@ "maximum_fpr": 0.05,
@ "relaxed_fpr": 0.05,
@ "threads": 1,
@ "sketch_bits": 12,
@ "tmax": 64,
Expand Down
3 changes: 2 additions & 1 deletion test/data/hibf/64bins.pack
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@
@ "version": 1,
@ "number_of_user_bins": 64,
@ "number_of_hash_functions": 2,
@ "maximum_false_positive_rate": 0.05,
@ "maximum_fpr": 0.05,
@ "relaxed_fpr": 0.05,
@ "threads": 1,
@ "sketch_bits": 12,
@ "tmax": 64,
Expand Down
Binary file modified test/data/hibf/three_levels.hibf
View file
Open in desktop
Binary file not shown.
3 changes: 2 additions & 1 deletion test/data/hibf/three_levels.pack
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,8 @@
@ "version": 1,
@ "number_of_user_bins": 128,
@ "number_of_hash_functions": 2,
@ "maximum_false_positive_rate": 0.05,
@ "maximum_fpr": 0.05,
@ "relaxed_fpr": 0.3,
@ "threads": 1,
@ "sketch_bits": 12,
@ "tmax": 64,
Expand Down
3 changes: 2 additions & 1 deletion test/data/test.layout
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@
@ "version": 1,
@ "number_of_user_bins": 64,
@ "number_of_hash_functions": 2,
@ "maximum_false_positive_rate": 0.05,
@ "maximum_fpr": 0.05,
@ "relaxed_fpr": 0.3,
@ "threads": 1,
@ "sketch_bits": 12,
@ "tmax": 64,
Expand Down
3 changes: 2 additions & 1 deletion test/data/test_nocfg.layout
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@
@ "version": 1,
@ "number_of_user_bins": 64,
@ "number_of_hash_functions": 2,
@ "maximum_false_positive_rate": 0.05,
@ "maximum_fpr": 0.05,
@ "relaxed_fpr": 0.3,
@ "threads": 1,
@ "sketch_bits": 12,
@ "tmax": 64,
Expand Down