Skip to content

Conversation

paulorsousa
Copy link
Collaborator

This PR adds two new memtier benchmark test suites for evaluating dense HyperLogLog implementations:

Changes:

  • PFCOUNT benchmark: Tests performance of counting multiple pre-loaded HyperLogLog keys
  • PFMERGE benchmark: Tests performance of merging multiple pre-loaded HyperLogLog keys

Test Configuration:

  • Pre-loads 3 HyperLogLog keys with 10K random elements each using PFADD (dense format)
  • Uses 100B sized random elements to ensure dense HyperLogLog representation
  • Runs with 50 connections and 4 threads for 180 seconds
  • Targets both amd64 and arm64 architectures

Purpose:

These benchmarks are specifically designed to evaluate different HyperLogLog dense implementations:

  • Scalar implementations
  • AVX SIMD (x86_64 optimization)
  • NEON SIMD (ARM64 optimization)

Note: the dense HyperLogLog format is particularly suitable for SIMD optimisations due to its regular data structure and parallel processing opportunities.

Pre-loads 3 HyperLogLog keys with 10K random elements each using PFADD (dense).
Benchmarks PFCOUNT performance counting the pre-loaded HLLs.

Useful for evaluating different HyperLogLog dense implementations (scalar, AVX, and NEON SIMD).
Pre-loads 3 HyperLogLog keys with 10K random elements each using PFADD (dense).
Benchmarks PFMERGE performance.

Useful for evaluating different HyperLogLog dense implementations (scalar, AVX, and NEON SIMD).
…ERGE tests

The operations are stable, so long test durations are unnecessary
Copy link

jit-ci bot commented Oct 12, 2025

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

@fcostaoliveira fcostaoliveira merged commit 34710fe into main Oct 12, 2025
2 of 9 checks passed
@fcostaoliveira fcostaoliveira deleted the hyperloglog-test-suites branch October 12, 2025 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants