total number of bytes decrement with a high number of items

[albertobsd]

I'm already using the version Two it works very well for some numer of items but I notice that the unsigned int bit limits the amount of real reserved memory.

bytes is always ceil bits/8

This result in a maximum number of bits: 4,294,967,295 but only 536870912 bytes

With some high value of items it result in some cases to a random decrement of bytes

I made this code as a proof of concept

#include <stdio.h>
#include <stdlib.h>
#include "bloom/bloom.h"

struct bloom bloom;

unsigned int items[10] = {4194304,8388608,16777216,33554432,67108864,134217728,268435456,536870912,1073741824,2147483648};

int main(int argc, char **argv)  {
  int i = 0;
  while(i < 10) {
    if(bloom_init2(&bloom,items[i],0.001)  == 1)  {
      fprintf(stderr,"error bloom_init\n");
      exit(0);
    }
    printf("Items %u bloom filter bytes %u\n",items[i],bloom.bytes);
    bloom_reset(&bloom);
    i++;
  }
}

His output is:

Items 4194304 bloom filter bytes 7537997
Items 8388608 bloom filter bytes 15075994
Items 16777216 bloom filter bytes 30151987
Items 33554432 bloom filter bytes 60303973
Items 67108864 bloom filter bytes 120607946
Items 134217728 bloom filter bytes 241215893
Items 268435456 bloom filter bytes 482431785
Items 536870912 bloom filter bytes 427992657
Items 1073741824 bloom filter bytes 319114402
Items 2147483648 bloom filter bytes 101357891

The last line is only 96 MB of RAM for a bloom filter of 2147483648 items.

I solve this in my code changing the bits variable type for "unsigned long long int" instead of "unsigned int"

Kinds regards

[jvirkki]

This should be working correctly now (in the development branch for now).