Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Or of two RunContainers can produce an under-sized BitmapContainer #717

Closed
larsk-db opened this issue Apr 18, 2024 · 1 comment · Fixed by #718
Closed

Or of two RunContainers can produce an under-sized BitmapContainer #717

larsk-db opened this issue Apr 18, 2024 · 1 comment · Fixed by #718
Labels

Comments

@larsk-db
Copy link
Contributor

Describe the bug
There is a very niche edge case, where or:ing two RunContainers can produce a BitmapContainer that is actually undersized, because it happens to serialize more efficiently than the resulting RunContainer. This container works perfectly fine in memory, but as soon as it gets serialized and we then try to deserialize it again, we will try to read the bytes as if they were of an ArrayContainer, because we determine this by cardinality. And obviously the result is garbage :D (if it doesn't just flat out fail during deserialization, which is what we originally observed).

To Reproduce
This is a bit of a pathological example that highlights the issue is possible. But we have observed it in real bitmaps that were created with the high-level APIs, not container level. They are just a bit too large to use as a test case for the library, so I hope the pathological example below will serve.

@Test
public void or6() {
  System.out.println("or6");
  RunContainer rc1 = new RunContainer();
  for (int i = 0; i < 6144; i += 6) {
    rc1.iadd(i, i+1);
  }

  RunContainer rc2 = new RunContainer();

  for (int i = 3; i < 6144; i += 6) {
    rc2.iadd(i, i+1);
  }

  Container result = rc1.or(rc2);
  assertTrue(result.getCardinality() < ArrayContainer.DEFAULT_MAX_SIZE);
  assertTrue(result instanceof ArrayContainer);
}

RoaringBitmap version: 1.0.6-SNAPSHOT
I.e. I tested this on latest master.

Java version: openjdk version "11.0.12" 2021-07-20 LTS

Fix
I already have a fix and will post a PR with test and fix soon.

@lemire
Copy link
Member

lemire commented Apr 19, 2024

Thanks. This is a very serious issue. Very glad for the report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants