Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BloomFilter Serialization #6068

Open
matebase opened this issue Jun 2, 2022 · 1 comment
Open

BloomFilter Serialization #6068

matebase opened this issue Jun 2, 2022 · 1 comment
Labels

Comments

@matebase
Copy link

matebase commented Jun 2, 2022

BloomFilter size keeps increasing when serializing and deserializing with kryo ?

Below is the relevant code

import com.esotericsoftware.kryo.kryo5.Kryo;
import com.esotericsoftware.kryo.kryo5.io.Input;
import com.esotericsoftware.kryo.kryo5.io.Output;
import com.esotericsoftware.kryo.kryo5.objenesis.strategy.StdInstantiatorStrategy;
import com.esotericsoftware.kryo.kryo5.util.DefaultInstantiatorStrategy;
import com.esotericsoftware.kryo.kryo5.util.Pool;
import com.google.common.hash.BloomFilter;
import com.google.common.hash.Funnels;

protected byte[] kryoSerialize(Object obj) {
    Kryo kryo = kryoPool.obtain();
    try {
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        Output output = new Output(byteArrayOutputStream);
        kryo.writeObject(output, obj);
        output.close();
        return byteArrayOutputStream.toByteArray();
    } catch (Exception e) {
        return null;
    } finally {
        kryoPool.free(kryo);
    }
}

protected <T> T kryoDeserialize(byte[] bytes, Class<T> clazz) {
    Kryo kryo = kryoPool.obtain();
    try {
        kryo.setInstantiatorStrategy(new DefaultInstantiatorStrategy(new StdInstantiatorStrategy()));
        ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);
        Input input = new Input(byteArrayInputStream);
        input.close();
        return (T) kryo.readObject(input, clazz);
    } catch (Exception e) {
        return null;
    } finally {
        kryoPool.free(kryo);
    }
}

Pool<Kryo> kryoPool = new Pool<Kryo>(true, false, 8) {
    @Override
    protected Kryo create() {
        Kryo kryo = new Kryo();
        kryo.setReferences(true);
        kryo.setRegistrationRequired(false);
        return kryo;
    }
};
@amalloy
Copy link
Contributor

amalloy commented Jun 10, 2022

Could you be clearer about how you're detecting this size increase? I see you've included adapter code to plug into Kryo, but nothing that specifically reproduces an issue. I also suggest that your minimum reproducible example use normal Java serialization (Object(Input|Output)Stream), rather than Kryo. If you can't reproduce the problem with normal serialization, then the issue would appear to be with Kryo rather than with BloomFilter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants