Hazelcast standalone cluster. There is no suitable de-serializer for type 10000.

[MonDeveloper]

Hi there, I'm facing a pretty common issue but even if I tried to dig into every single issue already open I was not able to make it working, so this is the reason I'm opening this issue.

Setup:

Java application with Spring Boot 2.7.18, Spring Framework 5.3.32 and Spring Cloud 2021.0.9 playing the Spring Cloud Gateway role.
We also have an Hazelcast standalone (version 5.3) made by 3 nodes running on different servers.
Both Java service and Hazlecast nodes are started using a temurin JDK 17.

First Implementation:

I successfully introduced here the Rate Limiting functionality using your Bucket4j + hazelcast library (using the standard java serialization) just adding this dependency to our java service pom

<dependency>
	<groupId>com.bucket4j</groupId>
	<artifactId>bucket4j-hazelcast</artifactId>
	<version>8.8.0</version>
</dependency>

and doing nothing at the Hazelcast cluster nodes side.

At this point everything was working fine, the maps were correctly created and updated in the hazelcast cluster and the API were correctly rate-limited at the java side.

Problem:

In order to speed up the entire rate limiting process I reed in your documentation we could use a better serialization so I made 2 things:

1. Added your 2 jars (com.bucket4j:bucket4j-core:8.8.0 and com.bucket4j:bucket4j-hazelcast:8.8.0) into the classpath of all the hazelcast cluster nodes (different servers than the java service one)

2. Modified the hazelcast client configuration used to create the hazelcastInstance adding your customSerializers using this code HazelcastProxyManager.addCustomSerializers(config.getSerializationConfig(), 10000);

And this is enough to make our UnitTests getting the exception com.hazelcast.nio.serialization.HazelcastSerializationException: There is no suitable de-serializer for type 10000. This exception is likely caused by differences in the serialization configuration between members or between clients and members at every single call.

Commenting out the HazelcastProxyManager.addCustomSerializers command make our APIs working again, of course using the slow java serialization.

Ipothesys:

In addiction to adding the jars into the classpath of the hazelcast cluster nodes, do we have to explicitly tell (programmatically or via config) the hazelcast cluster about those serializers?

• We know it is possible to add custom serializers directly in the hazelcast cluster configuration file, unfortunately the custom serializers added in this way require to have a parameterless constructor while your 3 custom serializers need to have an Integer passed into the constructor so this way seems to be unavailable.

• Do you need to develop a DataSerializableFactory and use your HazelcastProxyManager.addCustomSerializers(config.getSerializationConfig(), 10000); also there? Unfortunately the DataSerializableFactory does not provide any config hazelcast object to modify as your code requires.

• Do we need to develop a DataSerializableFactory and replicate the logic used within your HazelcastProxyManager.addCustomSerializers method?

Questions:

Are we missing something during our analysis? ...or do you think we did all the proper tasks and you also expected the whole system should work?
Could you suggest us the proper way to do a better diagnosis?

[fgapito]

Hi,

I'm following because I have quite the same question to ask.

Thank you.
f

[vladimir-bukhtoyarov]

Hello @fgapito @MonDeveloper

In addiction to adding the jars into the classpath of the hazelcast cluster nodes, do we have to explicitly tell (programmatically or via config) the hazelcast cluster about those serializers?

Exactly. Both side neet to take agreement about serialization. Looks that I missed to point this in documentation. I will fix docs in next release.

Are we missing something during our analysis? ...or do you think we did all the proper tasks and you also expected the whole system should work?

I think that you performed well investigation. It looks that guys who previously used this feature, have a full controll under the cluster(for example they run cluster as embeeded inside their own main), this is why nobody asked about configuration through XML. Unfortunatelly, inside the tests, I am also have a full controll under the cluster, because I run cluster from my code, for example https://github.com/bucket4j/bucket4j/blob/master/bucket4j-hazelcast-all/bucket4j-hazelcast/src/test/java/io/github/bucket4j/hazelcast/HazelcastWithCustomSerializersTest.java#L44

Feel free to make pull request for cases where it is not possible to run HazelcastProxyManager.addCustomSerializers during cluster initialization. Do not forget to copy HazelcastWithCustomSerializersTest and refactor it to start with non-java config that more precise for your use case.

[fgapito]

Hi @vladimir-bukhtoyarov

I really thank you for your answer and congratulations, but the investigation has been performed by @MonDeveloper .

F

[MonDeveloper]

Added the PR [#453]

[vladimir-bukhtoyarov]

Has been released with version 8.9.0