Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[query-cache] NPE in member when setting the same key value twice. #10556

Closed
blur0224 opened this issue May 10, 2017 · 3 comments
Closed

[query-cache] NPE in member when setting the same key value twice. #10556

blur0224 opened this issue May 10, 2017 · 3 comments

Comments

@blur0224
Copy link

@blur0224 blur0224 commented May 10, 2017

I originally found this issue because I was unknowingly setting a value twice. After hours of debugging I was able to determine the cause of the issue. The error should not be a NPE, it should be something that actually indicates what the problem is.

Hazelcast version 3.8.1
Cluster size 1-4 members
Number of the clients. (2 - 6 clients)
Version of Java. openjdk version "1.8.0_121"
Operating system. Linux and Windows

MapConfiguration:

QueryCacheConfig revocationQueryCacheConfig = new QueryCacheConfig(revocationTokenCacheName);
revocationQueryCacheConfig.getPredicateConfig().setImplementation((Predicate) entry -> true);

MapConfig revocationMapConfig = new MapConfig(revocationTokenMapName);
revocationMapConfig.addQueryCacheConfig(revocationQueryCacheConfig);
revocationMapConfig.setEvictionPolicy(EvictionPolicy.LRU);
revocationMapConfig.setTimeToLiveSeconds(revocationMapTimeToLive);
revocationMapConfig.setMaxIdleSeconds(revocationMapMaxIdle);

NearCacheConfig nearCacheConfig = new NearCacheConfig();
nearCacheConfig.setTimeToLiveSeconds(revocationCacheTimeToLive);
nearCacheConfig.setMaxIdleSeconds(revocationCacheMaxIdle);
revocationMapConfig.setNearCacheConfig(nearCacheConfig);

return revocationMapConfig;

Code to reproduce exception:

IEnterpriseMap<String, CacheToken> map = (IEnterpriseMap) instance.getMap(revocationMapName);
CacheToken cacheToken = new CacheToken();
cacheToken.setValue("abcd123");

map.set("abcd123", cacheToken);
map.set("abcd123", cacheToken);

Results in the exception in the member log:

java.lang.NullPointerException: null
        at com.hazelcast.query.impl.CachedQueryEntry.getTargetObject(CachedQueryEntry.java:100) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.query.impl.QueryableEntry.extractAttributeValue(QueryableEntry.java:81) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.query.impl.QueryableEntry.getAttributeValue(QueryableEntry.java:48) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.query.impl.predicates.AbstractPredicate.readAttributeValue(AbstractPredicate.java:132) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.query.impl.predicates.AbstractPredicate.apply(AbstractPredicate.java:57) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.query.SqlPredicate.apply(SqlPredicate.java:74) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.query.QueryEventFilter.eval(QueryEventFilter.java:50) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.AbstractFilteringStrategy.evaluateQueryEventFilter(AbstractFilteringStrategy.java:60) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.DefaultEntryEventFilteringStrategy.processQueryEventFilter(DefaultEntryEventFilteringStrategy.java:107) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.DefaultEntryEventFilteringStrategy.doFilter(DefaultEntryEventFilteringStrategy.java:79) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.QueryCacheEventPublisher.getCQCEventTypeOrNull(QueryCacheEventPublisher.java:164) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.QueryCacheEventPublisher.convertQueryCacheEventDataOrNull(QueryCacheEventPublisher.java:129) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.QueryCacheEventPublisher.addEventToQueryCache(QueryCacheEventPublisher.java:92) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.MapEventPublisherImpl.postPublishEvent(MapEventPublisherImpl.java:205) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.MapEventPublisherImpl.publishEvent(MapEventPublisherImpl.java:187) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.MapEventPublisherImpl.publishEvent(MapEventPublisherImpl.java:160) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.EnterpriseMapEventPublisherImpl.publishEvent(EnterpriseMapEventPublisherImpl.java:49) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.event.MapEventPublisherImpl.publishEvent(MapEventPublisherImpl.java:148) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.operation.BasePutOperation.afterRun(BasePutOperation.java:53) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.map.impl.operation.SetOperation.afterRun(SetOperation.java:40) ~[hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.afterRun(OperationRunnerImpl.java:287) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:188) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.run(OperationExecutorImpl.java:382) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl.runOrExecute(OperationExecutorImpl.java:409) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvokeLocal(Invocation.java:534) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.doInvoke(Invocation.java:519) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke0(Invocation.java:490) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.Invocation.invoke(Invocation.java:200) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.InvocationBuilderImpl.invoke(InvocationBuilderImpl.java:59) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.client.impl.protocol.task.AbstractPartitionMessageTask.processMessage(AbstractPartitionMessageTask.java:67) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.initializeAndProcessMessage(AbstractMessageTask.java:123) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.client.impl.protocol.task.AbstractMessageTask.run(AbstractMessageTask.java:103) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationservice.impl.OperationRunnerImpl.run(OperationRunnerImpl.java:142) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.process(OperationThread.java:127) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
        at com.hazelcast.spi.impl.operationexecutor.impl.OperationThread.run(OperationThread.java:102) [hazelcast-enterprise-all-3.8.1.jar:3.8.1]
@Donnerbart
Copy link
Contributor

@Donnerbart Donnerbart commented May 10, 2017

Could you please provide a compilable reproducer? I adapted your code to be executable and I don't get that NPE:

import com.hazelcast.config.Config;
import com.hazelcast.config.EvictionPolicy;
import com.hazelcast.config.MapConfig;
import com.hazelcast.config.NearCacheConfig;
import com.hazelcast.config.QueryCacheConfig;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IEnterpriseMap;
import com.hazelcast.query.Predicate;
import ee.License;

import java.io.Serializable;

public class Test {

    private static final String revocationMapName = "myMap";
    private static final String revocationTokenMapName = "myMap";
    private static final String revocationTokenCacheName = "myQueryCache";

    private static final int revocationCacheTimeToLive = 5;
    private static final int revocationCacheMaxIdle = 5;

    private static final int revocationMapTimeToLive = 5;
    private static final int revocationMapMaxIdle = 5;

    public static void main(String[] args) {
        HazelcastInstance instance = Hazelcast.newHazelcastInstance(getConfig());
        IEnterpriseMap<String, CacheToken> map = (IEnterpriseMap<String, CacheToken>) instance.<String, CacheToken>getMap(revocationMapName);

        CacheToken cacheToken = new CacheToken();
        cacheToken.setValue("abc123");

        map.set("abc123", cacheToken);
        map.set("abc123", cacheToken);

        System.out.println(map.get("abc123"));

        Hazelcast.shutdownAll();
    }

    private static Config getConfig() {
        QueryCacheConfig revocationQueryCacheConfig = new QueryCacheConfig(revocationTokenCacheName);
        revocationQueryCacheConfig.getPredicateConfig()
                .setImplementation((Predicate) entry -> true);

        NearCacheConfig nearCacheConfig = new NearCacheConfig()
                .setTimeToLiveSeconds(revocationCacheTimeToLive)
                .setMaxIdleSeconds(revocationCacheMaxIdle);

        MapConfig mapConfig = new MapConfig(revocationTokenMapName)
                .setEvictionPolicy(EvictionPolicy.LRU)
                .setTimeToLiveSeconds(revocationMapTimeToLive)
                .setMaxIdleSeconds(revocationMapMaxIdle)
                .addQueryCacheConfig(revocationQueryCacheConfig)
                .setNearCacheConfig(nearCacheConfig);

        return new Config()
                .setLicenseKey(License.LICENSE)
                .addMapConfig(mapConfig);
    }

    @SuppressWarnings("unused")
    private static class CacheToken implements Serializable {

        private String value;

        void setValue(String value) {
            this.value = value;
        }

        public String getValue() {
            return value;
        }

        @Override
        public String toString() {
            return "CacheToken{" +
                    "value='" + value + '\'' +
                    '}';
        }
    }
}

Is your configuration different from mine? I had to guess your timeout values etc. Which serialization does your CacheToken have?

@Donnerbart Donnerbart self-assigned this May 10, 2017
@blur0224
Copy link
Author

@blur0224 blur0224 commented May 11, 2017

This issue proved to be very difficult to reproduce, and only presents itself under very specific conditions. I've created three projects client1, client2, and member. In our environment, changing "set" to "putIfAbsent" when using the revocationMap seemed to resolve the issue, but I have no idea why.

These are gradle spring boot projects.

To run:
./gradlew clean build bootRun

Steps to reproduce:

  1. start member
  2. start client1
  3. make request to client 1's /test endpoint with a load tester. I'm using SoapUI 4 threads at a 500 millisecond delay
  4. there should be no errors in the hazelcast member log, even at under very high load
  5. start client2
  6. null pointer exceptions should start showing up in the member log, but only if there are concurrent threads hitting client 1's test endpoint

hazelcast-npe.tar.gz

We have a support account I can submit this too if that would be more appropriate.

@ahmetmircik
Copy link
Member

@ahmetmircik ahmetmircik commented Mar 9, 2018

as a workaround you can use put instead of set

@ahmetmircik ahmetmircik changed the title NPE in member when setting the same key value twice. [query-cache] NPE in member when setting the same key value twice. Mar 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

5 participants
You can’t perform that action at this time.