IndexImpl.getRecords fails with NPE #5807

Closed
YuriyKravtsov opened this Issue Jul 28, 2015 · 5 comments

Projects

None yet

3 participants

@YuriyKravtsov

In #4525 muratayan suggested using following:

final Predicate emptyFieldPredicate = Predicates.notEqual("field", "value");
final Predicate nullPredicate = Predicates.equal("field", null);
final Collection<TestClass> emptyFieldValues = map.values(Predicates.or(emptyFieldPredicate, nullPredicate));

On hazelcast launched with xml configuration contained:

<map name="map">
  <backup-count>0</backup-count>
  <async-backup-count>0</async-backup-count>
  <read-backup-data>true</read-backup-data>
  <indexes>
    <index>field</index>
  </indexes>
</map>

I tried it and got:

java.lang.NullPointerException: null
    at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333) ~[na:1.7.0_79]
    at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988) ~[na:1.7.0_79]
    at com.hazelcast.query.impl.UnsortedIndexStore.getRecords(UnsortedIndexStore.java:190) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.impl.IndexImpl.getRecords(IndexImpl.java:129) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.Predicates$EqualPredicate.filter(Predicates.java:779) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.Predicates$OrPredicate.filter(Predicates.java:578) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.PagingPredicate.filter(PagingPredicate.java:169) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.impl.IndexService.query(IndexService.java:95) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.operation.QueryOperation.run(QueryOperation.java:84) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.handle(BasicOperationService.java:750) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.access$500(BasicOperationService.java:726) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicOperationService.runOperationOnCallingThread(BasicOperationService.java:253) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocation.doInvokeLocal(BasicInvocation.java:280) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocation.doInvoke(BasicInvocation.java:262) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocation.invoke(BasicInvocation.java:229) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocationBuilder.invoke(BasicInvocationBuilder.java:48) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.client.AbstractMapQueryRequest.createInvocations(AbstractMapQueryRequest.java:140) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.client.AbstractMapQueryRequest.invoke(AbstractMapQueryRequest.java:74) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.impl.client.InvocationClientRequest.process(InvocationClientRequest.java:27) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:434) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:353) [hazelcast-3.4.4.jar:3.4.4]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
    at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:92) [hazelcast-3.4.4.jar:3.4.4]
2015-07-28 16:04:52,396 ERROR HZC     [hz._hzInstance_1_dev.client.thread-54] c.h.c.ClientEngine [192.168.168.77]:5701 [dev] [3.4.4] While executing request: com.hazelcast.map.impl.client.MapQueryRequest@ff1b76 -> null
java.lang.NullPointerException: null
    at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333) ~[na:1.7.0_79]
    at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988) ~[na:1.7.0_79]
    at com.hazelcast.query.impl.UnsortedIndexStore.getRecords(UnsortedIndexStore.java:190) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.impl.IndexImpl.getRecords(IndexImpl.java:129) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.Predicates$EqualPredicate.filter(Predicates.java:779) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.Predicates$OrPredicate.filter(Predicates.java:578) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.PagingPredicate.filter(PagingPredicate.java:169) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.impl.IndexService.query(IndexService.java:95) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.operation.QueryOperation.run(QueryOperation.java:84) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.handle(BasicOperationService.java:750) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.access$500(BasicOperationService.java:726) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicOperationService.runOperationOnCallingThread(BasicOperationService.java:253) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocation.doInvokeLocal(BasicInvocation.java:280) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocation.doInvoke(BasicInvocation.java:262) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocation.invoke(BasicInvocation.java:229) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.spi.impl.BasicInvocationBuilder.invoke(BasicInvocationBuilder.java:48) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.client.AbstractMapQueryRequest.createInvocations(AbstractMapQueryRequest.java:140) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.client.AbstractMapQueryRequest.invoke(AbstractMapQueryRequest.java:74) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.impl.client.InvocationClientRequest.process(InvocationClientRequest.java:27) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.processRequest(ClientEngineImpl.java:434) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.impl.ClientEngineImpl$ClientPacketProcessor.run(ClientEngineImpl.java:353) ~[hazelcast-3.4.4.jar:3.4.4]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
    at com.hazelcast.util.executor.HazelcastManagedThread.executeRun(HazelcastManagedThread.java:76) [hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.util.executor.HazelcastManagedThread.run(HazelcastManagedThread.java:92) [hazelcast-3.4.4.jar:3.4.4]

Another question is.

Is it by design that

Predicates.not(Predicates.equal("field", "value"));

still returns null values while

Predicates.notEquals("field", "value");

does not?

@serkan-ozal
Contributor

I think, the issue is caused by null value is not converted to IndexImpl::NULL internally. Normally, conversion is handled by TypeConverter implementations but as far as I see, IdentityConverter (as far as I see, it is used for unknown object types (not String, Date, long, int, etc ...)) doesn't handle this conversion. Not sure but maybe this is the cause of the issue.

@YuriyKravtsov

should I try

final Predicate emptyFieldPredicate = Predicates.notEqual("field", "value");
final Predicate nullPredicate = Predicates.equal("field", IndexImpl.NULL);
final Collection emptyFieldValues = map.values(Predicates.or(emptyFieldPredicate, nullPredicate));

?
I have strong feeling it won't help.

@serkan-ozal
Contributor

I think worth to try

@YuriyKravtsov

The matter is that I tried, that is why I wrote 'I have strong feeling it won't help'.

There is no suitable serializer for class com.hazelcast.query.impl.IndexImpl$NullObject
com.hazelcast.nio.serialization.HazelcastSerializationException: There is no suitable serializer for class com.hazelcast.query.impl.IndexImpl$NullObject
    at com.hazelcast.nio.serialization.SerializationServiceImpl.serializerFor(SerializationServiceImpl.java:514) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.SerializationServiceImpl.writeObject(SerializationServiceImpl.java:286) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:262) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.Predicates$EqualPredicate.writeData(Predicates.java:795) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:140) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:39) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.write(StreamSerializerAdapter.java:40) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.SerializationServiceImpl.writeObject(SerializationServiceImpl.java:294) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:262) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.Predicates$OrPredicate.writeData(Predicates.java:620) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:140) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:39) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.write(StreamSerializerAdapter.java:40) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.SerializationServiceImpl.writeObject(SerializationServiceImpl.java:294) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:262) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.query.PagingPredicate.writeData(PagingPredicate.java:304) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:140) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.DataSerializer.write(DataSerializer.java:39) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.write(StreamSerializerAdapter.java:40) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.SerializationServiceImpl.writeObject(SerializationServiceImpl.java:294) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.ByteArrayObjectDataOutput.writeObject(ByteArrayObjectDataOutput.java:262) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.client.MapQueryRequest.writePortableInner(MapQueryRequest.java:51) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.client.AbstractMapQueryRequest.write(AbstractMapQueryRequest.java:159) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.map.impl.client.MapQueryRequest.write(MapQueryRequest.java:28) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.impl.client.ClientRequest.writePortable(ClientRequest.java:107) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.PortableSerializer.write(PortableSerializer.java:65) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.PortableSerializer.write(PortableSerializer.java:30) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.StreamSerializerAdapter.toData(StreamSerializerAdapter.java:51) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.SerializationServiceImpl.toData(SerializationServiceImpl.java:227) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.nio.serialization.SerializationServiceImpl.toData(SerializationServiceImpl.java:208) ~[hazelcast-3.4.4.jar:3.4.4]
    at com.hazelcast.client.spi.impl.ClientInvocationServiceImpl.sendInternal(ClientInvocationServiceImpl.java:153) ~[hazelcast-client-3.4.4.jar:3.4.4]
    at com.hazelcast.client.spi.impl.ClientInvocationServiceImpl.doSend(ClientInvocationServiceImpl.java:144) ~[hazelcast-client-3.4.4.jar:3.4.4]
    at com.hazelcast.client.spi.impl.ClientInvocationServiceImpl.send(ClientInvocationServiceImpl.java:129) ~[hazelcast-client-3.4.4.jar:3.4.4]
    at com.hazelcast.client.spi.impl.ClientInvocationServiceImpl.invokeOnRandomTarget(ClientInvocationServiceImpl.java:61) ~[hazelcast-client-3.4.4.jar:3.4.4]
    at com.hazelcast.client.spi.ClientProxy.invoke(ClientProxy.java:150) ~[hazelcast-client-3.4.4.jar:3.4.4]
    at com.hazelcast.client.proxy.ClientMapProxy.entrySet(ClientMapProxy.java:755) ~[hazelcast-client-3.4.4.jar:3.4.4]
@jerrinot jerrinot self-assigned this Sep 11, 2015
@jerrinot jerrinot added this to the 3.6 milestone Sep 11, 2015
@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 12, 2015
@jerrinot jerrinot Change TypeConverter from interface into abstract class.
The abstract class does null check - it's no longer
a responsibility of each converter to convert NULL values
into NULL-objects.

Fixes #5807
3ea8c0c
@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 12, 2015
@jerrinot jerrinot Change TypeConverter from interface into abstract class.
The abstract class does null check - it's no longer
a responsibility of each converter to convert NULL values
into NULL-objects.

Fixes #5807
359d405
@jerrinot
Contributor

@YuriyKravtsov: Thank you for reporting the issue.
The NPE is indeed a bug; PR with a fix is already pending: #6192

The inconsistency between not(equals() and notEquals() is caused by different ways how predicates treat null values. It has been introduced to mimic behaviour of SQL, however not every predicate supports this. We have decided to revert it and treat NULL as any other values. There is already a pending PR with a fix, see this #6186

@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 12, 2015
@jerrinot jerrinot Change TypeConverter from interface into abstract class.
The abstract class does null check - it's no longer
a responsibility of each converter to convert NULL values
into NULL-objects.

Fixes #5807
29af921
@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 12, 2015
@jerrinot jerrinot Change TypeConverter from interface into abstract class.
The abstract class does null check - it's no longer
a responsibility of each converter to convert NULL values
into NULL-objects.

Fixes #5807
ef7f979
@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 12, 2015
@jerrinot jerrinot Change TypeConverter from interface into abstract class.
The abstract class does null check - it's no longer
a responsibility of each converter to convert NULL values
into NULL-objects.

Fixes #5807
3313e2b
@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 12, 2015
@jerrinot jerrinot Regression tests for #5807 added 4a3051b
@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 14, 2015
@jerrinot jerrinot Regression tests for #5807 added 99219f4
@jerrinot jerrinot added a commit to jerrinot/hazelcast that referenced this issue Sep 14, 2015
@jerrinot jerrinot Regression tests for #5807 added 6fa0069
@jerrinot jerrinot closed this in #6192 Sep 14, 2015
@Donnerbart Donnerbart added a commit to Donnerbart/hazelcast that referenced this issue Sep 17, 2015
@jerrinot @Donnerbart jerrinot + Donnerbart Change TypeConverter from interface into abstract class.
The abstract class does null check - it's no longer
a responsibility of each converter to convert NULL values
into NULL-objects.

Fixes #5807
5208045
@Donnerbart Donnerbart added a commit to Donnerbart/hazelcast that referenced this issue Sep 17, 2015
@jerrinot @Donnerbart jerrinot + Donnerbart Regression tests for #5807 added ec5076e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment