Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JCache client needs access to CacheLoader implementation to work #9453

Closed
JChrist opened this issue Dec 14, 2016 · 33 comments

Comments

Projects
None yet
3 participants
@JChrist
Copy link

commented Dec 14, 2016

Hello.
I am using hazelcast 3.7.4 on a server node, as well as a hazelcast client node.
I have configured a cache using XML on the server node similar to, using read-through and specifying a cache loader factory.

On the client side, I am receiving a com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException for my CacheLoader's implementation class, since it is not available in the client's classpath.

While searching, I've found a similar closed issue: #6676
Looking at the tests, I found one test that seems to test the exact issue I am experiencing, however I believe the test is incorrect and passes while it shouldn't, since it doesn't actually use a Hazelcast Client caching provider.

I gave it a go to write a test for the issue I'm trying to describe. While not quite good a test, I had some feedback that the CacheLoader is required on the client, it fails with an IllegalStateException on Factory's writeExternal method.

package com.hazelcast.client.cache;

import com.hazelcast.cache.impl.HazelcastServerCachingProvider;
import com.hazelcast.client.HazelcastClient;
import com.hazelcast.client.cache.impl.HazelcastClientCachingProvider;
import com.hazelcast.client.config.ClientConfig;
import com.hazelcast.client.impl.HazelcastClientInstanceImpl;
import com.hazelcast.config.CacheConfig;
import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.HazelcastInstanceAware;
import com.hazelcast.instance.HazelcastInstanceImpl;
import com.hazelcast.internal.util.ThreadLocalRandom;
import org.junit.Test;

import javax.cache.Cache;
import javax.cache.configuration.CompleteConfiguration;
import javax.cache.configuration.Factory;
import javax.cache.integration.CacheLoader;
import javax.cache.integration.CacheLoaderException;
import javax.cache.spi.CachingProvider;
import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;

public class ClientCacheReadThroughNoCacheLoader {
    @Test
    public void clientDoesNotNeedCacheLoader() throws Exception {
        final String serverName = UUID.randomUUID().toString();
        final Config serverConfig = new Config(serverName);
        final int port = ThreadLocalRandom.current().nextInt(5000, 15000);
        serverConfig.getNetworkConfig().setPort(port).setPublicAddress("localhost");

        final HazelcastInstance server = Hazelcast.newHazelcastInstance(serverConfig);
        CachingProvider serverProvider = HazelcastServerCachingProvider.createCachingProvider(server);
        CompleteConfiguration<Integer, String> cacheConfig =
                new CacheConfig<Integer, String>()
                        .setTypes(Integer.class, String.class)
                        .setReadThrough(true)
                        .setCacheLoaderFactory(new ServerSideCacheLoaderFactory());

        final String cacheName = UUID.randomUUID().toString();
        serverProvider.getCacheManager().createCache(cacheName, cacheConfig);

        System.out.println("Creating client now");

        final String clientName = UUID.randomUUID().toString();
        final ClientConfig clientConfig = new ClientConfig();
        clientConfig.setInstanceName(clientName);
        clientConfig.getNetworkConfig().addAddress("localhost:"+port);

        final HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig);
        CachingProvider clientCachingProvider = HazelcastClientCachingProvider.createCachingProvider(client);
        Cache<Integer, String> clientCache = clientCachingProvider.getCacheManager()
                .getCache(cacheName, Integer.class, String.class);
    }

    public static class ServerSideCacheLoaderFactory
            implements Factory<ServerSideCacheLoader>, HazelcastInstanceAware, Externalizable {

        private transient HazelcastInstance hazelcastInstance;
        public ServerSideCacheLoaderFactory() {
            System.out.println("initialized with instance: " + hazelcastInstance);
        }

        @Override
        public void setHazelcastInstance(HazelcastInstance hazelcastInstance) {
            this.hazelcastInstance = hazelcastInstance;
        }

        @Override
        public ServerSideCacheLoader create() {
            System.out.println("create called");
            if (hazelcastInstance instanceof HazelcastInstanceImpl) {
                return new ServerSideCacheLoader();
            } else {
                throw new IllegalStateException("This factory can only be used at server side!");
            }
        }

        @Override
        public void writeExternal(ObjectOutput out) throws IOException {
            if(hazelcastInstance != null) {
                if(hazelcastInstance instanceof HazelcastClientInstanceImpl) {
                    throw new IllegalStateException("This factory can only be used at server side!");
                }
            }
            System.out.println("write with instance: " + hazelcastInstance);
        }

        @Override
        public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
            if(hazelcastInstance != null) {
                if(hazelcastInstance instanceof HazelcastClientInstanceImpl) {
                    throw new IllegalStateException("This factory can only be used at server side!");
                }
            }
            System.out.println("read with instance: " + hazelcastInstance);
        }
    }

    private static class ServerSideCacheLoader implements CacheLoader<Integer, String> {

        static String valueOf(Integer key) {
            return "value-of-" + key;
        }

        @Override
        public String load(Integer key) {
            return valueOf(key);
        }

        @Override
        public Map<Integer, String> loadAll(Iterable<? extends Integer> keys) throws CacheLoaderException {
            Map<Integer, String> result = new HashMap<Integer, String>();
            for (Integer key : keys) {
                String value = load(key);
                result.put(key, value);
            }
            return result;
        }

    }
}

@jerrinot jerrinot added this to the 3.8 milestone Dec 14, 2016

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Dec 14, 2016

hi @JChrist,

many thanks for a great report. Let's see what we can do about it.

@jerrinot jerrinot self-assigned this Dec 14, 2016

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Dec 14, 2016

@JChrist,

can you try to use this in your configuration?

import javax.cache.configuration.Factory;
import javax.cache.configuration.FactoryBuilder;
[...]
Factory<? extends CacheLoader<K, V>> factory = FactoryBuilder.factoryOf("com.package.MapLoaderClass");
[...]
CompleteConfiguration<Integer, String> cacheConfig =
                new CacheConfig<Integer, String>()
                        .setTypes(Integer.class, String.class)
                        .setReadThrough(true)
                        .setCacheLoaderFactory(factory);

The factory itself is still serialized (=it has to be sent to the members somehow), but it depends on the JCache API only and it does not have any compile time or runtime dependency on the actual map loader.

It's just a configuration parameter.

@JChrist

This comment has been minimized.

Copy link
Author

commented Dec 14, 2016

Thanks for the response @jerrinot !
Is there a way to specify this using declarative XML syntax?
Currently I am using this:

<cache name="my_cache">
...
  <read-through>true</read-through>
  <cache-loader-factory class-name="MyCacheLoaderFactory" />
...
</cache>```
I do not know of another way to get at this
@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Dec 14, 2016

@JChrist that's a good one!

I guess there is no way how to use declarative configuration when you do not have a factory class on your classpath. I am afraid you would need to create a copy of the ClassFactory from the JSR107, add an argument-less constructor and have it on a classpath of all members and your client. That's a not good situation to be at.

I see 2 options how to solve it:

  1. introduce a new attribute into the cache-loader-factory . Then you could do something like this:
<cache-loader-factory 
  class-name="javax.cache.configuration.FactoryBuilder$ClassFactory" 
  constructor-argument="com.package.YourMapLoaderImpl" />
  1. introduce a new element cache-loader
<cache-loader class-name="com.package.YourMapLoaderImpl" />

The first option is somewhat more general and it's less invasive into the config schema. but it looks ugly and it's hard to use. The 2nd option requires introducing a new element, but it's clear and simple to use.

Do you have another idea?

@JChrist

This comment has been minimized.

Copy link
Author

commented Dec 14, 2016

The second option would work perfectly for my case and makes everything pretty self-explanatory and easy. The first option, as you mentioned, looks and feels ugly, so I would suggest the second one.

However, there still seems to be the issue that if an implementation actually required a factory (i.e. the CacheLoader instance needed extra configuration/code to be created), clients would either have to have that factory in the classpath, or they wouldn't be able to connect.

I can only guess for solutions to this: being able to recognize if the receiving party is a client, in which case submit null Factory?

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Dec 14, 2016

@JChrist what if the client wants to access/check the factory?

@JChrist

This comment has been minimized.

Copy link
Author

commented Dec 14, 2016

I can't think of a use-case (or at least a real-world one) where a client might need access to resources meant to be for the server only. If there would be any such needed, it wouldn't be a client, but a full member itself.

Actually, I would expect the client to have absolutely no idea even if the server has a factory/loader or not.

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Dec 14, 2016

@JChrist: what if you want to create a new cache with exactly the same configuration as some other well-known cache? or you want to use some other cache as a blueprint for your new cache.

btw: I sent a tentative fix for your use-case. @vbekiaris: can you please have a look?

I also realized there is probably the same problem with various listeners, filters, expiry policy, etc...

@JChrist

This comment has been minimized.

Copy link
Author

commented Dec 15, 2016

@jerrinot Yes, well, I couldn't get my head around why would someone want to create caches on the fly, but, in any case, if such a scenario was desirable, you are correct that what you mentioned would support it.
anyway, thanks a lot for your responses and fix!

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Dec 15, 2016

@JChrist: see this: #592
It's a quite popular feature request. We do not have a general solution yet and JCache is the only structure which supports it at the moment - as it's mandated by the spec.

it was a good conversation with you, let's see what others think about the #9456 fix.

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 24, 2017

@jerrinot

We are facing similar issue in our client cache. We are getting serialization exception while getting the cache and not sure where to put below code:

Factory<? extends CacheLoader<K, V>> factory = FactoryBuilder.factoryOf("com.package.MapLoaderClass");
[...]
CompleteConfiguration<Integer, String> cacheConfig =
new CacheConfig<Integer, String>()
.setTypes(Integer.class, String.class)
.setReadThrough(true)
.setCacheLoaderFactory(factory);

Our client configuration is pretty simple one...
HazelcastInstance instance = HazelcastClient.newHazelcastClient();
CachingProvider provider = getCachingProvider();
manager = provider.getCacheManager();
manager.getCache()

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Jul 24, 2017

@Sushil2055: what are you trying to achieve and what exactly is the exception you are getting?

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 24, 2017

We are having two web application running inside one tomcat server. One web application is loading the data and putting into the cache and basically which is acting as a server. In another web application, we are connecting to that server using hazel cast client and trying to get the loaded data but during deserialization it is asking me to load factory class and loader class.

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 24, 2017

Caused by: com.hazelcast.nio.serialization.HazelcastSerializationException: java.lang.ClassNotFoundException: ***LoaderFactory
at com.hazelcast.internal.serialization.impl.JavaDefaultSerializers$JavaSerializer.read(JavaDefaultSerializers.java:224)
at com.hazelcast.internal.serialization.impl.StreamSerializerAdapter.read(StreamSerializerAdapter.java:48)
at com.hazelcast.internal.serialization.impl.AbstractSerializationService.readObject(AbstractSerializationService.java:266)
at com.hazelcast.internal.serialization.impl.ByteArrayObjectDataInput.readObject(ByteArrayObjectDataInput.java:599)
at com.hazelcast.config.CacheConfig.readData(CacheConfig.java:579)

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 24, 2017

@jerrinot , Please help me resolve it..Let me know if you know any info. thanks in advance.

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Jul 24, 2017

@Sushil2055: what's your JCache configuration?

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 24, 2017

@ Client side ???

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Jul 24, 2017

the side where you actually create the JCache configuration. I assume it's a member-side in your case.

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 24, 2017





<network>
    
	  <cluster-members>
        <address>127.0.0.1:5802</address>
       
    </cluster-members>
    <smart-routing>true</smart-routing>
    <redo-operation>true</redo-operation>
    <connection-timeout>60000</connection-timeout>
    <connection-attempt-period>3000</connection-attempt-period>
    <connection-attempt-limit>2</connection-attempt-limit>
    <socket-options>
        <tcp-no-delay>false</tcp-no-delay>
        <keep-alive>true</keep-alive>
        <reuse-address>true</reuse-address>
        <linger-seconds>3</linger-seconds>
        <timeout>-1</timeout>
        <buffer-size>128</buffer-size>
    </socket-options>
    <socket-interceptor enabled="false">
        <class-name>com.hazelcast.examples.MySocketInterceptor</class-name>
        <properties>
            <property name="foo">bar</property>
        </properties>
    </socket-interceptor>

    <ssl enabled="false">
       
    </ssl>
    <aws enabled="false" connection-timeout-seconds="11">
       
    </aws>

</network>

<dependency>
	<groupId>com.hazelcast</groupId>
	<artifactId>hazelcast-spring</artifactId>
	<version>3.8.3</version>
</dependency>

HazelcastInstance instance = HazelcastClient.newHazelcastClient();
CachingProvider provider = getCachingProvider();
manager = provider.getCacheManager();
manager.getCache()

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 24, 2017

what do your mean member side ? (Client side )

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Jul 24, 2017

that is a Hazelcast instance configuration. I reckon you have a configuration for your JCache (data structure).
something like

<cache name="default">
 [...]
</cache>

or programmatic:

CompleteConfiguration<Integer, String> cacheConfig =
                new CacheConfig<Integer, String>()
                        .setTypes(Integer.class, String.class)
                        .setReadThrough(true)
                        .setCacheLoaderFactory(factory);

can you share this?

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 25, 2017

@Bean
public HazelcastInstance instance(SpringManagedContext managedContext) {
    Config config;
    try {
        String fName = System.getProperty("hazelcast.config");
        config = new FileSystemXmlConfig(fName);
    } catch (FileNotFoundException | NullPointerException ex) {
        try {
            config = new ClasspathXmlConfig("hazelcast_local.xml");
        } catch (IllegalArgumentException ex1) {
            throw new HazelcastException(ex1);
        }
    }
    config.setManagedContext(managedContext);
    return Hazelcast.newHazelcastInstance(config);
}

@Bean(name = "Cache")
public Cache<String, User> getCache(
        Factory<CacheLoader<String, User>> cacheLoaderFactory,
        CacheInitializer<String, User> cacheInitializer,
        HazelcastInstance hci,
        PropertyManager pm) {
    CachingProvider provider = getCachingProvider();
    URI cacheManagerName;
    try {
        cacheManagerName = new URI(hci.getName());
    } catch (URISyntaxException ex) {
        log.error("Bad Hazelcast instance name", ex);
        throw new RuntimeException(ex);
    }

    CacheManager manager = provider.getCacheManager(cacheManagerName, null);
    Cache<String, User> userData= manager.getCache("entitlements", String.class, User.class);
    if (userData== null) {
        MutableConfiguration<String, User> config = new MutableConfiguration<>();
        config.setTypes(String.class, User.class)
                .setReadThrough(true)
                .setCacheLoaderFactory(cacheLoaderFactory);
        entitlements = manager.createCache("entitlements", config);

        IScheduledExecutorService cacheReloader = hci.getScheduledExecutorService("CacheReloader");
        cacheReloader.schedule(cacheInitializer, 0, TimeUnit.SECONDS);
        cacheReloader.scheduleAtFixedRate(cacheInitializer,
                calcRefreshDelay(pm.getProperty(ApexConstants.PROP__REFRESH_START, "02:00:00")),
                parseRefreshInterval(pm.getProperty(ApexConstants.PROP_REFRESH_INTERVAL, "1440")),
                TimeUnit.MINUTES);
    }

    return userData;
}
@Sushil2055

This comment has been minimized.

Copy link

commented Jul 25, 2017

<?xml version="1.0" encoding="UTF-8"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-3.8.xsd"
           xmlns="http://www.hazelcast.com/schema/config"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
	<group>
		<name>local</name>
		<password>local</password>
	</group>
    <network>
        <port>5802</port>
    	<join>
            <multicast enabled="true">
                <multicast-port>54427</multicast-port>
            </multicast>
        	<tcp-ip enabled="false" />
        </join>
    </network>
</hazelcast>
@Sushil2055

This comment has been minimized.

Copy link

commented Jul 25, 2017

Now I am not getting exception but the cache entries are coming as null. Not sure what is wrong.

@jerrinot Please let me know.

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 26, 2017

@jerrinot

Sorry for troubling you again but anything that you can help me..

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2017

I think this part of your config is relevant:

@Bean(name = "Cache")
public Cache<String, User> getCache(
        Factory<CacheLoader<String, User>> cacheLoaderFactory,
        CacheInitializer<String, User> cacheInitializer,
        HazelcastInstance hci,
        PropertyManager pm) {

Now the question is what is Spring passing as implementation of cacheLoaderFactory. Do you see this bean being defined anywhere?

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 26, 2017

Are you looking for this ?

@component
public class LoaderFactory implements Factory<CacheLoader<String, User>>,Serializable {

private static final long serialVersionUID = 1L;

@Override
public CacheLoader<String, User> create() {
    try {
        Future<ApplicationContext> fctx = ApplicationContextProvider.getApplicationContextFuture();
        return fctx.get().getBean(Loader.class);
    } catch (InterruptedException | ExecutionException ex) {
        throw new RuntimeException(ex);
    }
}
@Sushil2055

This comment has been minimized.

Copy link

commented Jul 26, 2017

@jerrinot ...or your are looking for something else ?

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Jul 26, 2017

can you - for the sake of experiement - put the EntitlementsLoader class on a client classpath?

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 26, 2017

I tried that but it didnt worked..

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 26, 2017

When I create that cache , this message is get printed in server logs while cahce is getting created :

com.hazelcast.cache.impl.CacheService
Added cache config: CacheConfig{name='entitlements', managerPrefix='/hz/_hzInstance_1_sushil/', inMemoryFormat=BINARY, backupCount=1, hotRestart=HotRestartConfig{enabled=false, fsync=false}}

Can you please tell me what managerPrefix='/hz/_hzInstance_1_sushil/' ? Is this URI that client should connect to ?..

In Hazelcast client I added this portion of code 👍

manager = provider.getCacheManager(new URI("_hzInstance_1_sushil"),null);

But getting exception as ::

javax.cache.CacheException: Error opening URI [_hzInstance_1_sushil ]

at com.hazelcast.cache.impl.AbstractHazelcastCachingProvider.getCacheManager(AbstractHazelcastCachingProvider.java:99)
at com.hazelcast.cache.impl.AbstractHazelcastCachingProvider.getCacheManager(AbstractHazelcastCachingProvider.java:123)

@jerrinot Very thanks for your reply so far..

@jerrinot

This comment has been minimized.

Copy link
Contributor

commented Jul 27, 2017

@Sushil2055: is your loader factory on your client classpath?

I think at this point the quickest way to help you is this: you prepare a demo project demonstrating the problem and upload it to github. then I can have a look and see what exactly is going on. now it's all just a guesswork..

@Sushil2055

This comment has been minimized.

Copy link

commented Jul 27, 2017

@jerrinot, I was able to get the cache by importing factory class and loader class in client classpath but is their other workaround to get that done ???

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.