docker-compose up throws redis exception at startup

[dragorosson]

Describe the bug

This is a follow-on to #3534. That specific issue is resolved by #3535, but further problems remain. When conductor-server starts up now with redis, which was added in #3435, it throws an exception:

18435 [main] INFO  com.netflix.conductor.redis.config.RedisSentinelConfiguration [] - Starting conductor server using redis_standalone.
2023-03-15 15:45:40 38812 [main] ERROR com.netflix.conductor.redis.dao.RedisMetadataDAO [] - refresh TaskDefs failed 
2023-03-15 15:45:40 redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
2023-03-15 15:45:40     at redis.clients.jedis.util.Pool.getResource(Pool.java:84) ~[jedis-3.8.0.jar!/:?]
2023-03-15 15:45:40     at redis.clients.jedis.JedisPool.getResource(JedisPool.java:377) ~[jedis-3.8.0.jar!/:?]
2023-03-15 15:45:40     at com.netflix.conductor.redis.jedis.JedisStandalone.executeInJedis(JedisStandalone.java:53) ~[conductor-redis-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
2023-03-15 15:45:40     at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]

Details
Conductor version: 3f81d3f (current main)
Persistence implementation: whatever comes with the default docker-compose up now
Queue implementation: RabbitMQ I guess
Lock: Redis?
Workflow definition: N/A
Task definition: N/A
Event handler definition: N/A

To Reproduce
Steps to reproduce the behavior:

1. Use the docker-compose up tutorial at https://conductor.netflix.com/devguide/running/docker.html

[bdparrish]

I'll add to this. I was able to recreate this as well. I had to remove the health check for RabbitMQ because Docker will not see that it is healthy in the current state because it hangs up in a weird state. Once you remove the health check from conductor-server everything comes up. Here is the entire stack trace from conductor-server.

conductor-server    | 136932 [pool-18-thread-1] ERROR com.netflix.conductor.core.execution.tasks.SystemTaskWorker [] - Error polling system task in queue:SUB_WORKFLOW
conductor-server    | java.lang.RuntimeException: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
conductor-server    | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.loadScript(RedisQueueMonitor.java:81) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server    | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.<init>(RedisQueueMonitor.java:39) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server    | 	at io.orkes.conductor.mq.redis.single.ConductorRedisQueue.<init>(ConductorRedisQueue.java:50) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server    | 	at io.orkes.conductor.queue.dao.RedisQueueDAO.getConductorQueue(RedisQueueDAO.java:47) ~[orkes-conductor-queues-1.0.3.jar!/:?]
conductor-server    | 	at io.orkes.conductor.queue.dao.BaseRedisQueueDAO.lambda$get$0(BaseRedisQueueDAO.java:57) ~[orkes-conductor-queues-1.0.3.jar!/:?]
conductor-server    | 	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source) ~[?:?]
conductor-server    | 	at io.orkes.conductor.queue.dao.BaseRedisQueueDAO.get(BaseRedisQueueDAO.java:57) ~[orkes-conductor-queues-1.0.3.jar!/:?]
conductor-server    | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
conductor-server    | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
conductor-server    | 	at java.lang.Thread.run(Unknown Source) ~[?:?]
conductor-server    | Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
conductor-server    | 	at redis.clients.jedis.util.Pool.getResource(Pool.java:84) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:377) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:15) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
conductor-server    | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
conductor-server    | 	at java.lang.Thread.run(Unknown Source) ~[?:?]
conductor-server    | Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
conductor-server    | 	at redis.clients.jedis.util.Pool.getResource(Pool.java:84) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:377) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:15) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.loadScript(RedisQueueMonitor.java:75) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server    | 	... 15 more
conductor-server    | Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Failed to create socket.
conductor-server    | 	at redis.clients.jedis.DefaultJedisSocketFactory.createSocket(DefaultJedisSocketFactory.java:110) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at redis.clients.jedis.Connection.connect(Connection.java:226) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:144) ~[jedis-3.8.0.jar!/:?]
conductor-server    | 	at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:314) ~[jedis-3.8.0.jar!/:?]

[karczewsky]

I tried running the Conductor locally too and got the same problems: with Redis and with RabbitMQ.

Talking about Redis I think there are 2 issues:

1. port misconfiguration in config-local.properties
2. despite fixing the above I was still having problems while containers were using internal network & trying to resolve hostname rs. I've removed it (switched to default network) and everything seems to work fine now

Above changes made on a fork: 20eab6d

[bdparrish]

Good points @karczewsky - I was able to add redis to the internal network and it worked as expected. However now is the next issue...

On startup es is failing -

3665 [main] ERROR com.netflix.conductor.es6.dao.index.ElasticSearchRestDAOV6 [] - Failed to init template_task_log
conductor-server    | org.elasticsearch.client.ResponseException: method [PUT], host [http://es:9200], URI [/_template/template_task_log], status line [HTTP/1.1 400 Bad Request]
conductor-server    | Warnings: [Legacy index templates are deprecated in favor of composable templates.]
conductor-server    | {"error":{"root_cause":[{"type":"parse_exception","reason":"unknown key [template] in the template "}],"type":"parse_exception","reason":"unknown key [template] in the template "},"status":400}
conductor-server    | 	at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:936) ~[elasticsearch-rest-client-6.8.12.jar!/:6.8.12]
conductor-server    | 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:233) ~[elasticsearch-rest-client-6.8.12.jar!/:6.8.12]
conductor-server    | 	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:327) ~[elasticsearch-rest-client-6.8.12.jar!/:6.8.12]
conductor-server    | 	at com.netflix.conductor.es6.dao.index.ElasticSearchRestDAOV6.initIndexTemplate(ElasticSearchRestDAOV6.java:274) ~[conductor-es6-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
conductor-server    | 	at com.netflix.conductor.es6.dao.index.ElasticSearchRestDAOV6.initIndexesTemplates(ElasticSearchRestDAOV6.java:257) ~[conductor-es6-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
conductor-server    | 	at com.netflix.conductor.es6.dao.index.ElasticSearchRestDAOV6.createIndexesTemplates(ElasticSearchRestDAOV6.java:247) ~[conductor-es6-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
conductor-server    | 	at com.netflix.conductor.es6.dao.index.ElasticSearchRestDAOV6.setup(ElasticSearchRestDAOV6.java:239) ~[conductor-es6-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
conductor-server    | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
conductor-server    | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
conductor-server    | 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
conductor-server    | 	at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
conductor-server    | 	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:440) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.addCandidateEntry(DefaultListableBeanFactory.java:1609) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:1573) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveMultipleBeans(DefaultListableBeanFactory.java:1462) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1349) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955) ~[spring-beans-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918) ~[spring-context-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583) ~[spring-context-5.3.22.jar!/:5.3.22]
conductor-server    | 	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:147) ~[spring-boot-2.7.3.jar!/:2.7.3]
conductor-server    | 	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734) ~[spring-boot-2.7.3.jar!/:2.7.3]
conductor-server    | 	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408) ~[spring-boot-2.7.3.jar!/:2.7.3]
conductor-server    | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) ~[spring-boot-2.7.3.jar!/:2.7.3]
conductor-server    | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) ~[spring-boot-2.7.3.jar!/:2.7.3]
conductor-server    | 	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) ~[spring-boot-2.7.3.jar!/:2.7.3]
conductor-server    | 	at com.netflix.conductor.Conductor.main(Conductor.java:39) ~[classes!/:3.14.0-SNAPSHOT]
conductor-server    | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
conductor-server    | 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
conductor-server    | 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
conductor-server    | 	at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
conductor-server    | 	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) ~[conductor-server-3.14.0-SNAPSHOT-boot.jar:3.14.0-SNAPSHOT]
conductor-server    | 	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) ~[conductor-server-3.14.0-SNAPSHOT-boot.jar:3.14.0-SNAPSHOT]
conductor-server    | 	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) ~[conductor-server-3.14.0-SNAPSHOT-boot.jar:3.14.0-SNAPSHOT]
conductor-server    | 	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65) ~[conductor-server-3.14.0-SNAPSHOT-boot.jar:3.14.0-SNAPSHOT]
conductor-server    | Caused by: org.elasticsearch.client.ResponseException: method [PUT], host [http://es:9200], URI [/_template/template_task_log], status line [HTTP/1.1 400 Bad Request]
conductor-server    | Warnings: [Legacy index templates are deprecated in favor of composable templates.]
conductor-server    | {"error":{"root_cause":[{"type":"parse_exception","reason":"unknown key [template] in the template "}],"type":"parse_exception","reason":"unknown key [template] in the template "},"status":400}
conductor-server    | 	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:552) ~[elasticsearch-rest-client-6.8.12.jar!/:6.8.12]
conductor-server    | 	at org.elasticsearch.client.RestClient$1.completed(RestClient.java:537) ~[elasticsearch-rest-client-6.8.12.jar!/:6.8.12]
conductor-server    | 	at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) ~[httpcore-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182) ~[httpasyncclient-4.1.5.jar!/:4.1.5]
conductor-server    | 	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87) ~[httpasyncclient-4.1.5.jar!/:4.1.5]
conductor-server    | 	at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40) ~[httpasyncclient-4.1.5.jar!/:4.1.5]
conductor-server    | 	at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[httpcore-nio-4.4.15.jar!/:4.4.15]
conductor-server    | 	at java.lang.Thread.run(Unknown Source) ~[?:?]

[karczewsky]

Heads Up: I'm pretty new to the Conductor.

I see that recently there was a change in docker-compose from elastic 6.x.x to 8.x.x. Maybe that's a clue - could be some breaking API change. The code seems to require either es6 or es7.

[bdparrish]

I am new as well.

Yes, this is all from the recent update. Docker works as expected with the previous configuration.

I want to get the updated changes because it looks like some items are in pace to make SQS connections easier.

[dragorosson]

Good points @karczewsky - I was able to add redis to the internal network and it worked as expected. However now is the next issue...

On startup es is failing -

The elasticsearch image bump to 8.6.2 was in #3503. I was seeing the es failure too. I wanted to create an issue for it, but when I tried to reproduce it again by just reverting #3435 (leaving #3503) I wasn't seeing the exception. I don't know what would make that difference, but I wanted to throw that out there.

[bdparrish]

I didn't see anything in the diff between the two that stands out, but this is killing my analysis to use Netflix Conductor at all.

[v1r3n]

@bdparrish we are going to take a look.

[darcyzhc]

I encountered a similar problem:

conductor-server  | java.lang.RuntimeException: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
conductor-server  | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.loadScript(RedisQueueMonitor.java:81) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server  | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.<init>(RedisQueueMonitor.java:39) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server  | 	at io.orkes.conductor.mq.redis.single.ConductorRedisQueue.<init>(ConductorRedisQueue.java:50) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server  | 	at io.orkes.conductor.queue.dao.RedisQueueDAO.getConductorQueue(RedisQueueDAO.java:47) ~[orkes-conductor-queues-1.0.3.jar!/:?]
conductor-server  | 	at io.orkes.conductor.queue.dao.BaseRedisQueueDAO.lambda$get$0(BaseRedisQueueDAO.java:57) ~[orkes-conductor-queues-1.0.3.jar!/:?]
conductor-server  | 	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source) ~[?:?]
conductor-server  | 	at io.orkes.conductor.queue.dao.BaseRedisQueueDAO.get(BaseRedisQueueDAO.java:57) ~[orkes-conductor-queues-1.0.3.jar!/:?]
conductor-server  | 	at io.orkes.conductor.queue.dao.BaseRedisQueueDAO.pop(BaseRedisQueueDAO.java:108) ~[orkes-conductor-queues-1.0.3.jar!/:?]
conductor-server  | 	at com.netflix.conductor.core.execution.tasks.SystemTaskWorker.pollAndExecute(SystemTaskWorker.java:111) ~[conductor-core-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
conductor-server  | 	at com.netflix.conductor.core.execution.tasks.SystemTaskWorker.lambda$startPolling$0(SystemTaskWorker.java:81) ~[conductor-core-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
conductor-server  | 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?]
conductor-server  | 	at java.util.concurrent.FutureTask.runAndReset(Unknown Source) ~[?:?]
conductor-server  | 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[?:?]
conductor-server  | 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
conductor-server  | 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
conductor-server  | 	at java.lang.Thread.run(Unknown Source) ~[?:?]
conductor-server  | Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
conductor-server  | 	at redis.clients.jedis.util.Pool.getResource(Pool.java:84) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:377) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:15) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.loadScript(RedisQueueMonitor.java:75) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server  | 	... 15 more
conductor-server  | Caused by: redis.clients.jedis.exceptions.JedisConnectionException: Failed to create socket.
conductor-server  | 	at redis.clients.jedis.DefaultJedisSocketFactory.createSocket(DefaultJedisSocketFactory.java:110) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.Connection.connect(Connection.java:226) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:144) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:314) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryJedis.initializeFromClientConfig(BinaryJedis.java:92) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryJedis.<init>(BinaryJedis.java:297) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.Jedis.<init>(Jedis.java:169) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:177) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:571) ~[commons-pool2-2.11.1.jar!/:2.11.1]
conductor-server  | 	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:298) ~[commons-pool2-2.11.1.jar!/:2.11.1]
conductor-server  | 	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:223) ~[commons-pool2-2.11.1.jar!/:2.11.1]
conductor-server  | 	at redis.clients.jedis.util.Pool.getResource(Pool.java:75) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:377) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:15) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.loadScript(RedisQueueMonitor.java:75) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server  | 	... 15 more
conductor-server  | Caused by: java.net.UnknownHostException: rs
conductor-server  | 	at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[?:?]
conductor-server  | 	at java.net.SocksSocketImpl.connect(Unknown Source) ~[?:?]
conductor-server  | 	at java.net.Socket.connect(Unknown Source) ~[?:?]
conductor-server  | 	at redis.clients.jedis.DefaultJedisSocketFactory.createSocket(DefaultJedisSocketFactory.java:80) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.Connection.connect(Connection.java:226) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:144) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:314) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryJedis.initializeFromClientConfig(BinaryJedis.java:92) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.BinaryJedis.<init>(BinaryJedis.java:297) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.Jedis.<init>(Jedis.java:169) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:177) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:571) ~[commons-pool2-2.11.1.jar!/:2.11.1]
conductor-server  | 	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:298) ~[commons-pool2-2.11.1.jar!/:2.11.1]
conductor-server  | 	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:223) ~[commons-pool2-2.11.1.jar!/:2.11.1]
conductor-server  | 	at redis.clients.jedis.util.Pool.getResource(Pool.java:75) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:377) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at redis.clients.jedis.JedisPool.getResource(JedisPool.java:15) ~[jedis-3.8.0.jar!/:?]
conductor-server  | 	at io.orkes.conductor.mq.redis.single.RedisQueueMonitor.loadScript(RedisQueueMonitor.java:75) ~[orkes-redis-queues-1.0.3.jar!/:?]
conductor-server  | 	... 15 more

Is this same to yours? And is this solved now? It really messed me up.

[Graynder]

hello,

Any news about this issue and how to fix it ?
Currently, I use conductor with postgres and no problem with the version 3.13.3 but with the new version (tag 3.13.5), I have many logs as @darcyzhc

[akramh]

I am having the same issue right off the bat when running docker compose up. I am using WSL 2.0 and getting failure on Rabbitmq. Anyone able to figure it out!

dependency failed to start: container for service "rabbitmq" exited (0)

[dragorosson]

I just noticed #3543 which was opened two weeks ago, approved, but still not merged. @v1r3n @manan164 what's the status of the PR and does it fix this issue?

[coolabhi95]

@Graynder I seem to be facing issue with connecting to postgres, can you help me with what changes you did to connect to postgres?

[coolabhi95]

@darcyzhc were you able to resolve the issue?

[RomanAlberdaSoftserveInc]

I'm getting the same issue. Does anyone resolve it?

[Graynder]

no fix discover so I use old version off conductor because I don't want to install a redis for the moment ^^'

[srichandar]

i'm also stuck with the same issue, i tried using postgres and dynomite but both doesn't work. pls suggest old stable version to proceed where postgres and dynomite docker works

[JoshCrosby]

same... stuck on this issue.

[riemann89]

any updates on this?

[missioncoding]

I am hitting same issue when trying to execute with postgres yaml. I am struck with this issue. please provide a solution for this. thanks

[manan164]

Hi @RomanAlberdaSoftserveInc , @Graynder , @srichandar , @JoshCrosby @riemann89 and @missioncoding

Can you please share the configuration you are using? Please mask the sensitive fields.
This PR fixes the redis issue.
We can chat here for more realtime collaboration.

[missioncoding]

Hello @manan164 thanks for the response. I didn't make any configuration change apart from adding the conductor.postgres.persistance runtime dependency in the server/build.gradle which solves another issue of MetaDataDao constructor parameter.

After which i applied docker compose -f docker-compose.yaml -f docker-compose-postgres.yaml build command followed with up

the exceptions traces going infinitely

no changes apart from these steps.

[manan164]

Hi all,
I have raised PR for the fix,#3692
Please pull the changes manually if this is blocking the use case.

[manan164]

Folks, Just a note on this, If you are still seeing the issue please run docker compose build before running docker compose up since the docker image needs to pick the property file changes done in the #3692

[Graynder]

Hi @manan164 , from what I can see in the PR, a Redis service is added, but that's precisely the problem. With version 3.13.3 (the version I'm currently using), there's no need for Redis, but with version 3.13.5, I have the impression that a Redis service is required for it to work, and this is a change of breacking from my point of view, which is blocking migration to a higher version.

[manan164]

Hi @Graynder , Since we are now using orkes-queue, redis is required. Do you see any issue in 3.13.3 ?