ERR unknown command 'CONFIG' when using Secured Redis

[danveloper]

Redis security recommends disabling the CONFIG command so that remote users cannot reconfigure an instance. The RedisHttpSessionConfiguration requires access to this during its initialization. Hosted Redis services, like AWS ElastiCache disable this command by default, with no option to re-enable it.

[rwinch]

Thanks for the report @danveloper! This indeed seems to be a bug with the RedisHttpSessionConfiguration and thus the @EnableRedisHttpSession annotation.

UPDATE Fixing in 1.0.1

As of Spring Session 1.0.1 this can be disabled by exposing ConfigureRedisAction.NO_OP as a bean.

An XML Configuration example

<util:constant
        static-field="org.springframework.session.data.redis.config.ConfigureRedisAction.NO_OP"/>

A Java Configuration example

@Bean
public static ConfigureRedisAction configureRedisAction() {
    return ConfigureRedisAction.NO_OP;
}

Fixing the Issue

I'm debating what the best approach to fixing this would be though and wondering what your thoughts were @danveloper.

There is certainly a need for a fix, so I'm not debating that we need to fix something. However, I like the fact that it updates the Redis configuration by default for two reasons:

• It makes it very easy to get things up and working in a development environment
• Making it enabled by default means that users would have to explicitly disable for production. Since there is an explicit step to disable the configuration, they should be aware that it is necessary to configure Redis to send the namespace notifications. This is critical for applications that require SessionDestroyedEvent to be fired to clean up resources. In particular, this is important for WebSocket applications to ensure open WebSockets are closed when the HttpSession expires.

My initial thoughts on how we should update the configuration is:

• RedisHttpSessionConfiguration should by default update the Redis configuration only if Spring WebSocket support is enabled.
• RedisHttpSessionConfiguration should allow disabling updating the Redis configuration
• RedisHttpSessionConfiguration should by default try to subscribe to keyspace notifications only if Spring WebSocket support is enabled. This will help increase performance for applications simply using Spring Session for HttpSession which typically does not need to receive the SessionDestroyedEvent
• RedisHttpSessionConfiguration should allow explicitly configuring if the application should subscribe to keyspace notifications
• We should update the documentation to discuss the changes

Workaround

In the meantime, a workaround is to remove @EnableRedisHttpSession from your configuration and then include a configuration with a fix. For example:

import java.util.Arrays;
import java.util.List;
import java.util.Map;

import org.springframework.beans.factory.BeanClassLoaderAware;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.ApplicationEventPublisher;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.ImportAware;
import org.springframework.core.annotation.AnnotationAttributes;
import org.springframework.core.annotation.AnnotationUtils;
import org.springframework.core.type.AnnotationMetadata;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.PatternTopic;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.session.ExpiringSession;
import org.springframework.session.SessionRepository;
import org.springframework.session.data.redis.RedisOperationsSessionRepository;
import org.springframework.session.data.redis.SessionMessageListener;
import org.springframework.session.web.http.HttpSessionStrategy;
import org.springframework.session.web.http.SessionRepositoryFilter;
import org.springframework.util.ClassUtils;

@Configuration
@EnableScheduling
public class RedisHttpSessionConfiguration {
    @Value("${spring.session.maxInactive ?: 1800}")
    private Integer maxInactiveIntervalInSeconds;

    private HttpSessionStrategy httpSessionStrategy;

    @Autowired
    private ApplicationEventPublisher eventPublisher;

    @Bean
    public RedisMessageListenerContainer redisMessageListenerContainer(
            RedisConnectionFactory connectionFactory) {
        RedisMessageListenerContainer container = new RedisMessageListenerContainer();
        container.setConnectionFactory(connectionFactory);
        container.addMessageListener(redisSessionMessageListener(),
                Arrays.asList(new PatternTopic("__keyevent@*:del"),new PatternTopic("__keyevent@*:expired")));
        return container;
    }

    @Bean
    public SessionMessageListener redisSessionMessageListener() {
        return new SessionMessageListener(eventPublisher);
    }

    @Bean
    public RedisTemplate<String,ExpiringSession> sessionRedisTemplate(RedisConnectionFactory connectionFactory) {
        RedisTemplate<String, ExpiringSession> template = new RedisTemplate<String, ExpiringSession>();
        template.setKeySerializer(new StringRedisSerializer());
        template.setHashKeySerializer(new StringRedisSerializer());
        template.setConnectionFactory(connectionFactory);
        return template;
    }

    @Bean
    public RedisOperationsSessionRepository sessionRepository(RedisTemplate<String, ExpiringSession> sessionRedisTemplate) {
        RedisOperationsSessionRepository sessionRepository = new RedisOperationsSessionRepository(sessionRedisTemplate);
        sessionRepository.setDefaultMaxInactiveInterval(maxInactiveIntervalInSeconds);
        return sessionRepository;
    }

    @Bean
    public <S extends ExpiringSession> SessionRepositoryFilter<? extends ExpiringSession> springSessionRepositoryFilter(SessionRepository<S> sessionRepository) {
        SessionRepositoryFilter<S> sessionRepositoryFilter = new SessionRepositoryFilter<S>(sessionRepository);
        if(httpSessionStrategy != null) {
            sessionRepositoryFilter.setHttpSessionStrategy(httpSessionStrategy);
        }
        return sessionRepositoryFilter;
    }

    @Autowired(required = false)
    public void setHttpSessionStrategy(HttpSessionStrategy httpSessionStrategy) {
        this.httpSessionStrategy = httpSessionStrategy;
    }
}

If you are not using the SessionDestroyedEvent you can also disable subscribing to the notifications which should improve performance. For example:

import java.util.Arrays;
import java.util.List;
import java.util.Map;

import org.springframework.beans.factory.BeanClassLoaderAware;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.ApplicationEventPublisher;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.ImportAware;
import org.springframework.core.annotation.AnnotationAttributes;
import org.springframework.core.annotation.AnnotationUtils;
import org.springframework.core.type.AnnotationMetadata;
import org.springframework.data.redis.connection.RedisConnection;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.PatternTopic;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.session.ExpiringSession;
import org.springframework.session.SessionRepository;
import org.springframework.session.data.redis.RedisOperationsSessionRepository;
import org.springframework.session.data.redis.SessionMessageListener;
import org.springframework.session.web.http.HttpSessionStrategy;
import org.springframework.session.web.http.SessionRepositoryFilter;
import org.springframework.util.ClassUtils;

@Configuration
public class RedisHttpSessionConfiguration {
    @Value("${spring.session.maxInactive ?: 1800}")
    private Integer maxInactiveIntervalInSeconds;

    private HttpSessionStrategy httpSessionStrategy;

    @Bean
    public RedisTemplate<String,ExpiringSession> sessionRedisTemplate(RedisConnectionFactory connectionFactory) {
        RedisTemplate<String, ExpiringSession> template = new RedisTemplate<String, ExpiringSession>();
        template.setKeySerializer(new StringRedisSerializer());
        template.setHashKeySerializer(new StringRedisSerializer());
        template.setConnectionFactory(connectionFactory);
        return template;
    }

    @Bean
    public RedisOperationsSessionRepository sessionRepository(RedisTemplate<String, ExpiringSession> sessionRedisTemplate) {
        RedisOperationsSessionRepository sessionRepository = new RedisOperationsSessionRepository(sessionRedisTemplate);
        sessionRepository.setDefaultMaxInactiveInterval(maxInactiveIntervalInSeconds);
        return sessionRepository;
    }

    @Bean
    public <S extends ExpiringSession> SessionRepositoryFilter<? extends ExpiringSession> springSessionRepositoryFilter(SessionRepository<S> sessionRepository) {
        SessionRepositoryFilter<S> sessionRepositoryFilter = new SessionRepositoryFilter<S>(sessionRepository);
        if(httpSessionStrategy != null) {
            sessionRepositoryFilter.setHttpSessionStrategy(httpSessionStrategy);
        }
        return sessionRepositoryFilter;
    }

    @Autowired(required = false)
    public void setHttpSessionStrategy(HttpSessionStrategy httpSessionStrategy) {
        this.httpSessionStrategy = httpSessionStrategy;
    }
}

[danveloper]

I think it should be enabled by default, but fail gracefully with a warning. This would allow the same configuration to be used between dev and prod, where dev would JustWork(tm) and prod would require some manual intervention (which would be obvious from the warning).

I was able to work around the problem by subclassing the RedisHttpSessionConfiguration with an implementation that disables the keyspace notifications initializer, and bringing it in through normal configuration means:

package org.springframework.session.data.redis.config.annotation.web.http

import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.core.type.AnnotationMetadata
import org.springframework.data.redis.connection.RedisConnectionFactory
import org.springframework.data.redis.core.RedisTemplate
import org.springframework.session.ExpiringSession
import org.springframework.session.data.redis.RedisOperationsSessionRepository

@Configuration
class GateRedisHttpSessionConfiguration extends RedisHttpSessionConfiguration {

  @Value('${session.expiration:1800}')
  int expiration

  public void setImportMetadata(AnnotationMetadata importMetadata) {
  }

  @Bean
  public RedisOperationsSessionRepository sessionRepository(RedisTemplate<String, ExpiringSession> sessionRedisTemplate) {
    RedisOperationsSessionRepository sessionRepository = new RedisOperationsSessionRepository(sessionRedisTemplate);
    sessionRepository.setDefaultMaxInactiveInterval(expiration);
    return sessionRepository;
  }

  @Override
  public RedisHttpSessionConfiguration.EnableRedisKeyspaceNotificationsInitializer enableRedisKeyspaceNotificationsInitializer(RedisConnectionFactory connectionFactory) {
    null
  }
}

---

For posterity, here are the steps for enabling the keyspace notifications on AWS:

Log into the AWS console and choose the ElastiCache service

Choose the Cache Parameter Groups and click Create Parameter Group

Give the new group and name and description and click Create

With the new parameter group created, select it and click Edit Parameters

Page through the parameters until you find notify-keyspace-events and enter "eA" in the Value field and click Save Changes

Choose Cache Clusters from the context navigation and create a new Redis cache cluster

When specifying your cluster detail, choose the newly created parameter group

[rwinch]

@danveloper I'm glad you were able to work around the issue. Thank you for your feedback and for the useful directions on enabling keyspace notifications in AWS.

I have concerns with the configuration not failing when it tries to update the Redis configuration.

Developers likely did thorough testing within another environment. For example, they might be integrating with WebSocket support and validate that when an HttpSession is terminated the WebSocket is indeed closing properly.

This means they are confident that everything is working by the time the get to production. Many users may not even notice a warning being logged. Without Redis keyspace notifications being enabled, the applications WebSocket connections will not properly closed when the HttpSession is destroyed.

Obviously developers should read the instructions completely, look for warnings, and perform some smoke tests in production to avoid such issues. However, I fear that by not failing we are setting up users for failure.

A compromise might be to require users to explicitly set a property to allow "fall back". This means users would be aware that they need to do something in production. Of course this is one more piece of configuration users would need. Thoughts?

[danveloper]

Could it provide a different strategy for ascertaining the database's configuration? Maybe in the absence of the CONFIG command, the event notification initializer could run a test to determine if expirations were working? This might introduce some slowness during startup, but might be favorable to total failure.

I'd probably prefer a slower startup over managing feature flags between dev & prod. If this isn't possible or is otherwise infeasible, then feature flag might be the only way to go.

[rwinch]

Great idea! I think this may be doable, but we will see when I get into the details. Thanks for your feedback!

[njariwala]

RedisHttpSessionConfiguration.EnableRedisKeyspaceNotificationsInitializer is not visible for outsiders extending RedisHttpSessionConfiguration. The workaround doesn't work. Can you help me with this?

[rwinch]

@njariwala You would need to place it in the same package as illustrated in the example above.

[chrylis]

If the class is semantically usable for non-core extensions, it seems like a bad practice to trespass on the distribution package, especially since that can fail in interesting ways with particular classloader hierarchies.

[rwinch]

@chrylis Yes this is considered a workaround until we can get a fix out. The alternative is to use the more verbose workaround I provided.

[andirdju]

@rwinch , @danveloper ,
I think calling the config commands is a bit like hibernate.hbm2ddl.auto option, its better to make it disabled by default.

what do you think?

[rwinch]

@andirdju Thanks for your thoughts. I think this is quite a bit different:

• Hibernate would fail to store any data if the schema is not created. The failure would be obvious to anyone who tested the application
• If Redis is not configured correctly, the sessions would still be created. The failure is that session timeouts would not be handled properly (causing a security vulnerability) The failure would not be obvious unless someone waited until the session timed out and verified it cleaned up the related resources (i.e. websocket connections)

[rwinch]

I've been giving this quite a bit of thought and I think the best route is to:

• Add a flag that disables the automatic configuration
• Update the error message to include more details about why it isn't working

The reason being:

• This approach allows secure us to be secure by default (we know session expiration events will be fired)
• The approach works by default out of the box. If there is a problem, then users are informed and know what steps they need to do.
• Adding a check to startup time can have a significant impact to startup time. It also means that the Redis cluster must be up at the same time the application is (a disable flag would prevent this)
• If users want the same configuration between production and dev, they should setup their Redis instance the same. It is not that difficult and a onetime thing to setup the dev instance to broadcast events)
• We could also allow for the flag to be set with the Spring Environment which means we could disable the check via a System Property.

@danveloper I know you disagreed with this approach, so I'd like to encourage a response from you about this.

Anyone else following this issue, I'd love to hear your feedback as well.

[ccit-spence]

@rwinch Is there anything wrong with adding @Profile(value = "production") to the beans so that local Redis testing is still allowed?

[rwinch]

@ccit-spence I generally do not like to use a different setup for production than development. The problem is that you are testing your production setup throughout development which means you are almost sure to run into "unforeseen" problems. Given Redis is so easy to setup, I would highly recommend using the same setup in development as production.

With all that said, you are obviously free to do as you choose. Using a profile will work, but I wouldn't recommend it for the reasons above.

[ccit-spence]

@rwinch We basically have 2 sets of testing. One: local for making sure nothing obvious is broken, Two: Test ran on CI servers i.e. production. A problem good or bad is from a dev machine you can't access Elasticache. Elasticache is only accessible via EC2 instances.

But yes, I do agree having 2 sets of tests is not something I like to do. In this case since Elasticache is only accessible from EC2 I don't see any other way for a dev to test locally.

[rwinch]

@ccit-spence Are you just using Redis within Elasticache or do you have a custom implementation of Spring Sessions' SessionRepository? If you are just using Redis within Elasticache, you can easily install Redis locally.

[ccit-spence]

@rwinch Right now it is pretty basic. Just using Spring Session defaults. Main motivation for Elasticache is not having to maintain the cluster/nodes.

[rwinch]

@ccit-spence So you could run a local Redis instance right? Not saying you have to, but it seems like an option that would make the development and production much more similar. Anyways, the choice is totally up to you :)

[oak-tree]

I'm reopen this issue because if facing the same issue on azure with their redis server. Any idea how to over come this as I cannot change some of the config. According to the doc:

https://azure.microsoft.com/en-us/documentation/articles/cache-configure/

Important: Because configuration and management of Azure Redis Cache instances is managed by Microsoft the following commands are disabled. If you try to invoke them you will receive an error message similar to "(error) ERR unknown command".

[rwinch]

@oak-tree See #124 (comment) and http://docs.spring.io/spring-session/docs/current/reference/html5/#api-redisoperationssessionrepository-sessiondestroyedevent

[oak-tree]

Thanks!

[amit-jadhav]

I am facing the same issue even after trying the solutions provided above

[JesusIgnacio]

I think it should be enabled by default, but fail gracefully with a warning. This would allow the same configuration to be used between dev and prod, where dev would JustWork(tm) and prod would require some manual intervention (which would be obvious from the warning).

I was able to work around the problem by subclassing the RedisHttpSessionConfiguration with an implementation that disables the keyspace notifications initializer, and bringing it in through normal configuration means:

package org.springframework.session.data.redis.config.annotation.web.http

import org.springframework.beans.factory.annotation.Value
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.core.type.AnnotationMetadata
import org.springframework.data.redis.connection.RedisConnectionFactory
import org.springframework.data.redis.core.RedisTemplate
import org.springframework.session.ExpiringSession
import org.springframework.session.data.redis.RedisOperationsSessionRepository

@Configuration
class GateRedisHttpSessionConfiguration extends RedisHttpSessionConfiguration {

  @Value('${session.expiration:1800}')
  int expiration

  public void setImportMetadata(AnnotationMetadata importMetadata) {
  }

  @Bean
  public RedisOperationsSessionRepository sessionRepository(RedisTemplate<String, ExpiringSession> sessionRedisTemplate) {
    RedisOperationsSessionRepository sessionRepository = new RedisOperationsSessionRepository(sessionRedisTemplate);
    sessionRepository.setDefaultMaxInactiveInterval(expiration);
    return sessionRepository;
  }

  @Override
  public RedisHttpSessionConfiguration.EnableRedisKeyspaceNotificationsInitializer enableRedisKeyspaceNotificationsInitializer(RedisConnectionFactory connectionFactory) {
    null
  }
}

For posterity, here are the steps for enabling the keyspace notifications on AWS:

Log into the AWS console and choose the ElastiCache service

Choose the Cache Parameter Groups and click Create Parameter Group

Give the new group and name and description and click Create

With the new parameter group created, select it and click Edit Parameters

Page through the parameters until you find notify-keyspace-events and enter "eA" in the Value field and click Save Changes__

__ #### Choose Cache Clusters from the context navigation and create a new Redis cache cluster #### When specifying your cluster detail, choose the newly created parameter group __

The value to set in notify-keyspace-events obbeys to the below information :

K — Keyspace events, published with a prefix of __keyspace@<db>__
E — Key-event events, published with a prefix of __keyevent@<db>__
g — Generic, non-specific commands such as DEL, EXPIRE, RENAME, etc.
$ — String commands
l — List commands
s — Set commands
h — Hash commands
z — Sorted set commands
x — Expired events (events generated every time a key expires)
e — Evicted events (events generated when a key is evicted for maxmemory)
A — An alias for g$lshzxe

This information is located in https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/ParameterGroups.Redis.html#ParameterGroups.Redis.2-8-6

Thanks @danveloper for your explanation.

[LukasVyhlidka]

When I configured the elasticache (Redis on AWS) using the guide from previous post I fugured out that the eA parameter for notify-keyspace-events does not work correctly. Correct configuration is EA (E with capital letter).
In the documentation (https://redis.io/topics/notifications) you can see that the A is an alias e.g. for e which means, that eA is effectively just A.
Redis Spring Session functionality needs this E because it consumes those key-events.
So, specify there EA.

[ViliusS]

Did anyone found how to use this with Memorystore for Redis on Google Cloud? It looks like it has the same issue, however it doesn't have any interface to control data :|

[jackstraw66]

We encountered this issue recently after securing our AWS Redis (ElasticCache) 6.2.6 cluster (running in clustered mode). Adding a ConfigureRedisAction.NO_OP bean meant adding the spring-session-data-redis (v2.5.1) dependency, previously not required.

Issue
This has led to sporadic exceptions on startup when the container is subscribing to patterned topics.

Exception in thread "springSessionRedisMessageListenerContainer-2" java.lang.NullPointerException
	at redis.clients.jedis.BinaryJedisPubSub.psubscribe(BinaryJedisPubSub.java:57)
	at org.springframework.data.redis.connection.jedis.JedisSubscription.doPsubscribe(JedisSubscription.java:59)
	at org.springframework.data.redis.connection.util.AbstractSubscription.pSubscribe(AbstractSubscription.java:159)
	at org.springframework.data.redis.listener.RedisMessageListenerContainer$SubscriptionTask$PatternSubscriptionTask.run(RedisMessageListenerContainer.java:707)
	at java.lang.Thread.run(Thread.java:750)

When these failures occur, keyspace events are no longer received. I believe the sporadic nature of these exceptions is due to the potentially asynchronous behavior of PatternSubscriptionTask.

Application Details
We do not use any patterned topics in our application, and only use ChannelTopics. I understand Spring Session may be subscribing to patterned topics (e.g. spring:session:event:0:created:*)?

The notify-keyspace-events property is configured with KE$. (We only care about the keyspace events on strings.)

I learned another aspect of subscribing to keyspace events in a secured Redis cluster is that the application must subscribe to the topic on each node in the cluster. Redis keystore events are not broadcast cluster-wide, and are only broadcast to topics local to that node. We have 9 nodes in our cluster and the application is configured to subscribe to the topic on each node.

Could this be causing issues with how RedisHttpSessionConfiguration is subscribing to its patterned topics?

Is there a way to disable Spring Session from subscribing to these patterned topics, or disabling Spring Session altogether?

Like I mentioned above, we didn't need Spring Session and only included it when we needed to add the ConfigureRedisAction.NO_OP bean.

Desired Result
We want to prevent Spring Redis from auto-configuring our Redis cluster, and for the application to subscribe to the topic on each node of the cluster to receive the keyspace notifications. We don't care about "delete" or "expiration" notifications - only notifications triggered by the SET operations on a key in the keystore.

[vpavic]

Adding a ConfigureRedisAction.NO_OP bean meant adding the spring-session-data-redis (v2.5.1) dependency, previously not required.

That doesn't sound right - ConfigureRedisAction.NO_OP is intended specifically as a configuration option for Spring Session's Redis integration.

If you're not using Spring Session at all, you need something else to resolve the issue. I'd suggest taking a look at Spring Data Redis reference manual, or opening an issue over there if you don't find anything.