Run SD Redis test suite against GemFire/Geode's Redis server implementation [DATAREDIS-632]

[spring-projects-issues]

John Blum opened DATAREDIS-632 and commented

This task is for capturing the results of running the Spring Data Redis test suite against either Pivotal GemFire or Apache Geode.

Currently, I have created the issue/DATAREDIS-632 branch to capture the changes required to the SD Redis codebase to enable SD Redis tests to be ran against Apache Geode.

Initially, I attempted to run the Spring Data Redis Kay tests (with Reactive support) against Spring Data GemFire Kay build snapshots. SDG Kay pulls in Pivotal GemFire 9.0.3, which is based on Apache Geode 1.0.0-incubating, which includes the RedisServer adapter/service (protocol implementation). However, the test suite hung rather early on.

Thinking the Reactive Redis drivers (e.g. Lettuce) might possibly be posing challenges for GemFire's Redis server (protocol) implementation, I switched to SD Redis Ingalls along with using SD Geode. Note, SD GemFire Ingalls does not include Redis support since GemFire 8.2.4 (upon which SD GemFire Ingalls is based) does not include Redis support. GemFire 8.2.x is not based on Apache Geode.

To run the SD Redis test suite against Geode, I...

1. Created a run configuration in my IDE for the GeodeRedisServer class, then...

2. ~/spring-data-redis$ mvn clean install

---

Affects: 2.0 M2 (Kay), 1.8.3 (Ingalls SR3)

[spring-projects-issues]

John Blum commented

Some notable test failures...

1. DefaultZSetOperationsTests failures...

Running org.springframework.data.redis.core.DefaultZSetOperationsTests
Tests run: 150, Failures: 0, Errors: 150, Skipped: 0, Time elapsed: 0.757 sec <<< FAILURE! - in org.springframework.data.redis.core.DefaultZSetOperationsTests
zCardRetrievesDataCorrectly[0](org.springframework.data.redis.core.DefaultZSetOperationsTests)  Time elapsed: 0.004 sec  <<< ERROR!
java.lang.ExceptionInInitializerError
	at org.springframework.data.redis.core.DefaultZSetOperationsTests.<init>(DefaultZSetOperationsTests.java:64)
Caused by: redis.clients.jedis.exceptions.JedisDataException: ERR Unable to process uknown command
	at org.springframework.data.redis.core.DefaultZSetOperationsTests.<init>(DefaultZSetOperationsTests.java:64)

And many more failures like...

testZScanShouldReadEntireValueRange[0](org.springframework.data.redis.core.DefaultZSetOperationsTests)  Time elapsed: 0.001 sec  <<< ERROR!
java.lang.NoClassDefFoundError: Could not initialize class org.springframework.data.redis.test.util.MinimumRedisVersionRule
	at org.springframework.data.redis.core.DefaultZSetOperationsTests.<init>(DefaultZSetOperationsTests.java:64)

Seems there is even a problem when fetching the version of the Redis protocol implemented by the server. Seems that Jedis.info() sends a Command.INFO message/request to the server and Geode's Redis server (protocol) implementation cannot handle/process the command properly.

2. DefaultValueOperationsTests failures...

testSetIfAbsent[8](org.springframework.data.redis.core.DefaultValueOperationsTests)  Time elapsed: 0.001 sec  <<< ERROR!
java.lang.NoSuchMethodError: com.fasterxml.jackson.annotation.JsonInclude$Value.merge(Lcom/fasterxml/jackson/annotation/JsonInclude$Value;Lcom/fasterxml/jackson/annotation/JsonInclude$Value;)Lcom/fasterxml/jackson/annotation/JsonInclude$Value;
	at org.springframework.data.redis.core.DefaultValueOperationsTests.testSetIfAbsent(DefaultValueOperationsTests.java:246)

This error above may have been due to a Jackson version classpath issue as it appears Apache Geode is pulling in an older version of Jackson, 2.7.6. Need to retry by adding an Jackson dependency exclusion to the spring-data-geode dependency.

3. DefaultListOperationsTests failures, for example...

testRightPushAll[6](org.springframework.data.redis.core.DefaultListOperationsTests)  Time elapsed: 4.013 sec  <<< ERROR!
org.springframework.data.redis.RedisConnectionFailureException: java.net.SocketTimeoutException: Read timed out; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
	at org.springframework.data.redis.core.DefaultListOperationsTests.testRightPushAll(DefaultListOperationsTests.java:184)
Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
	at org.springframework.data.redis.core.DefaultListOperationsTests.testRightPushAll(DefaultListOperationsTests.java:184)
Caused by: java.net.SocketTimeoutException: Read timed out
	at org.springframework.data.redis.core.DefaultListOperationsTests.testRightPushAll(DefaultListOperationsTests.java:184)

4. Other interesting DefaultListOperationsTests failures...

testRightPushWithPivot[5](org.springframework.data.redis.core.DefaultListOperationsTests)  Time elapsed: 0.029 sec  <<< ERROR!
org.springframework.dao.InvalidDataAccessApiUsageException: ERR This key could not be created. Gemfire does not allow certain characters to used in keys; nested exception is redis.clients.jedis.exceptions.JedisDataException: ERR This key could not be created. Gemfire does not allow certain characters to used in keys
	at org.springframework.data.redis.core.DefaultListOperationsTests.testRightPushWithPivot(DefaultListOperationsTests.java:160)
Caused by: redis.clients.jedis.exceptions.JedisDataException: ERR This key could not be created. Gemfire does not allow certain characters to used in keys
	at org.springframework.data.redis.core.DefaultListOperationsTests.testRightPushWithPivot(DefaultListOperationsTests.java:160)

And...

testRightPushWithPivot[3](org.springframework.data.redis.core.DefaultListOperationsTests)  Time elapsed: 0.012 sec  <<< ERROR!
org.springframework.dao.InvalidDataAccessApiUsageException: ERR Unfortunately GemFireRedis server does not support LINSERT; nested exception is redis.clients.jedis.exceptions.JedisDataException: ERR Unfortunately GemFireRedis server does not support LINSERT
	at org.springframework.data.redis.core.DefaultListOperationsTests.testRightPushWithPivot(DefaultListOperationsTests.java:162)
Caused by: redis.clients.jedis.exceptions.JedisDataException: ERR Unfortunately GemFireRedis server does not support LINSERT
	at org.springframework.data.redis.core.DefaultListOperationsTests.testRightPushWithPivot(DefaultListOperationsTests.java:162)

5. TransactionalSrpIntegrationTests failures...

Running org.springframework.data.redis.connection.srp.TransactionalSrpItegrationTests
2017-04-19 14:49:30,142 ERROR ction.TransactionalTestExecutionListener: 290 - Exception encountered while executing @AfterTransaction method [public void org.springframework.data.redis.connection.AbstractTransactionalTestBase.verifyTransactionResult()] for test context [DefaultTestContext@7b7fdc8 testClass = TransactionalSrpItegrationTests, testInstance = org.springframework.data.redis.connection.srp.TransactionalSrpItegrationTests@51c693d, testMethod = listOperationLPushShoudBeRolledBackCorrectly@AbstractTransactionalTestBase, testException = [null], mergedContextConfiguration = [MergedContextConfiguration@25df00a0 testClass = TransactionalSrpItegrationTests, locations = '{}', classes = '{class org.springframework.data.redis.connection.srp.TransactionalSrpItegrationTests$SrpContextConfiguration}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextCustomizers = set[[empty]], contextLoader = 'org.springframework.test.context.support.DelegatingSmartContextLoader', parent = [null]]]
java.lang.AssertionError: Values for spring should NOT have been found.
Expected: is <false>
     but: was <true>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.junit.Assert.assertThat(Assert.java:956)
	at org.springframework.data.redis.connection.AbstractTransactionalTestBase.verifyTransactionResult(AbstractTransactionalTestBase.java:108)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.test.context.transaction.TransactionalTestExecutionListener.runAfterTransactionMethods(TransactionalTestExecutionListener.java:283)
	at org.springframework.test.context.transaction.TransactionalTestExecutionListener.afterTestMethod(TransactionalTestExecutionListener.java:231)
	at org.springframework.test.context.TestContextManager.afterTestMethod(TestContextManager.java:319)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:94)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)

Same for the Lettuce driver...

Running org.springframework.data.redis.connection.lettuce.TransactionalLettuceItegrationTests
2017-04-19 14:49:19,062 ERROR ction.TransactionalTestExecutionListener: 290 - Exception encountered while executing @AfterTransaction method [public void org.springframework.data.redis.connection.AbstractTransactionalTestBase.verifyTransactionResult()] for test context [DefaultTestContext@7b139eab testClass = TransactionalLettuceItegrationTests, testInstance = org.springframework.data.redis.connection.lettuce.TransactionalLettuceItegrationTests@4e76dac, testMethod = valueOperationSetShouldBeRolledBackCorrectly@AbstractTransactionalTestBase, testException = [null], mergedContextConfiguration = [MergedContextConfiguration@30bce90b testClass = TransactionalLettuceItegrationTests, locations = '{}', classes = '{class org.springframework.data.redis.connection.lettuce.TransactionalLettuceItegrationTests$LettuceContextConfiguration}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{}', contextCustomizers = set[[empty]], contextLoader = 'org.springframework.test.context.support.DelegatingSmartContextLoader', parent = [null]]]
java.lang.AssertionError: Values for spring should NOT have been found.
Expected: is <false>
     but: was <true>

6. Other similar failures in DefaultLettucePoolTests to above failures...

testCreateWithDbIndex(org.springframework.data.redis.connection.lettuce.DefaultLettucePoolTests)  Time elapsed: 0.01 sec  <<< ERROR!
org.springframework.data.redis.connection.PoolException: Could not get a resource from the pool; nested exception is java.util.NoSuchElementException: Unable to activate object
	at org.springframework.data.redis.connection.lettuce.DefaultLettucePoolTests.testCreateWithDbIndex(DefaultLettucePoolTests.java:161)
Caused by: java.util.NoSuchElementException: Unable to activate object
	at org.springframework.data.redis.connection.lettuce.DefaultLettucePoolTests.testCreateWithDbIndex(DefaultLettucePoolTests.java:161)
Caused by: com.lambdaworks.redis.RedisCommandExecutionException: ERR Unable to process uknown command
	at org.springframework.data.redis.connection.lettuce.DefaultLettucePoolTests.testCreateWithDbIndex(DefaultLettucePoolTests.java:161)
Caused by: com.lambdaworks.redis.RedisCommandExecutionException: ERR Unable to process uknown command

[spring-projects-issues]

Mark Paluch commented

No plans to implement test suites against Geode/GemFire