Auto Discovery support for Amazon ElastiCache

* Introducing Dynamic and Static client modes for the memcached client.
  Dynamic allows the client to automatically pick up the node list using
  the ElastiCache configuration endpoint throughout the lifetime of the
  client object. Static mode is the existing behavior, where the client
  is initialized only once using the configuration provided by the user.

* Support for a new configuration command used in ElastiCache Nodes.
  This command supports the config get, set and delete commands in ASCII
  and binary mode.

* Periodic poller to check for node list changes in the ElastiCache
  cluster and dynamically update client connections according to the
  configuration.  This operates only in Dynamic mode.

* In dynamic mode, if IP addresses are specified in the configuration,
  then the connection is established using the IP address directly.

For more information, please see
http://docs.amazonwebservices.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.html

LICENSE.txt

@@ -1,3 +1,15 @@
+Copyright (C) 2012-2012 Amazon.com, Inc. or its affiliates. All Rights Reserved. 
+
+Licensed under the Amazon Software License (the "License"). You may not use this 
+file except in compliance with the License. A copy of the License is located at
+  http://aws.amazon.com/asl/
+or in the "license" file accompanying this file. This file is distributed on 
+an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
+implied. See the License for the specific language governing permissions and 
+limitations under the License. 
+
+Derived from spymemcached 2.8.1 under the MIT license. 
+
 Copyright (c) 2006-2009  Dustin Sallings
 Copyright (c) 2009-2011  Couchbase, Inc.
 

README.markdown

@@ -1,27 +1,20 @@
 # Building
 
-Spymemcached can be compiled using Apache Ant by running the following
+AmazonElastiCacheClusterClient can be compiled using Apache Ant by running the following
 command:
 
     ant
 
 This will generate binary, source, and javadoc jars in the build
 directory of the project.
 
-To run the Spymemcached tests against Membase Server run the
-following command:
-
-    ant test -Dserver.type=membase
-
-To test Spymemcached against Membase running on a different host
-use the following command:
-
-    ant test -Dserver.type=membase \
-        -Dserver.address_v4=ip_address_of_membase
+More test info will be updated shortly.
 
 # Testing
 
-The latest version of spymemcached has a set of command line arguments
+_Note: The ant test target is in the process of being updated to run the additional tests written for Auto Discovery._
+
+The latest version of AmazonElastiCacheClusterClient has a set of command line arguments
 that can be used to configure the location of your testing server. The
 arguments are listed below.
 
@@ -30,12 +23,6 @@ arguments are listed below.
 This argument is used to specify the ipv4 address of your testing
 server. By default it is set to localhost.
 
-    -Dserver.address_v6=ipv6_address_of_testing_server
-
-This argument is used to set the ipv6 address of your testing server.
-By default it is set to ::1. If an ipv6 address is specified then an
-ipv4 address must be specified otherwise there may be test failures.
-
     -Dserver.port_number=port_number_of_memcached
 
 This argument is used when memcahched is started on a port other than
@@ -46,11 +33,13 @@ This argument is used when memcahched is started on a port other than
 This argument is used for CI testing where certain unit tests might
 be temporarily failing.
 
-# More Information
+# More Information for Amazon ElastiCache Cluster Client
+Github link: https://github.com/amazonwebservices/aws-elasticache-cluster-client-memcached-for-java
+This repository is a fork of the spymemcached Java client for connecting to memcached (specifically the https://github.com/dustin/java-memcached-client repo).
 
-For more information about Spymemcached see the links below:
+Additional changes have been made to support Amazon ElastiCache Auto Discovery. To read more about Auto Discovery, please go here: http://docs.amazonwebservices.com/AmazonElastiCache/latest/UserGuide/AutoDiscovery.html.
 
-## Project Page The
+For more information about Spymemcached see the links below:
 
 [Spymemcached Project Home](http://code.google.com/p/spymemcached/)
 contains a wiki, issue tracker, and downloads section.

build.xml

@@ -18,9 +18,19 @@
   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
   SOFTWARE.
+
+  Portions Copyright (C) 2012-2012 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+   
+  Licensed under the Amazon Software License (the "License"). You may not use this 
+  file except in compliance with the License. A copy of the License is located at
+   http://aws.amazon.com/asl/
+  or in the "license" file accompanying this file. This file is distributed on 
+  an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
+  implied. See the License for the specific language governing permissions and 
+  limitations under the License.
 -->
 
-<project name="spymemcached" default="package"
+<project name="AmazonElastiCacheClusterClient" default="package"
     xmlns:artifact="urn:maven-artifact-ant"
     xmlns:ivy="antlib:org.apache.ivy.ant">
 
@@ -31,8 +41,8 @@
     </classpath>
   </taskdef>
 
-  <property name="name" value="spymemcached"/>
-  <property name="copyright" value="2006-2011 Dustin Sallings, Matt Ingenthron" />
+  <property name="name" value="AmazonElastiCacheClusterClient"/>
+  <property name="copyright" value="2006-2011 Dustin Sallings, Matt Ingenthron, 2012-2012 Amazon.com, Inc. or its affiliates" />
   <property name="group" value="spy" />
 
   <property name="base.src.dir" value="${basedir}/src" />

ivy/libraries.properties

@@ -17,6 +17,16 @@
 # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 # SOFTWARE.
+
+# Portions Copyright (C) 2012-2012 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+# Licensed under the Amazon Software License (the "License"). You may not use this 
+# file except in compliance with the License. A copy of the License is located at
+#  http://aws.amazon.com/asl/
+# or in the "license" file accompanying this file. This file is distributed on 
+# an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
+# implied. See the License for the specific language governing permissions and 
+# limitations under the License.
 
 checkstyle.version=5.0
 
@@ -26,6 +36,6 @@ ivy.version=2.2.0
 mvn.version=2.0.10
 
 jmock.version=1.2.0
-junit.version=4.7
+junit.version=4.8.2
 log4j.version=1.2.16
 spring-beans.version=3.0.3.RELEASE

ivy/spymemcached.xml

@@ -20,13 +20,23 @@
   LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
   SOFTWARE.
+
+  Portions Copyright (C) 2012-2012 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+   
+  Licensed under the Amazon Software License (the "License"). You may not use this 
+  file except in compliance with the License. A copy of the License is located at
+   http://aws.amazon.com/asl/
+  or in the "license" file accompanying this file. This file is distributed on 
+  an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
+  implied. See the License for the specific language governing permissions and 
+  limitations under the License.
 -->
 <ivy-module version="1.0">
   <info organisation="spy" module="${name}" revision="${version}">
-    <license name="Apache 2.0"/>
-    <ivyauthor name="Couchbase" url="http://github.com/dustin/java-memcached-client" />
+    <license name="Amazon Software License"/>
+    <ivyauthor name="Amazon.com Inc" url="https://github.com/amazonwebservices/aws-elasticache-cluster-client-memcached-for-java" />
     <description>
-        Spymemcached
+        AmazonElastiCacheClusterClient
     </description>
   </info>
   <configurations defaultconfmapping="default">

src/main/java/net/spy/memcached/AddrUtil.java

@@ -19,6 +19,17 @@
  * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
  * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALING
  * IN THE SOFTWARE.
+ * 
+ * 
+ * Portions Copyright (C) 2012-2012 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * 
+ * Licensed under the Amazon Software License (the "License"). You may not use this 
+ * file except in compliance with the License. A copy of the License is located at
+ *  http://aws.amazon.com/asl/
+ * or in the "license" file accompanying this file. This file is distributed on 
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
+ * implied. See the License for the specific language governing permissions and 
+ * limitations under the License.
  */
 
 package net.spy.memcached;
@@ -28,10 +39,14 @@
 import java.util.ArrayList;
 import java.util.List;
 
+import net.spy.memcached.config.ClusterConfiguration;
+import net.spy.memcached.config.NodeEndPoint;
+
 /**
  * Convenience utilities for simplifying common address parsing.
  */
 public final class AddrUtil {
+  public static final char HOST_CONFIG_DELIMITER = '|';
 
   private AddrUtil() {
     // Empty
@@ -103,4 +118,64 @@ public static List<InetSocketAddress> getAddresses(List<String> servers) {
     }
     return addrs;
   }
+
+  /**
+   * Parse response from getConfig for cluster type.
+   * version number
+   * hostname1|ipaddress1|port hostname2|ipaddress2|port
+   *  
+   * returns the ClusterConfiguration object which contains the parsed results. 
+   */
+  public static ClusterConfiguration parseClusterTypeConfiguration(String configurationResponse) {
+    if (configurationResponse == null) {
+      throw new NullPointerException("Null configuration");
+    }
+    if (configurationResponse.trim().equals("")) {
+      throw new IllegalArgumentException("No configuration in the response:" + configurationResponse);
+    }
+    String[] lines = configurationResponse.trim().split("(?:\\r?\\n)");
+    if(lines == null || lines.length != 2) {
+      throw new IllegalArgumentException("Incorrect response format. Response:" + configurationResponse);
+    }
+
+    String versionString = lines[0].trim();
+    if(versionString.equals("")){
+      throw new IllegalArgumentException("Version number is missing. Response:" + configurationResponse);
+    }
+
+    long versionNumber = Long.parseLong(versionString);
+
+    String hostList = lines[1].trim();
+    if (hostList.equals("")) {
+      throw new IllegalArgumentException("Empty host list in the response:" + configurationResponse);
+    }
+
+    List<NodeEndPoint> endPoints = new ArrayList<NodeEndPoint>();
+
+    for (String hostDetails : hostList.split("(?:\\s)+")) {
+      if (hostDetails.equals("")) {
+        continue;
+      }
+
+      int firstDelimiter = hostDetails.indexOf(HOST_CONFIG_DELIMITER);
+      int secondDelimiter = hostDetails.lastIndexOf(HOST_CONFIG_DELIMITER);
+      if (firstDelimiter < 1 || firstDelimiter == secondDelimiter) {
+        throw new IllegalArgumentException("Invalid server ''" + hostDetails
+            + "'' in response:  " + configurationResponse);
+      }
+      String hostName = hostDetails.substring(0, firstDelimiter).trim();
+      String ipAddress = hostDetails.substring(firstDelimiter+1, secondDelimiter).trim();
+      String portNum = hostDetails.substring(secondDelimiter + 1).trim();
+      int port = Integer.parseInt(portNum);
+
+      NodeEndPoint endPoint = new NodeEndPoint(hostName, ipAddress, port);
+      endPoints.add(endPoint);
+    }
+    assert !endPoints.isEmpty() : "No endpoints found";
+
+    ClusterConfiguration config = new ClusterConfiguration(versionNumber, endPoints);
+
+    return config;
+  }
+
 }

src/main/java/net/spy/memcached/BinaryConnectionFactory.java

@@ -19,6 +19,17 @@
  * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
  * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALING
  * IN THE SOFTWARE.
+ * 
+ * 
+ * Portions Copyright (C) 2012-2012 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * 
+ * Licensed under the Amazon Software License (the "License"). You may not use this 
+ * file except in compliance with the License. A copy of the License is located at
+ *  http://aws.amazon.com/asl/
+ * or in the "license" file accompanying this file. This file is distributed on 
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
+ * implied. See the License for the specific language governing permissions and 
+ * limitations under the License.
  */
 
 package net.spy.memcached;
@@ -35,18 +46,39 @@
 public class BinaryConnectionFactory extends DefaultConnectionFactory {
 
   /**
-   * Create a DefaultConnectionFactory with the default parameters.
+   * Create a BinaryConnectionFactory with the default parameters.
    */
   public BinaryConnectionFactory() {
     super();
   }
+
+  /**
+   * Create a BinaryConnectionFactory with the given clientMode.
+   * @param clientMode
+   */
+  public BinaryConnectionFactory(ClientMode clientMode){
+    super(clientMode);
+  }
 
   /**
-   * Create a BinaryConnectionFactory with the given maximum operation queue
-   * length, and the given read buffer size.
+   * Create a BinaryConnectionFactory with the given parameters
+   * 
+   * @param len the queue length.
+   * @param bufSize the buffer size
    */
   public BinaryConnectionFactory(int len, int bufSize) {
-    super(len, bufSize);
+    super(ClientMode.Dynamic, len, bufSize);
+  }
+
+  /**
+   * Create a BinaryConnectionFactory with the given parameters
+   * 
+   * @param clientMode the mode of the client to indicate whether dynamic server list management is done.
+   * @param len the queue length.
+   * @param bufSize the buffer size
+   */
+  public BinaryConnectionFactory(ClientMode clientMode, int len, int bufSize) {
+    super(clientMode, len, bufSize);
   }
 
   /**
@@ -57,7 +89,19 @@ public BinaryConnectionFactory(int len, int bufSize) {
    * @param hash the algorithm to use for hashing
    */
   public BinaryConnectionFactory(int len, int bufSize, HashAlgorithm hash) {
-    super(len, bufSize, hash);
+    super(ClientMode.Dynamic, len, bufSize, hash);
+  }
+
+  /**
+   * Construct a BinaryConnectionFactory with the given parameters.
+   *
+   * @param clientMode the mode of the client to indicate whether dynamic server list management is done.
+   * @param len the queue length.
+   * @param bufSize the buffer size
+   * @param hash the algorithm to use for hashing
+   */
+  public BinaryConnectionFactory(ClientMode clientMode, int len, int bufSize, HashAlgorithm hash) {
+    super(clientMode, len, bufSize, hash);
   }
 
   @Override

src/main/java/net/spy/memcached/ClientMode.java

@@ -0,0 +1,29 @@
+/**
+ * Copyright (C) 2012-2012 Amazon.com, Inc. or its affiliates. All Rights Reserved. 
+ *
+ * Licensed under the Amazon Software License (the "License"). You may not use this 
+ * file except in compliance with the License. A copy of the License is located at
+ *  http://aws.amazon.com/asl/
+ * or in the "license" file accompanying this file. This file is distributed on 
+ * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
+ * implied. See the License for the specific language governing permissions and 
+ * limitations under the License. 
+ */
+package net.spy.memcached;
+
+/**
+ * The modes in which the client can operate. 
+ */
+public enum ClientMode {
+
+  /**
+   * In Static Client mode, the set of endpoints specified during initialization is used throughout the lifetime of the client object.
+   */
+  Static,
+  /**
+   * In Dynamic Client mode, the set of cache node endpoints and any updates to it is dynamically managed in this mode. 
+   * The client is initialized with a configuration endpoint. The client will periodically learn about the cache nodes in the
+   * cluster.
+   */
+  Dynamic
+}