ISPN-8726 Memcached connector rewrite

* use our protocol parser generator * implement both the text and binary protocol * implement SASL authentication * implement text authentication * implement automatic protocol detection * enable memcached by default ISPN-13548 Allow cache expiration for Memcached
infinispan · Mar 7, 2023 · acf1d3a · acf1d3a
1 parent c3a2257
commit acf1d3a
Show file tree

Hide file tree

Showing 175 changed files with 4,866 additions and 8,221 deletions.
diff --git a/core/src/main/java/org/infinispan/configuration/cache/ContentTypeConfigurationBuilder.java b/core/src/main/java/org/infinispan/configuration/cache/ContentTypeConfigurationBuilder.java
@@ -39,6 +39,11 @@ public ContentTypeConfigurationBuilder mediaType(String mediaType) {
       return this;
    }
 
+   public ContentTypeConfigurationBuilder mediaType(MediaType mediaType) {
+      attributes.attribute(MEDIA_TYPE).set(mediaType);
+      return this;
+   }
+
    public MediaType mediaType() {
       return attributes.attribute(MEDIA_TYPE).get();
    }

diff --git a/core/src/main/java/org/infinispan/security/Security.java b/core/src/main/java/org/infinispan/security/Security.java
@@ -203,6 +203,13 @@ public static Principal getSubjectUserPrincipal(Subject s) {
       return null;
    }
 
+   public static String getSubjectUserPrincipalName(Subject s) {
+      if (s != null && !s.getPrincipals().isEmpty()) {
+         return s.getPrincipals().iterator().next().getName();
+      }
+      return null;
+   }
+
    /**
     * A simplified version of Subject.toString() with the following advantages:
     * <ul>

diff --git a/documentation/src/main/asciidoc/stories/assembly_memcached.adoc b/documentation/src/main/asciidoc/stories/assembly_memcached.adoc
@@ -2,10 +2,12 @@
 :context: memcached
 = Memcached Server
 
-The {brandname} server distribution contains a server module that implements the link:http://github.com/memcached/memcached/blob/master/doc/protocol.txt[Memcached text protocol]. This allows Memcached clients to talk to one or several {brandname} backed Memcached servers. These servers can be standalone just like Memcached, where each server acts independently and does not communicate with the rest. They can also be clustered, where the servers replicate or distribute their contents to other {brandname} backed Memcached servers, providing clients with failover capabilities.
+The {brandname} server distribution contains a server module that implements the Memcached link:http://github.com/memcached/memcached/blob/master/doc/protocol.txt[text] and link:https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped[binary] protocols. This allows Memcached clients to talk to one or several {brandname} backed Memcached servers. These servers can be standalone just like Memcached, where each server acts independently and does not communicate with the rest. They can also be clustered, where the servers replicate or distribute their contents to other {brandname} backed Memcached servers, providing clients with failover capabilities.
 
 
 include::{topics}/con_memcached_client_encoding.adoc[leveloffset=+1]
+include::{topics}/con_memcached_protocol_detection.adoc[leveloffset=+1]
+include::{topics}/con_memcached_security.adoc[leveloffset=+1]
 include::{topics}/ref_memcached_flush_all.adoc[leveloffset=+2]
 include::{topics}/ref_memcached_unsupported.adoc[leveloffset=+2]
 

diff --git a/documentation/src/main/asciidoc/titles/memcached/memcached.asciidoc b/documentation/src/main/asciidoc/titles/memcached/memcached.asciidoc
@@ -16,9 +16,9 @@ include::../{topics}/attributes/community-attributes.adoc[]
 :context: memcached
 
 //Title
-= Using the Memcached protocol endpoint with Infinispan
+= Using the Memcached protocol endpoint with {brandname}
 
-Infinispan Server provides a Memcached endpoint that you can enable alongside the Hot Rod and REST endpoints.
+{brandname} Server provides a Memcached endpoint that you can enable alongside the Hot Rod and REST endpoints.
 This guide provides details about interacting with remote caches with Memcached clients.
 
 //User stories

diff --git a/documentation/src/main/asciidoc/topics/con_memcached_client_encoding.adoc b/documentation/src/main/asciidoc/topics/con_memcached_client_encoding.adoc
@@ -1,9 +1,10 @@
 [[memcached_client_encoding]]
 = Client Encoding
 
-The Memcached text protocol assumes data values read and written by clients are raw bytes. The support for type negotiation will come
-with link:https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#data-types[the Memcached binary protocol] implementation.
+The Memcached text protocol assumes data values read and written by clients are raw bytes. Type negotiation was
+implemented in link:https://github.com/memcached/memcached/wiki/BinaryProtocolRevamped#data-types[the Memcached binary protocol],
+but no alternative data types have been introduced since.
 
 It's not possible for a Memcached client to negotiate the data type to obtain data from the server or send data in different formats. The server can optionally be configured to handle values encoded with a certain Media Type. By setting the `client-encoding` attribute in the `memcached-connector` element, the server will return content in this configured format, and clients can also send data in this format.
 
-The `client-encoding` is useful when a single cache is accessed from multiple remote endpoints (Rest, HotRod, Memcached) and it allows users to tailor the responses/requests to Memcached text clients. For more infomarmation on interoperability between endpoints, consult the endpoint interoperability documentation.
+The `client-encoding` is useful when a single cache is accessed from multiple remote endpoints (Rest, HotRod, Memcached) and it allows users to tailor the responses/requests to Memcached text clients. For more information on interoperability between endpoints, consult the endpoint interoperability documentation.
diff --git a/documentation/src/main/asciidoc/topics/con_memcached_protocol_detection.adoc b/documentation/src/main/asciidoc/topics/con_memcached_protocol_detection.adoc
@@ -0,0 +1,4 @@
+[[memcached_client_protocol_detection]]
+= Protocol detection
+
+The {brandname} Memcached connector supports both manual and automatic protocol configuration by setting the `protocol` attribute in the `memcached-connector` element to one of `TEXT`, `BINARY` and `AUTO`. Automatic detection depends on whether the Memcached connector is on a dedicated socket binding or is part of a single port endpoint. Detection of text protocol clients in a single port endpoint requires authentication, in order to disambiguate the request from other protocols.
diff --git a/documentation/src/main/asciidoc/topics/con_memcached_security.adoc b/documentation/src/main/asciidoc/topics/con_memcached_security.adoc
@@ -0,0 +1,14 @@
+[[memcached_client_security]]
+= Security
+
+The {brandname} Memcached connector supports authentication for both the text and binary protocols. Because the two
+variants implement authentication in a different way, the supported mechanisms depend on the protocol in use.
+
+== Text protocol
+
+Authentication in the Memcached text protocol is performed by an initial `set` command where the key is ignored and the value is a concatenation of the username and the password separated by a space. This authentication scheme requires a security realm which supports plain-text.
+
+== Binary protocol
+
+Authentication in the Memcached binary protocol is performed via dedicated operations which wrap the SASL challenge/response scheme. This authentication scheme works with all the {brandname} security realm implementations.
+
diff --git a/documentation/src/main/asciidoc/topics/con_server_endpoints.adoc b/documentation/src/main/asciidoc/topics/con_server_endpoints.adoc
@@ -8,7 +8,7 @@ access and improved performance in comparison to text-based protocols.
 
 {brandname} provides Hot Rod client libraries in Java, C++, C#, Node.js and other programming languages.
 
-.Topology state transfer
+.Topology caches
 
 {brandname} uses topology caches to provide clients with cluster views.
 Topology caches contain entries that map internal JGroups transport addresses
@@ -34,25 +34,17 @@ balancing and failover capabilities. However, HTTP load balancers maintain
 static cluster views and require manual updates when cluster topology changes
 occur.
 
-ifdef::community[]
 == RESP
 
 {brandname} provides an implementation of the link:https://github.com/redis/redis-specifications/blob/master/protocol/RESP3.md[RESP3] protocol.
 
 The RESP connector supports a subset of the Redis commands.
-endif::community[]
 
 == Memcached
 
-{brandname} provides an implementation of the Memcached text protocol for
+{brandname} provides an implementation of the Memcached text and binary protocols for
 remote client access.
 
-[IMPORTANT]
-====
-The Memcached endpoint is deprecated and planned for removal in a future
-release.
-====
-
 The {brandname} Memcached endpoint supports clustering with replicated and
 distributed cache modes.
 

diff --git a/documentation/src/main/asciidoc/topics/proc_configuring_server_endpoints.adoc b/documentation/src/main/asciidoc/topics/proc_configuring_server_endpoints.adoc
@@ -1,7 +1,7 @@
 [id='configuring-server-endpoints_{context}']
 = Configuring {brandname} Server endpoints
 
-Control how Hot Rod and REST endpoints bind to sockets and use security realm configuration.
+Control how the different protocol endpoints bind to sockets and use security realm configuration.
 You can also configure multiple endpoints and disable administrative capabilities.
 
 [NOTE]

diff --git a/documentation/src/main/asciidoc/topics/proc_enabling_access_logs.adoc b/documentation/src/main/asciidoc/topics/proc_enabling_access_logs.adoc
@@ -1,12 +1,12 @@
 [id='enabling-access-logs_{context}']
 = Enabling access logs
 
-To record Hot Rod and REST endpoint access messages, you need to enable the logging categories in `log4j2.xml`.
+To record Hot Rod, REST and Memcached endpoint access messages, you need to enable the logging categories in `log4j2.xml`.
 
 .Procedure
 
 . Open `{server_home}/server/conf/log4j2.xml` with any text editor.
-. Change the level for the `org.infinispan.HOTROD_ACCESS_LOG` and `org.infinispan.REST_ACCESS_LOG` logging categories to `TRACE`.
+. Change the level for the `org.infinispan.HOTROD_ACCESS_LOG`, `org.infinispan.REST_ACCESS_LOG` and `org.infinispan.MEMCACHED_ACCESS_LOG` logging categories to `TRACE`.
 . Save and close `log4j2.xml`.
 
 [source,xml,options="nowrap",subs=attributes+]

diff --git a/documentation/src/main/asciidoc/topics/ref_access_log_properties.adoc b/documentation/src/main/asciidoc/topics/ref_access_log_properties.adoc
@@ -27,10 +27,10 @@ Logging properties use the `%X{name}` notation and let you modify the format of
 | Principal name, if using authentication.
 
 | `method`
-| Method used. `PUT`, `GET`, and so on.
+| The protocol-specific method used. `PUT`, `GET`, and so on.
 
 | `protocol`
-| Protocol used. `HTTP/1.1`, `HTTP/2`, `HOTROD/2.9`, and so on.
+| Protocol used. `HTTP/1.1`, `HTTP/2`, `HOTROD/2.9`, `MCTXT`, `MCBIN` and so on.
 
 | `status`
 | An HTTP status code for the REST endpoint. `OK` or an exception for the Hot Rod endpoint.

diff --git a/...mentation/src/main/asciidoc/topics/ref_endpoints_authentication_mechanisms.adoc b/...mentation/src/main/asciidoc/topics/ref_endpoints_authentication_mechanisms.adoc
@@ -18,19 +18,19 @@ Currently, you cannot use the Lightweight Directory Access Protocol (LDAP) proto
 |Security realm |SASL authentication mechanism
 
 |Property realms and LDAP realms
-|SCRAM-+*+, DIGEST-+*+, `+SCRAM-*+`
+|`SCRAM`, `DIGEST`
 
 |Token realms
-|OAUTHBEARER
+|`OAUTHBEARER`
 
 |Trust realms
-|EXTERNAL
+|`EXTERNAL`
 
 |Kerberos identities
-|GSSAPI, GS2-KRB5
+|`GSSAPI`, `GS2-KRB5`
 
 |SSL/TLS identities
-|PLAIN
+|`PLAIN`
 |===
 
 .REST endpoints
@@ -42,17 +42,88 @@ Currently, you cannot use the Lightweight Directory Access Protocol (LDAP) proto
 |Security realm |HTTP authentication mechanism
 
 |Property realms and LDAP realms
-|DIGEST
+|`DIGEST`
 
 |Token realms
-|BEARER_TOKEN
+|`BEARER_TOKEN`
 
 |Trust realms
-|CLIENT_CERT
+|`CLIENT_CERT`
 
 |Kerberos identities
-|SPNEGO
+|`SPNEGO`
 
 |SSL/TLS identities
-|BASIC
+|`BASIC`
 |===
+
+.Memcached endpoints
+
+{brandname} Server enables the following SASL authentication mechanisms for Memcached binary protocol endpoints when your configuration includes the corresponding security realm:
+
+[%autowidth,cols="1,1",stripes=even]
+|===
+|Security realm |SASL authentication mechanism
+
+|Property realms and LDAP realms
+|`SCRAM`, `DIGEST`
+
+|Token realms
+|`OAUTHBEARER`
+
+|Trust realms
+|`EXTERNAL`
+
+|Kerberos identities
+|`GSSAPI, GS2-KRB5`
+
+|SSL/TLS identities
+|`PLAIN`
+|===
+
+{brandname} Server enables authentication on Memcached text protocol endpoints only with security realms which support password-based authentication:
+
+[%autowidth,cols="1,1",stripes=even]
+|===
+|Security realm |Memcached text authentication
+
+|Property realms and LDAP realms
+|Yes
+
+|Token realms
+|No
+
+|Trust realms
+|No
+
+|Kerberos identities
+|No
+
+|SSL/TLS identities
+|No
+|===
+
+.RESP endpoints
+
+{brandname} Server enables authentication on RESP endpoints only with security realms which support password-based authentication:
+
+[%autowidth,cols="1,1",stripes=even]
+|===
+|Security realm |RESP authentication
+
+|Property realms and LDAP realms
+|Yes
+
+|Token realms
+|No
+
+|Trust realms
+|No
+
+|Kerberos identities
+|No
+
+|SSL/TLS identities
+|No
+|===
+
diff --git a/documentation/src/main/asciidoc/topics/ref_endpoints_comparison.adoc b/documentation/src/main/asciidoc/topics/ref_endpoints_comparison.adoc
@@ -1,40 +1,18 @@
 [id='endpoint-protocol-comparison_{context}']
 = Comparison of endpoint protocols
 
-//Community includes Memcached and RESP
-ifdef::community[]
-[cols="20,^20,^20,^20",options="header"]
+[cols="20,^20,^20,^20,^20",options="header"]
 |============================================================
 |                    | Hot Rod    | HTTP / REST     | Memcached   | RESP
 | Topology-aware     | [green]*Y* | [red]*N*        | [red]*N*    | [red]*N*
 | Hash-aware         | [green]*Y* | [red]*N*        | [red]*N*    | [red]*N*
-| Encryption         | [green]*Y* | [green]*Y*      | [red]*N*    | [green]*Y*
-| Authentication     | [green]*Y* | [green]*Y*      | [red]*N*    | [green]*Y*
+| Encryption         | [green]*Y* | [green]*Y*      | [green]*Y*    | [green]*Y*
+| Authentication     | [green]*Y* | [green]*Y*      | [green]*Y*    | [green]*Y*
 | Conditional ops    | [green]*Y* | [green]*Y*      | [green]*Y*  | [red]*N*
-| Bulk ops           | [green]*Y* | [red]*N*        | [red]*N*    | [green]*Y*
+| Bulk ops           | [green]*Y* | [red]*N*        | [green]*Y*    | [green]*Y*
 | Transactions       | [green]*Y* | [red]*N*        | [red]*N*    | [red]*N*
 | Listeners          | [green]*Y* | [red]*N*        | [red]*N*    | [green]*Y*
 | Query              | [green]*Y* | [green]*Y*      | [red]*N*    | [red]*N*
 | Execution          | [green]*Y* | [red]*N*        | [red]*N*    | [red]*N*
 | Cross-site failover| [green]*Y* | [red]*N*        | [red]*N*    | [red]*N*
 |============================================================
-endif::community[]
-
-//RHDG does not support Memcached
-ifdef::downstream[]
-[cols="20,^20,^20",options="header"]
-|==============================================
-|                    | Hot Rod    | HTTP / REST
-| Topology-aware     | [green]*Y* | [red]*N*
-| Hash-aware         | [green]*Y* | [red]*N*
-| Encryption         | [green]*Y* | [green]*Y*
-| Authentication     | [green]*Y* | [green]*Y*
-| Conditional ops    | [green]*Y* | [green]*Y*
-| Bulk ops           | [green]*Y* | [red]*N*
-| Transactions       | [green]*Y* | [red]*N*
-| Listeners          | [green]*Y* | [red]*N*
-| Query              | [green]*Y* | [green]*Y*
-| Execution          | [green]*Y* | [red]*N*
-| Cross-site failover| [green]*Y* | [red]*N*
-|==============================================
-endif::downstream[]
diff --git a/documentation/src/main/asciidoc/topics/ref_endpoints_connectors.adoc b/documentation/src/main/asciidoc/topics/ref_endpoints_connectors.adoc
@@ -22,15 +22,13 @@ include::xml/server_endpoints_default.xml[]
 |Includes the Hot Rod endpoint in the `endpoint` configuration.
 
 |`rest-connector`
-|Includes the Hot Rod endpoint in the `endpoint` configuration.
+|Includes the REST endpoint in the `endpoint` configuration.
 
-ifdef::community[]
 |`resp-connector`
 |Includes the RESP endpoint in the `endpoint` configuration.
-endif::community[]
 
 |`memcached-connector`
-|Configures the Memcached endpoint and is disabled by default.
+|Includes the Memcached endpoint in the `endpoint` configuration.
 |===
 
 [role="_additional-resources"]

diff --git a/documentation/src/main/asciidoc/topics/ref_endpoints_sasl_mechanisms.adoc b/documentation/src/main/asciidoc/topics/ref_endpoints_sasl_mechanisms.adoc
@@ -1,7 +1,7 @@
 [id='sasl-authentication-mechanisms_{context}']
 = SASL authentication mechanisms
 
-{brandname} Server supports the following SASL authentications mechanisms with Hot Rod endpoints:
+{brandname} Server supports the following SASL authentications mechanisms with Hot Rod and Memcached binary protocol endpoints:
 
 [%autowidth,cols="1,1,1,1",stripes=even]
 |===

diff --git a/documentation/src/main/asciidoc/topics/ref_endpoints_sasl_policies.adoc b/documentation/src/main/asciidoc/topics/ref_endpoints_sasl_policies.adoc
@@ -1,7 +1,7 @@
 [id='sasl-policies-hot-rod-authentication_{context}']
 = SASL policies
 
-SASL policies provide fine-grain control over Hot Rod authentication mechanisms.
+SASL policies provide fine-grain control over Hot Rod and Memcached authentication mechanisms.
 
 [TIP]
 ====

diff --git a/documentation/src/main/asciidoc/topics/ref_endpoints_sasl_qop.adoc b/documentation/src/main/asciidoc/topics/ref_endpoints_sasl_qop.adoc
@@ -1,7 +1,7 @@
 [id='sasl-quality-of-protection_{context}']
 = SASL quality of protection (QoP)
 
-If SASL mechanisms support integrity and privacy protection (QoP) settings, you can add them to your Hot Rod endpoint configuration with the `qop` attribute.
+If SASL mechanisms support integrity and privacy protection (QoP) settings, you can add them to your Hot Rod and Memcached endpoint configuration with the `qop` attribute.
 
 [%autowidth,cols="1,1",stripes=even]
 |===

diff --git a/documentation/src/main/asciidoc/topics/ref_memcached_unsupported.adoc b/documentation/src/main/asciidoc/topics/ref_memcached_unsupported.adoc
@@ -8,7 +8,6 @@ There are differences in the nature of the original memcached implementation, wh
 
 .Unsupported statistics
 
-*  pid
 *  pointer_size
 *  rusage_user
 *  rusage_system
@@ -36,4 +35,9 @@ Since the arguments that can be sent to the Memcached server are not documented,
 Memcached no longer honors the optional hold time parameter for the delete command. The {brandname} based Memcached server does not implement the feature.
 
 == Verbosity Command
-The verbosity command is not supported since {brandname} logging cannot be simplified to defining the logging level alone.
+
+The verbosity command is not supported since {brandname} logging cannot be simplified to defining the logging level alone.
+
+== Meta protocol
+
+The Memcached meta protocol was introduced in 1.6. {brandname}'s connector doesn't support it yet.