Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CASSANDRA-19366 trunk - Expose auth mode in system_views.clients, clientstats, metrics #3085

Closed
wants to merge 21 commits into from
Closed
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
cf89185
Expose auth mode in system_views.clients, clientstats, metrics
tolbertam Jan 31, 2024
605d9cb
Process Feedback
tolbertam Feb 6, 2024
f359c39
refactor to see AuthenticationMode to be a class
smiklosovic Feb 6, 2024
0d2b7b3
Feedback: Adjust docs to use capitalized authentication mode names
tolbertam Feb 6, 2024
3ef7191
Feedback: Factor out markAuthMeter, remove public static
tolbertam Feb 6, 2024
b6b7886
Feedback: Add unit tests around ConnectionTracker
tolbertam Feb 7, 2024
3a869c3
Spelling
tolbertam Feb 7, 2024
0d1853b
Update src/java/org/apache/cassandra/auth/AllowAllAuthenticator.java
tolbertam Feb 7, 2024
25bfea3
Update src/java/org/apache/cassandra/auth/AuthenticatedUser.java
tolbertam Feb 7, 2024
248ba2e
Update src/java/org/apache/cassandra/auth/AuthenticatedUser.java
tolbertam Feb 7, 2024
465109e
Update src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
tolbertam Feb 7, 2024
e17c35f
Update src/java/org/apache/cassandra/tools/nodetool/ClientStats.java
tolbertam Feb 7, 2024
8f082a1
Feedback: Mtls -> MutualTls for MTLS display name
tolbertam Feb 7, 2024
cb9439e
AuthenticatedNativeClients -> ConnectedNativeClients
tolbertam Feb 7, 2024
ea6e2fd
Feedback: remove braces for single line condition
tolbertam Feb 7, 2024
e1e3b2d
Feedback: handle --all and --verbose, comments about metadata exposure
tolbertam Feb 8, 2024
c48f792
Feedback: Adjust existing faulty behavior with --all --client-options
tolbertam Feb 8, 2024
f75d7b3
Feedback: Move Auth columns after Client-Options
tolbertam Feb 8, 2024
0ef40b3
Fix pattern in --verbose test
tolbertam Feb 8, 2024
dddae45
Feedback: Tests that rely on cassandra user should block on existence
tolbertam Feb 12, 2024
54da742
Feedback: Limit scope of waitForExistingRoles change
tolbertam Feb 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
81 changes: 77 additions & 4 deletions doc/modules/cassandra/pages/managing/operating/metrics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -913,14 +913,87 @@ Reported name format:
[cols=",,",options="header",]
|===
|Name |Type |Description
|connectedNativeClients |Gauge<Integer> |Number of clients connected to
|AuthFailure |Meter | Rate of failed authentications

|AuthSuccess |Meter | Rate of successful authentications

|ClientsByProtocolVersion |Gauge<List<Map<String, Integer>> | List of
all connections' protocol version and ip address

|ConnectedNativeClients |Gauge<Integer> |Number of clients connected to
this nodes native protocol server

|connections |Gauge<List<Map<String, String>> |List of all connections
|ConnectedNativeClientsByUser |Gauge<Map<String, Integer>> |Number of
connnective native clients by username

|Connections |Gauge<List<Map<String, String>> |List of all connections
and their state information

|connectedNativeClientsByUser |Gauge<Map<String, Int> |Number of
connnective native clients by username
|PausedConnections|Gauge<Integer>|Number of connections currently
paused by rate limiter

|ProtocolException|Meter|Rate of requests resulting in a protocol
exception

|UnknownException|Meter|Rate of requests resulting in an unknown
exception

|RequestDiscarded|Meter|Rate of requests discarded by rate limiter

|RequestDispatched|Meter|Rate of requests dispatched (not discarded)

|RequestsSizeByIpDistribution|Histogram|Histogram of distribution of
requests coming from unique IPs

|===

== Client Encryption Metrics

Metrics specific to Client encryption

*Metric Name*::
`org.apache.cassandra.metrics.Client.<Encrypted|Unencrypted>.<MetricName>`
*JMX MBean*::
`org.apache.cassandra.metrics:type=Client scope=<Encrypted|Unencrypted> name=<MetricName>`

[cols=",,",options="header",]
|===
|Name |Type |Description
|ConnectedNativeClients |Gauge<Integer> |Number of clients connected in
this way to this nodes native protocol server

|===

== Client Authentication Mode-Specific Metrics

Metrics specific to connectivity for a given Authentication 'mode'.

*Metric Name*::
`org.apache.cassandra.metrics.Client.<Mode>.<MetricName>`
*JMX MBean*::
`org.apache.cassandra.metrics:type=Client scope=<Mode> name=<MetricName>`

An authentication mode is a supported method of authentication. The following
authentication modes exist for the given supported `IAuthenticators`:

* `PasswordAuthenticator` - Password
* `MutualTlsAuthenticator` - Mtls
* `MutualTlsWithPasswordFallbackAuthenticator` - Mtls, Password
* `AllowAllAuthenticator` - Unauthenticated

A custom implementation of `IAuthenticator` may expose metrics on their own
custom mode by implementing `IAuthenticator.getSupportedAuthenticationModes()`.

[cols=",,",options="header",]
|===
|Name |Type |Description
|AuthenticatedNativeClients |Gauge<Int> |Number of clients connected and
authenticated to this nodes native protocol server using this mode

|AuthFailure |Meter | Rate of failed authentications using this mode

|AuthSuccess |Meter | Rate of successful authentications using this mode

|===

== Batch Metrics
Expand Down
10 changes: 10 additions & 0 deletions src/java/org/apache/cassandra/auth/AllowAllAuthenticator.java
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,17 @@
import java.util.Map;
import java.util.Set;

import com.google.common.collect.Sets;

import org.apache.cassandra.exceptions.AuthenticationException;
import org.apache.cassandra.exceptions.ConfigurationException;

public class AllowAllAuthenticator implements IAuthenticator
{
private static final SaslNegotiator AUTHENTICATOR_INSTANCE = new Negotiator();

private static final Set<AuthenticationMode> AUTHENTICATION_MODES = Sets.newHashSet(AuthenticationMode.UNAUTHENTICATED);
tolbertam marked this conversation as resolved.
Show resolved Hide resolved

public boolean requireAuthentication()
{
return false;
Expand All @@ -52,6 +56,12 @@ public SaslNegotiator newSaslNegotiator(InetAddress clientAddress)
return AUTHENTICATOR_INSTANCE;
}

@Override
public Set<AuthenticationMode> getSupportedAuthenticationModes()
{
return AUTHENTICATION_MODES;
}

public AuthenticatedUser legacyAuthenticate(Map<String, String> credentialsData)
{
return AuthenticatedUser.ANONYMOUS_USER;
Expand Down
45 changes: 44 additions & 1 deletion src/java/org/apache/cassandra/auth/AuthenticatedUser.java
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,17 @@
package org.apache.cassandra.auth;

import java.net.InetSocketAddress;
import java.util.Collections;
import java.util.Map;
import java.util.Set;
import com.google.common.base.Objects;

import org.apache.cassandra.auth.IAuthenticator.AuthenticationMode;
import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.dht.Datacenters;

import static org.apache.cassandra.auth.IAuthenticator.AuthenticationMode.UNAUTHENTICATED;

/**
* Returned from IAuthenticator#authenticate(), represents an authenticated user everywhere internally.
*
Expand Down Expand Up @@ -55,13 +60,29 @@ public static void init()

private final String name;

// Primary Role of the logged in user
private final AuthenticationMode authenticationMode;

private final Map<String, Object> metadata;

// Primary Role of the logged-in user
private final RoleResource role;

public AuthenticatedUser(String name)
{
this(name, UNAUTHENTICATED);
}

public AuthenticatedUser(String name, AuthenticationMode authenticationMode)
{
this(name, authenticationMode, Collections.emptyMap());
}

public AuthenticatedUser(String name, AuthenticationMode authenticationMode, Map<String, Object> metadata)
{
this.name = name;
this.role = RoleResource.role(name);
this.authenticationMode = authenticationMode;
this.metadata = metadata;
}

public String getName()
Expand All @@ -74,6 +95,22 @@ public RoleResource getPrimaryRole()
return role;
}

/**
* The mode of authentication used to authenticate this user.
tolbertam marked this conversation as resolved.
Show resolved Hide resolved
*/
public AuthenticationMode getAuthenticationMode()
{
return authenticationMode;
smiklosovic marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* {@link IAuthenticator}-contextual metadata about how the user was authenticated.
smiklosovic marked this conversation as resolved.
Show resolved Hide resolved
tolbertam marked this conversation as resolved.
Show resolved Hide resolved
*/
public Map<String, Object> getMetadata()
{
return metadata;
}

/**
* Checks the user's superuser status.
* Only a superuser is allowed to perform CREATE USER and DROP USER queries.
Expand Down Expand Up @@ -179,6 +216,12 @@ public boolean equals(Object o)
@Override
public int hashCode()
{
// Note: for reasons of maintaining the invariant that an object that equals maintains the same hashCode,
// we do not include mode and metadata in the hashCode calculation.
// This is particularly salient as there are cases where AuthenticatedUser is used as a key in
// Role/Permissions cache. In effect, we would like to treat all connections sharing the same name as the same
// user, where mode and metadata are just additional context about how the user authenticated that
// should not factor into 'equivalence' of users.
return Objects.hashCode(name);
}
}
97 changes: 86 additions & 11 deletions src/java/org/apache/cassandra/auth/IAuthenticator.java
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,13 @@

import java.net.InetAddress;
import java.security.cert.Certificate;
import java.util.Collections;
import java.util.Map;
import java.util.Objects;
import java.util.Set;

import javax.annotation.Nonnull;

import org.apache.cassandra.exceptions.AuthenticationException;
import org.apache.cassandra.exceptions.ConfigurationException;

Expand Down Expand Up @@ -53,7 +57,7 @@ default boolean supportsEarlyAuthentication()
return false;
}

/**
/**
* Set of resources that should be made inaccessible to users and only accessible internally.
*
* @return Keyspaces, column families that will be unmodifiable by users; other resources.
Expand All @@ -69,7 +73,7 @@ default boolean supportsEarlyAuthentication()

/**
* Setup is called once upon system startup to initialize the IAuthenticator.
*
* <p>
* For example, use this method to create any required keyspaces/column families.
*/
void setup();
Expand All @@ -78,6 +82,7 @@ default boolean supportsEarlyAuthentication()
* Provide a SASL handler to perform authentication for an single connection. SASL
* is a stateful protocol, so a new instance must be used for each authentication
* attempt.
*
* @param clientAddress the IP address of the client whom we wish to authenticate, or null
* if an internal client (one not connected over the remote transport).
* @return org.apache.cassandra.auth.IAuthenticator.SaslNegotiator implementation
Expand All @@ -90,12 +95,13 @@ default boolean supportsEarlyAuthentication()
* is a stateful protocol, so a new instance must be used for each authentication
* attempt. This method accepts certificates as well. Authentication strategies can
* override this method to gain access to client's certificate chain, if present.
*
* @param clientAddress the IP address of the client whom we wish to authenticate, or null
* if an internal client (one not connected over the remote transport).
* @param certificates the peer's Certificate chain, if present.
* It is expected that these will all be instances of {@link java.security.cert.X509Certificate},
* but we pass them as the base {@link Certificate} in case future implementations leverage
* other certificate types.
* @param certificates the peer's Certificate chain, if present.
* It is expected that these will all be instances of {@link java.security.cert.X509Certificate},
* but we pass them as the base {@link Certificate} in case future implementations leverage
* other certificate types.
* @return org.apache.cassandra.auth.IAuthenticator.SaslNegotiator implementation
* (see {@link org.apache.cassandra.auth.PasswordAuthenticator.PlainTextSaslAuthenticator})
*/
Expand All @@ -104,12 +110,23 @@ default SaslNegotiator newSaslNegotiator(InetAddress clientAddress, Certificate[
return newSaslNegotiator(clientAddress);
}

/**
* @return The supported authentication 'modes' of this authenticator.
* scheme.
smiklosovic marked this conversation as resolved.
Show resolved Hide resolved
* <p>
* This is currently only used for registering metrics tied to authentication by mode.
*/
default Set<AuthenticationMode> getSupportedAuthenticationModes()
{
return Collections.emptySet();
}

/**
* A legacy method that is still used by JMX authentication.
*
* <p>
* You should implement this for having JMX authentication through your
* authenticator.
*
* <p>
* Should never return null - always throw AuthenticationException instead.
* Returning AuthenticatedUser.ANONYMOUS_USER is an option as well if authentication is not required.
*
Expand All @@ -129,7 +146,7 @@ public interface SaslNegotiator
/**
* Evaluates the client response data and generates a byte[] response which may be a further challenge or purely
* informational in the case that the negotiation is completed on this round.
*
* <p>
* This method is called each time a {@link org.apache.cassandra.transport.messages.AuthResponse} is received
* from a client. After it is called, {@link #isComplete()} is checked to determine whether the negotiation has
* finished. If so, an AuthenticatedUser is obtained by calling {@link #getAuthenticatedUser()} and that user
Expand All @@ -141,8 +158,7 @@ public interface SaslNegotiator
*
* @param clientResponse The non-null (but possibly empty) response sent by the client
* @return The possibly null response to send to the client.
* @throws AuthenticationException
* see {@link javax.security.sasl.SaslServer#evaluateResponse(byte[])}
* @throws AuthenticationException see {@link javax.security.sasl.SaslServer#evaluateResponse(byte[])}
*/
public byte[] evaluateResponse(byte[] clientResponse) throws AuthenticationException;

Expand Down Expand Up @@ -179,5 +195,64 @@ default boolean shouldSendAuthenticateMessage()
{
return true;
}

/**
* @return The assumed mode of authentication attempted using this negotiator, this will usually be some value
* of {@link AuthenticationMode#toString()}} unless an implementor provides their own custom authentication
* scheme.
*/
default AuthenticationMode getAuthenticationMode()
{
return AuthenticationMode.UNAUTHENTICATED;
}
}

/**
* Known modes of authentication supported by Cassandra's provided {@link IAuthenticator} implementations.
*/
abstract class AuthenticationMode
{
private final String displayName;

public AuthenticationMode(@Nonnull String displayName)
{
this.displayName = displayName;
}

/**
* User was not authenticated in any particular way.
*/
public static final AuthenticationMode UNAUTHENTICATED = new AuthenticationMode("Unauthenticated") {};

/**
* User authenticated using a password.
*/
public static final AuthenticationMode PASSWORD = new AuthenticationMode("Password") {};

/**
* User authenticated using a trusted identity in their client certificate.
*/
public static final AuthenticationMode MTLS = new AuthenticationMode("Mtls") {};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we spell out the label?

Suggested change
public static final AuthenticationMode MTLS = new AuthenticationMode("Mtls") {};
public static final AuthenticationMode MTLS = new AuthenticationMode("Mutual TLS") {};

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would break JMX as I am not sure how a metric with a space in its name would behave.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably check that the display name does not contain space

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, I'm not sure what's the behavior for jmx. But if spaces break JMX, we should add some guards around it

Copy link
Contributor Author

@tolbertam tolbertam Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll test whether spaces work w/ JMX, I suspect it does, but isn't desirable as might require clients to escape; If it doesn't work, I suggest we throw a RuntimeException in the AuthenticationMode constructor.

As far as the name, regardless of whether spaces are allowed, how about MutualTls (for consistency with how the authenticator is named)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it does work:

beans
...
org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Encrypted,type=Client
org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Mutual Tls,type=Client
org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Password,type=Client
org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Unencrypted,type=Client
org.apache.cassandra.metrics:name=ConnectedNativeClients,type=Client

But it isn't easy to use, and I imagine will give some tooling trouble. For it to work with jmxterm for example I had to double \ escape the space:

$>bean org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Mutual Tls,type=Client
#IllegalArgumentException: Please specify domain using either -d option or domain command
$>bean org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Mutual\\ Tls,type=Client
#bean is set to org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Mutual Tls,type=Client
$>get *
#mbean = org.apache.cassandra.metrics:name=ConnectedNativeClients,scope=Mutual Tls,type=Client:
Value = 5;

I think we should document that spaces in the mode name aren't preferred as it may not work well with some tooling. I will do that!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 8f082a1 and renamed Mtls -> MutualTls. I also realized was still using String in ClientMetrics instead of AuthenticationMode so I updated that as well, hopefully that looks good!


@Override
public String toString()
{
return displayName;
}

@Override
public boolean equals(Object o)
{
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
AuthenticationMode that = (AuthenticationMode) o;
return displayName.equals(that.displayName);
}

@Override
public int hashCode()
{
return Objects.hash(displayName);
}
}
}