Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CASSANDRA-14572 Expose all table metrics in virtual tables [without annotation processor] #3152

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from
Open

CASSANDRA-14572 Expose all table metrics in virtual tables [without annotation processor] #3152

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 6 additions & 4 deletions doc/modules/cassandra/pages/managing/operating/metrics.adoc
View file
Open in desktop
Expand Up @@ -792,22 +792,24 @@ Reported name format:
*Metric Name*::
`org.apache.cassandra.metrics.Storage.<MetricName>`
*JMX MBean*::
`org.apache.cassandra.metrics:type=Storage name=<MetricName>`
`org.apache.cassandra.metrics:type=Storage,name=<MetricName>`

[cols=",,",options="header",]
|===
|Name |Type |Description
|Exceptions |Counter |Number of internal exceptions caught. Under normal
exceptions this should be zero.

|Load |Counter |Size, in bytes, of the on disk data size this node
manages.

|TotalHints |Counter |Number of hint messages written to this node since
[re]start. Includes one entry for each host to be hinted per hint.

|TotalHintsInProgress |Counter |Number of hints attemping to be sent
currently.
|BootstrapFilesReceived | Gauage<Long> | Number of files received during bootstrap.
|BootstrapFilesTotal | Gauge<Long> | Total number of files to be received during bootstrap.
|BootstrapFilesThroughput | Meter | Throughput of files received during bootstrap.
|BootstrapLastSeenError | Gauge<String> | Last error message seen during bootstrap.
|BootstrapLastSeenStatus | Gauge<String> | Last status message seen during bootstrap.
|===

[[handoff-metrics]]
Expand Down
60 changes: 58 additions & 2 deletions doc/modules/cassandra/pages/managing/operating/virtualtables.adoc
View file
Open in desktop
Expand Up @@ -24,10 +24,11 @@ Since a virtual table is not replicated the partitioner sorts in order of partit

== Virtual Keyspaces

Apache Cassandra 4.0 has added two new keyspaces for virtual tables:
Apache Cassandra has the following virtual keyspaces:

* `system_virtual_schema`
* `system_views`.
* `system_views`
* `system_metrics`

The `system_virtual_schema` keyspace has three tables: `keyspaces`,
`columns` and `tables` for the virtual keyspace, table, and column definitions, respectively.
Expand All @@ -36,6 +37,10 @@ It is used by Cassandra internally and a user should not access it directly.

The `system_views` keyspace contains the actual virtual tables.

The `system_metrics` keyspace contains the virtual tables of all the metrics available in Apache Cassandra
which can be used to monitor the health of the cluster. These metrics are split into several tables each
containing a different set of metrics corresponding to the type of metric or the component of the system.

== Virtual Table Limitations

Before discussing virtual keyspaces and tables, note that virtual keyspaces and tables have some limitations.
Expand All @@ -55,6 +60,57 @@ In fact, mutations for virtual and regular tables cannot occur in the same batch

== Virtual Tables

=== Virtual Tables _system_metrics_ Keyspace

The `system_metrics` virtual keyspace contains the virtual tables of all the metrics available in Apache Cassandra.
Currently, the `system_metrics` keyspace contains the following virtual tables (subject to change), which can be found
in the `system_metrics.all_groups` virtual table:

[source, console]
----
cqlsh> select * from system_metrics.all_groups ;

group_name | virtual_table
-------------------+---------------------------
Batch | batch_group
BufferPool | buffer_pool_group
CIDRAuthorizer | cidr_authorizer_group
CQL | cql_group
Cache | cache_group
ChunkCache | chunk_cache_group
Client | client_group
ClientMessageSize | client_message_size_group
ClientRequest | client_request_group
ClientRequestSize | client_request_size_group
ColumnFamily | column_family_group
CommitLog | commit_log_group
Compaction | compaction_group
Connection | connection_group
DroppedMessage | dropped_message_group
HintsService | hints_service_group
Index | index_group
IndexColumnFamily | index_column_family_group
IndexTable | index_table_group
InternodeInbound | internode_inbound_group
Keyspace | keyspace_group
MemtablePool | memtable_pool_group
Messaging | messaging_group
Paxos | paxos_group
ReadRepair | read_repair_group
Repair | repair_group
Storage | storage_group
StorageProxy | storage_proxy_group
Streaming | streaming_group
TCM | tcm_group
Table | table_group
ThreadPools | thread_pools_group
TrieMemtable | trie_memtable_group
UnweightedCache | unweighted_cache_group

----

=== Virtual Tables _system_views_ Keyspace

Each of the virtual tables in the `system_views` virtual keyspace contain different information.

The following table describes the virtual tables:
Expand Down
5 changes: 3 additions & 2 deletions pylib/cqlshlib/cql3handling.py
View file
Open in desktop
Expand Up @@ -34,8 +34,9 @@ def __str__(self):


SYSTEM_KEYSPACES = ('system', 'system_schema', 'system_traces', 'system_auth', 'system_distributed', 'system_views',
'system_virtual_schema', 'system_cluster_metadata')
NONALTERBALE_KEYSPACES = ('system', 'system_schema', 'system_views', 'system_virtual_schema', 'system_cluster_metadata')
'system_metrics', 'system_virtual_schema', 'system_cluster_metadata')
NONALTERBALE_KEYSPACES = ('system', 'system_schema', 'system_views', 'system_metrics', 'system_virtual_schema',
'system_cluster_metadata')


class Cql3ParsingRuleSet(CqlParsingRuleSet):
Expand Down
8 changes: 4 additions & 4 deletions pylib/cqlshlib/test/test_cqlsh_completion.py
View file
Open in desktop
Expand Up @@ -588,7 +588,7 @@ def test_complete_in_create_type(self):
self.trycompletions('CREATE TYPE foo ', choices=['(', '.'])

def test_complete_in_drop_type(self):
self.trycompletions('DROP TYPE ', choices=['IF', 'system_views.',
self.trycompletions('DROP TYPE ', choices=['IF', 'system_views.', 'system_metrics.',
'tags', 'system_traces.', 'system_distributed.', 'system_cluster_metadata.',
'phone_number', 'quote_udt', 'band_info_type', 'address', 'system.', 'system_schema.',
'system_auth.', 'system_virtual_schema.', self.cqlsh.keyspace + '.'
Expand Down Expand Up @@ -895,7 +895,7 @@ def test_complete_in_truncate(self):

def test_complete_in_use(self):
self.trycompletions('US', immediate='E ')
self.trycompletions('USE ', choices=[self.cqlsh.keyspace, 'system', 'system_auth',
self.trycompletions('USE ', choices=[self.cqlsh.keyspace, 'system', 'system_auth', 'system_metrics',
'system_distributed', 'system_schema', 'system_traces', 'system_views',
'system_virtual_schema', 'system_cluster_metadata' ])

Expand Down Expand Up @@ -991,7 +991,7 @@ def test_complete_in_alter_table(self):
'dynamic_columns',
'twenty_rows_composite_table',
'utf8_with_special_chars',
'system_traces.', 'songs', 'system_views.',
'system_traces.', 'songs', 'system_views.', 'system_metrics.',
'system_virtual_schema.',
'system_schema.', 'system_distributed.',
'system_cluster_metadata.',
Expand Down Expand Up @@ -1020,7 +1020,7 @@ def test_complete_in_alter_table(self):
def test_complete_in_alter_type(self):
self.trycompletions('ALTER TYPE I', immediate='F EXISTS ')
self.trycompletions('ALTER TYPE ', choices=['IF', 'system_views.',
'tags', 'system_traces.', 'system_distributed.',
'tags', 'system_traces.', 'system_distributed.', 'system_metrics.',
'phone_number', 'quote_udt', 'band_info_type', 'address', 'system.', 'system_schema.',
'system_auth.', 'system_virtual_schema.', 'system_cluster_metadata.', self.cqlsh.keyspace + '.'
])
Expand Down
14 changes: 12 additions & 2 deletions src/java/org/apache/cassandra/db/virtual/BatchMetricsTable.java
View file
Open in desktop
Expand Up @@ -26,10 +26,16 @@
import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.metrics.BatchMetrics;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.service.ClientWarn;

/**
* @deprecated The "batch_metrics" virtual table is no longer used, and will be removed in a future release. Please,
* use either "system_metrics.batch_group", or "system_metrics.type_histogram" virtual tables instead.
*/
@Deprecated(since = "5.1")
public class BatchMetricsTable extends AbstractVirtualTable
{

protected static final String TABLE_NAME = "batch_metrics";
private static final String PARTITIONS_PER_LOGGED_BATCH = "partitions_per_logged_batch";
private static final String PARTITIONS_PER_UNLOGGED_BATCH = "partitions_per_unlogged_batch";
private static final String PARTITIONS_PER_COUNTER_BATCH = "partitions_per_counter_batch";
Expand All @@ -40,7 +46,7 @@ public class BatchMetricsTable extends AbstractVirtualTable

BatchMetricsTable(String keyspace)
{
super(TableMetadata.builder(keyspace, "batch_metrics")
super(TableMetadata.builder(keyspace, TABLE_NAME)
.comment("Metrics specific to batch statements")
.kind(TableMetadata.Kind.VIRTUAL)
.partitioner(new LocalPartitioner(UTF8Type.instance))
Expand All @@ -61,6 +67,10 @@ public DataSet data()
addRow(result, PARTITIONS_PER_UNLOGGED_BATCH, metrics.partitionsPerUnloggedBatch.getSnapshot());
addRow(result, PARTITIONS_PER_COUNTER_BATCH, metrics.partitionsPerCounterBatch.getSnapshot());

ClientWarn.instance.warn(String.format("The \"%s\" virtual table is deprecated. " +
Mmuzaf marked this conversation as resolved.
Show resolved Hide resolved
"Please, use either \"%s.batch_group\", or \"%s.type_histogram\" " +
"virtual tables instead.",
TABLE_NAME, metadata.keyspace, metadata.keyspace));
return result;
}

Expand Down
11 changes: 9 additions & 2 deletions src/java/org/apache/cassandra/db/virtual/CQLMetricsTable.java
View file
Open in desktop
Expand Up @@ -20,14 +20,19 @@

import com.google.common.annotations.VisibleForTesting;

import org.apache.cassandra.cql3.QueryProcessor;
import org.apache.cassandra.db.marshal.DoubleType;
import org.apache.cassandra.db.marshal.UTF8Type;
import org.apache.cassandra.dht.LocalPartitioner;
import org.apache.cassandra.metrics.CQLMetrics;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.cql3.QueryProcessor;

import org.apache.cassandra.service.ClientWarn;

/**
* @deprecated The "cql_metrics" virtual table is no longer used, and will be removed in a future release. Please,
* use "system_metrics.cql_group" virtual table instead.
*/
@Deprecated(since = "5.1")
final class CQLMetricsTable extends AbstractVirtualTable
{
public static final String TABLE_NAME = "cql_metrics";
Expand Down Expand Up @@ -70,6 +75,8 @@ public DataSet data()
addRow(result, PREPARED_STATEMENTS_RATIO, cqlMetrics.preparedStatementsRatio.getValue());
addRow(result, REGULAR_STATEMENTS_EXECUTED, cqlMetrics.regularStatementsExecuted.getCount());

ClientWarn.instance.warn("The \"" + TABLE_NAME + "\" virtual table is deprecated. " +
"Please, use \"system_metrics.cql_group\" virtual table instead.");
return result;
}

Expand Down