sql: add system.vcpu_hours_audit table#170328
Conversation
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
This change is |
sadaf-crl
force-pushed
the
CC-35137
branch
4 times, most recently
from
f0f3f21 to
b957903
Compare
|
Detected infrastructure failure (matched: self-hosted runner lost communication with the server). Automatically rerunning failed jobs. (run link) |
sadaf-crl
force-pushed
the
CC-35137
branch
6 times, most recently
from
918209d to
c5855d4
Compare
|
Detected infrastructure failure (matched: self-hosted runner lost communication with the server). Automatically rerunning failed jobs. (run link) |
sadaf-crl
force-pushed
the
CC-35137
branch
7 times, most recently
from
26acee7 to
27e71ef
Compare
Add the system.vcpu_hours_audit table to track per-node, per-hour vCPU consumption for license auditing. This table stores: - node_id: ID of the node reporting vCPU usage - license_id: license identifier under which vCPUs are consumed - hour_timestamp: timestamp of the hour bucket for this measurement - num_vcpu: number of vCPUs on the node during this hour The table has a composite primary key on (node_id, license_id, hour_timestamp) and is configured with admin read/write privileges. This is part of the self-hosted license audit feature to enable customers to generate vCPU consumption reports for compliance tracking and renewals. Changes include: - Schema definition and descriptor in systemschema/system.go - Table name constant in catconstants/constants.go - Bootstrap metadata registration in bootstrap/metadata.go - Privilege configuration in catprivilege/system.go - Backup configuration in backup/system_schema.go (opted out) - Cluster version gate V26_3_AddVcpuHoursAuditTable - Upgrade migration for existing clusters - SystemDatabaseSchemaBootstrapVersion updated - NumSystemTablesForSystemTenant incremented to 71 - Generated settings documentation updated - Regenerated golden files for updated system table counts Epic: CC-35515 Release note: None
sadaf-crl
requested review from
a team,
angles-n-daemons,
bghal,
kev-cao,
rahulcrl,
vishalv1994 and
visheshbardia
and removed request for
a team
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| // * license_id: the license ID under which vCPUs are consumed (NULL if no license installed). | ||
| // * hour_timestamp: the timestamp of the hour bucket for this measurement. | ||
| // * node_id: the ID of the node reporting vCPU usage. | ||
| // * num_vcpu: the number of vCPUs on the node during this hour. | ||
| VcpuHoursAuditTableSchema = ` | ||
| CREATE TABLE system.vcpu_hours_audit ( | ||
| license_id STRING, | ||
| hour_timestamp TIMESTAMPTZ NOT NULL, | ||
| node_id INT8 NOT NULL, | ||
| num_vcpu FLOAT NOT NULL, | ||
| CONSTRAINT "primary" PRIMARY KEY (license_id, hour_timestamp, node_id), |
There was a problem hiding this comment.
Bug: The comment on line 1446 documents license_id as "NULL if no license installed," but license_id is part of the PRIMARY KEY (line 1456), which makes it implicitly NOT NULL. The bootstrap testdata confirms the column resolves to license_id STRING NOT NULL. Any future insertion with license_id = NULL for unlicensed nodes will fail with a NOT NULL violation.
Suggested fix: either update the comment to document a sentinel value (e.g., empty string '') instead of NULL when no license is installed, or restructure the primary key to exclude license_id if NULL semantics are truly needed.
AI Review: Potential Issue DetectedAn inline comment has been added to If helpful: add |
github-actions
Bot
added
the
o-AI-Review-Potential-Issue-Detected
| license_id STRING, | ||
| hour_timestamp TIMESTAMPTZ NOT NULL, | ||
| node_id INT8 NOT NULL, | ||
| num_vcpu FLOAT NOT NULL, |
There was a problem hiding this comment.
We are using FLOAT for vCPUs. Can it be fractional?
| node_id INT8 NOT NULL, | ||
| num_vcpu FLOAT NOT NULL, | ||
| CONSTRAINT "primary" PRIMARY KEY (license_id, hour_timestamp, node_id), | ||
| FAMILY "primary" (license_id, hour_timestamp, node_id, num_vcpu) |
There was a problem hiding this comment.
Using the PK with license_id + hour_timestamp prefix, will hotspot writes on a single range. One license per cluster + same hour bucket means every node's hourly insert shares the same key prefix, so they all
sort adjacent and land in the same range IIUC and one leaseholder serializes the whole cluster's writes. One range might do all the work, which can be a problem if deployment has a huge no. of nodes.
Can we use a hash-sharded leading column like statement_statistics does? https://github.com/cockroachdb/cockroach/blob/master/pkg/sql/catalog/systemschema/system.go#L778
| // * hour_timestamp: the timestamp of the hour bucket for this measurement. | ||
| // * node_id: the ID of the node reporting vCPU usage. | ||
| // * num_vcpu: the number of vCPUs on the node during this hour. | ||
| VcpuHoursAuditTableSchema = ` |
There was a problem hiding this comment.
Query: The PR desc says retention is 30 days "via a future GC mechanism". Are we planning to add a TTL later?
3 participants
Add the system.vcpu_hours_audit table to track per-node, per-hour vCPU consumption for license auditing. This table stores:
The table has a composite primary key on (node_id, license_id, hour_timestamp) and is configured with admin read/write privileges. Data retention is intended to be 30 days via a future GC mechanism.
This is part of the self-hosted license audit feature to enable customers to generate vCPU consumption reports for compliance tracking and renewals.
Changes include:
Epic: CC-35515
Release note: None