Commits on Oct 6, 2023

1. sql: create new system observability tables and update job …

   This commit is the combination of two separate work streams,
   brought together for resolving logic test fallout simultaneously.
   
   The first, authored by @koorosh is the creation of
   system.transaction_exec_insights and system.statement_exec_insights.
   
   The second, authored by @zachlite in cockroachdb#111365 is the creation of
   system.mvcc_statistics and the MVCCStatisticsUpdate job.
   
   Regarding persisted insights:
   Before, this data was kept in memory only and tracked limited
   number of latest insights. These tables will be used to persist
   this data periodically.
   
   Tables allow to store the same information as in memory insights
   without aggregation.
   
   To control the amount of data stored in tables, there will be
   follow up PR to run GC job and prune old records.
   
   To make tables flexible to changes when some columns might become
   obsolete, most of the columns defined as nullable.
   
   Regarding persisted MVCC Statistics:
   The system.mvcc_statistics table stores historical mvcc data
   for a tenant's SQL objects. It's purpose it to serve mvcc data for a
   SQL object quickly - The span stats API is too slow to use in a hot path.
   Storing data over time unlocks new use cases like showing a table or
   index's accumulated garbage over time.
   
   The MVCCStatisticsUpdate Job is responsible for managing the contents of
   the table, decoupled from the read-hotpath.
   
   Both the table and job are baked when a cluster bootstraps itself, or upgrades
   itself from a previous version.
   
   This PR supersedes cockroachdb#111365 with the following changes:
   - Descriptor fixes to the mvcc_statistics table. No logical changes,
   just housekeeping to make sure that the create table schema and descriptors
   produce the same table.
   - Fixes to the job to make sure the job system can wind down.
   
   Partially resolves: cockroachdb#104582
   Epic: CRDB-25491
   Release note: None

   

   Zach Lite authored and koorosh committed Oct 6, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for a87f0c7
   • Browse repository at this point

   Copy the full SHA

   a87f0c7 View commit details

   Browse the repository at this point in the history