Global Resource Quota Control

[nolouch]

Feature Request

Background

A Global Resource Control mechanism that limits multiple applications' resource usage in a shared TiDB cluster, avoids interference between applications.

Design

RFC: https://github.com/pingcap/tidb/blob/master/docs/design/2022-11-25-global-resource-control.md

Details

Interface and Configuration Part

Here a tiny example of the SQL interface.

CREATE RESOURCE GROUP `group_name` RRU_PER_SECOND=10000, WRU_PER_SECOND=10000.
ALTER RESOURCE GROUP `group_name` RRU_PER_SECOND= 20000;
/* bind resource group to user*/
ALTER USER user_name RESOURCE GROUP resouce_group_name;
/* reset to default systemn resource group for the current session*/
SET CURRENT RESOURCE GROUP DEFAULT;

Above the example shown, it's easy to set up for users or tenants. In theory, we can measure at the requested level, it is able to aggregate at different levels, so it is possible to bind the resource unit limit to the user or the session to a group.

Tasks

SQL Interface

• Supports Resource Group Syntax
  • Supports Create/Alter Syntax parser: supports create/alter/drop resource group #39546 @nolouch
• DDL Part
  • Supports Create/Alter/Delete Resource Group *: Supports create/alter/drop resource group #40188 @nolouch
  • Supports Bind Resource Group for Users *: bind user to some resource group  #39561
  • Resource Group supports BURSTABLE make resources elastic #40380
• Usability
  • Show Resource Groups with SQL interface
  • Add a memory table to show Resource Groups @tiancaiamao #39781
  • History RU Metrics @JmPotato
    • Provide history RU metrics record and query tikv/pd#5854
    • Provide an information schema table to query the history RU metrics #40526
• Privilege check & constraint syntax check. privilege,planner/core: add dynamic privilege RESOURCE_GROUP_ADMIN for resource group operation | tidb-test=pr/2068 #40632 @tiancaiamao
• Test&Compatibility
  • Resource group backup from upstream to downstream  #40371
  • Test check if the resource group sync with resource manager #40370

Proto

• Protobuf relative
  • Add resource group support kvproto#1019
  • Add resource group name kvproto#1029

Resource Group Meta Manager

• Admission Control Server [Resource Manager] Meta Manage tikv/pd#5784 @nolouch
  • resource_manager: basic implement about manage resource group tikv/pd#5785
• Resource Group Sync to storage Watch the resource group changes tikv/pd#5794 @HuSharp
  • resource_manager: add watch for resource group  tikv/tikv#14022

Storage Node Level Control [For Scheduling]

• Supports make priority by resource group information
• Supports scheduling for read path @glorv
  • supports priority queue in yatp. queue: support priority queue tikv/yatp#72
  • Implement mclock scheduling for read path in tikv *: add resource group for the read path tikv/tikv#14001
• Supports scheduling for write path @Connor1996
  • Introduce priority queue for priority scheduling  tikv/tikv#14002
  • storage: add priority scheduling for scheduler worker tikv/tikv#14057

Global Quote Flow Control Module [For Quota Limit]

• *: Setup Global Resource Controller #40732 @nolouch
• Global Token Buckets: grant quota to the client, consume in client @nolouch @CabinfeverB
  • resource_manager: implement token assignment in server tikv/pd#5809
  • client, resource_manager: resource manager client tikv/pd#5810
  • resouce_manager: impl resource controller for tokens client tikv/pd#5811
  • resource_manager: impl limiter for token controlloer tikv/pd#5868
• Request Unit Cost Model #40645 @JmPotato
  • Introduce the resource control client tikv/client-go#648
  • resource_manager: introduce the resource unit model and config tikv/pd#5862 @JmPotato
• Aggregate resource consumption of resource groups in each client
  • resource_manager: implement the metrics info collecting tikv/pd#5855

Decouple priority

done

Compatibility With Other Components

BR #42266

• Disable BR for v7.0 br: filter resource group in br full restore #42267

[JmPotato]

/cc @JmPotato

[CabinfeverB]

/cc @CabinfeverB

[BornChanger]

@nolouch please make an entry for @Connor1996 on write path scheduling

[BornChanger]

and make an entry to change protobuf for @hnes