Tracking issue for the MetricEngine

[waynexia]

What type of enhancement is this?

Performance

What does the enhancement do?

RFC metric-engine proposes a new region engine Metric. Here is the tracking issue for implementing it.

Related progress is also tracked in GitHub Project https://github.com/orgs/GreptimeTeam/projects/34

• RFC document docs: rfc of metric engine #1925
• Meta Server
  • Reserve the first 8 bits of region number for region id group feat: define region group and sequence #2100
  • Add logical route entry to table route
  • Add special logic to create progress when table engine is metric
  • New alter procedure that acquires the physical table's resource lock before operating the logical table feat: metric engine support alter #3098
  • Support grouping CREATE requests
  • Translate table name of physical table to its table id when handle CREATE request of logical table
  • Support alter to logical table
• Frontend
  • Also support automatically creating the physical table under public schema.
  • Convert requests from prometheus remote write into one request to the physical table
  • Generate DDL for logical table from incoming prom remote write requests and submit to metasrv (for updating metadata of logical table)
  • Separate GreptimeDatabase service into two different services
• Datanode
  • Define RegionEngine and RegionServer feat: define region server and related requests #2160
  • Define MetricEngine on top of MitoEngine
  • Define MetadataRegion to handle operations to metadata feat: Metric engine skeleton #2687
  • DataRegion to handle operations to data feat: create logical region to metric engine #2706
  • Handle Create request feat: implement create region request for metric engine #2694
  • Handle Write request feat: write logical region to metric engine #2759
  • Handle Read request feat: handle scan request on metric engine #2793
  • Handle Alter request feat: alter logical region in metric region #2726
  • Forward INSERT/SELECT to physical table to mito engine (is implemented together with corresponding logical operation)
  • Be able to recognize physical table (by their table id?)
  • Implement failover/heartbeat/... for MetricEngine
  • Change a platform-independent hash algorithm for TSID
  • Maintains the schema of physical table in metasrv

Implementation challenges

No response

[tisonkun]

Is the TODO list above updated? Perhaps we can create subtasks issue for each task so that we can keep the status synced.

Take https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/about-task-lists as the reference.

[killme2008]

From now on, we need to keep compatibility with older versions as possible as we can.