Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce rw_worker_nodes, rw_parallel_units and support show clusters command #10656

Merged
merged 5 commits into from Jul 3, 2023

Conversation

yezizp2012
Copy link
Contributor

@yezizp2012 yezizp2012 commented Jun 29, 2023

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Related #8803.

  1. introduce system tables rw_worker_nodes and rw_parallel_units:
  • rw_worker_nodes
Field Type Description
id Int32 Unique identifier of the worker node
host Varchar Host of the worker node
port Varchar Serving port of the worker node
type Varchar Type of the worker node
state Varchar State of the worker node, should be one of [RUNNING, STARTING]
parallelism Int32 Streaming parallelism of the worker node
is_streaming Boolean Whether the worker node is in streaming mode
is_serving Boolean Whether the worker node is in batch serving mode
is_unschedulable Boolean Whether the worker node is unschedulable for streaming job
  • rw_parallel_units
Field Type Description
id Int32 Unique identifier of the parallel unit
worker_id Int32 Identifier of the worker id that the parallel unit belongs to
  1. support command show clusters:
dev=> show clusters;
      Addr      |  State  |     Parallel Units     | Is Streaming | Is Serving | Is Unschedulable
----------------+---------+------------------------+--------------+------------+------------------
 127.0.0.1:5688 | RUNNING | 3000, 3001, 3002, 3003 | true         | true       | false
(1 row)
  1. change type of columns in rw_sources to array type of varchar and remove column primary_keys.
  2. change type of arg_type_ids in rw_functions to array type of integer.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

  • My PR contains user-facing changes.
Click here for Documentation

Types of user-facing changes

Please keep the types that apply to your changes, and remove the others.

  • SQL commands, functions, and operators

Release note

  1. introduce system tables rw_worker_nodes and rw_parallel_units.
  2. support command SHOW CLUSTERS.
  3. remove primary_keys in rw_sources.

@github-actions github-actions bot added type/feature user-facing-changes Contains changes that are visible to users labels Jun 29, 2023
@yezizp2012
Copy link
Contributor Author

Some examples of a local run:

dev=> select * from rw_worker_nodes;
  id  |   host    | port |     type     |  state  | parallelism | is_streaming | is_serving | is_unschedulable
------+-----------+------+--------------+---------+-------------+--------------+------------+------------------
 3001 | 127.0.0.1 | 5688 | COMPUTE_NODE | RUNNING |           4 | t            | t          | f
(1 row)

dev=> show clusters;
      Addr      |  State  |     Parallel Units     | Is Streaming | Is Serving | Is Unschedulable
----------------+---------+------------------------+--------------+------------+------------------
 127.0.0.1:5688 | RUNNING | 3000, 3001, 3002, 3003 | true         | true       | false
(1 row)

dev=> select * from rw_parallel_units ;
  id  | worker_id
------+-----------
 3000 |      3001
 3001 |      3001
 3002 |      3001
 3003 |      3001
(4 rows)

@codecov
Copy link

codecov bot commented Jun 29, 2023

Codecov Report

Merging #10656 (297acee) into main (0ede620) will decrease coverage by 0.03%.
The diff coverage is 18.78%.

@@            Coverage Diff             @@
##             main   #10656      +/-   ##
==========================================
- Coverage   70.22%   70.19%   -0.03%     
==========================================
  Files        1280     1282       +2     
  Lines      220315   220452     +137     
==========================================
+ Hits       154718   154753      +35     
- Misses      65597    65699     +102     
Flag Coverage Δ
rust 70.19% <18.78%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ntend/src/catalog/system_catalog/rw_catalog/mod.rs 0.00% <0.00%> (ø)
src/frontend/src/handler/show.rs 27.31% <0.00%> (-4.41%) ⬇️
src/meta/src/manager/cluster.rs 70.24% <ø> (ø)
src/sqlparser/src/ast/mod.rs 85.03% <0.00%> (-0.07%) ⬇️
src/sqlparser/src/keywords.rs 100.00% <ø> (ø)
src/sqlparser/src/parser.rs 86.81% <0.00%> (-0.03%) ⬇️
src/frontend/src/catalog/system_catalog/mod.rs 45.20% <100.00%> (+0.76%) ⬆️
.../catalog/system_catalog/rw_catalog/rw_functions.rs 100.00% <100.00%> (ø)
...rc/catalog/system_catalog/rw_catalog/rw_sources.rs 100.00% <100.00%> (ø)

... and 5 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@st1page st1page left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, up to you @BugenZhao

Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@yezizp2012 yezizp2012 added this pull request to the merge queue Jul 3, 2023
Merged via the queue into main with commit da25e2d Jul 3, 2023
36 checks passed
@yezizp2012 yezizp2012 deleted the zp/rw-sys-cluster-table branch July 3, 2023 04:10
@emile-00 emile-00 added the 📖✓ Covered or will be covered in the user docs. label Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature user-facing-changes Contains changes that are visible to users 📖✓ Covered or will be covered in the user docs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants