Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: http debug api for config #733

Merged
merged 4 commits into from
Mar 15, 2023
Merged

Conversation

ShiKaiWi
Copy link
Member

@ShiKaiWi ShiKaiWi commented Mar 14, 2023

Which issue does this PR close?

Closes #693

Rationale for this change

Refer to #693

What changes are included in this PR?

  • Support debug api for reviewing the config when running;

Are there any user-facing changes?

Yes. Try the new api /debug/config, which will output the runtime config in json format.

How does this change test

Manually. Here is the output of the api:

[node]
addr = "127.0.0.1"
zone = ""
idc = ""
binary_version = ""

[server]
bind_addr = "0.0.0.0"
mysql_port = 3307
http_port = 5440
grpc_port = 8831
http_max_body_size = 65536
grpc_server_cq_count = 20
resp_compress_min_length = "4MiB"
auto_create_table = true

[server.forward]
enable = false
thread_num = 4
max_send_msg_len = 20971520
max_recv_msg_len = 1073741824
keep_alive_while_idle = true

[server.forward.keep_alive_interval]
secs = 600
nanos = 0

[server.forward.keep_alive_timeout]
secs = 3
nanos = 0

[server.forward.connect_timeout]
secs = 3
nanos = 0

[server.forward.forward_timeout]
secs = 60
nanos = 0

[runtime]
read_thread_num = 8
write_thread_num = 8
meta_thread_num = 2
background_thread_num = 8

[logger]
level = "info"
enable_async = true
async_channel_len = 102400

[tracing]
prefix = "tracing"
dir = "/tmp/ceresdb"
level = "info"

[analytic]
replay_batch_size = 500
max_replay_tables_per_batch = 64
write_group_worker_num = 8
write_group_command_channel_cap = 128
sst_meta_cache_cap = 1000
sst_data_cache_cap = 1000
space_write_buffer_size = 0
db_write_buffer_size = 0
scan_batch_size = 500
sst_background_read_parallelism = 8

[analytic.storage]
mem_cache_capacity = "512MiB"
mem_cache_partition_bits = 6
disk_cache_capacity = "0KiB"
disk_cache_page_size = "2MiB"
disk_cache_dir = "/tmp/ceresdb"

[analytic.storage.object_store]
type = "Local"
data_dir = "/tmp/ceresdb"

[analytic.table_opts]
update_mode = "Overwrite"
storage_format_hint = "Auto"
enable_ttl = true
ttl = "7d"
arena_block_size = 2097152
write_buffer_size = 33554432
compaction_strategy = "Default"
num_rows_per_row_group = 8192
compression = "Zstd"

[analytic.compaction_config]
schedule_channel_len = 16
schedule_interval = "30m"
max_ongoing_tasks = 8
max_unflushed_duration = "5h"
memory_limit = "4GiB"

[analytic.manifest]
snapshot_every_n_updates = 10000
scan_timeout = "5s"
scan_batch_size = 100
store_timeout = "5s"

[analytic.wal]
type = "RocksDB"
data_dir = "/tmp/ceresdb"

[analytic.remote_engine_client]
connect_timeout = "3s"
channel_pool_max_size_per_partition = 16
channel_pool_partition_num = 16
channel_keep_alive_while_idle = true
channel_keep_alive_timeout = "3s"
channel_keep_alive_interval = "10m"
route_cache_max_size_per_partition = 16
route_cache_partition_num = 16

[query_engine]
read_parallelism = 8

[limiter]
write_block_list = []
read_block_list = []
rules = []

@codecov-commenter
Copy link

Codecov Report

Merging #733 (d919118) into main (8837aa6) will increase coverage by 0.16%.
The diff coverage is 79.20%.

❗ Current head d919118 differs from pull request most recent head a4298fb. Consider uploading reports for the commit a4298fb to get more accurate results

@@            Coverage Diff             @@
##             main     #733      +/-   ##
==========================================
+ Coverage   68.26%   68.42%   +0.16%     
==========================================
  Files         288      294       +6     
  Lines       45335    45710     +375     
==========================================
+ Hits        30946    31278     +332     
- Misses      14389    14432      +43     
Impacted Files Coverage Δ
analytic_engine/src/memtable/mod.rs 81.25% <ø> (ø)
analytic_engine/src/row_iter/chain.rs 55.80% <0.00%> (-0.76%) ⬇️
cluster/src/config.rs 75.00% <0.00%> (ø)
common_util/src/config.rs 87.02% <0.00%> (ø)
components/logger/src/lib.rs 27.98% <0.00%> (ø)
components/tracing_util/src/logging.rs 0.00% <0.00%> (ø)
query_engine/src/config.rs 83.33% <0.00%> (ø)
router/src/rule_based.rs 0.00% <0.00%> (ø)
server/src/config.rs 37.64% <0.00%> (ø)
server/src/grpc/forward.rs 79.48% <0.00%> (ø)
... and 33 more

... and 2 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

src/setup.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@jiacai2050 jiacai2050 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jiacai2050 jiacai2050 added this pull request to the merge queue Mar 15, 2023
Merged via the queue into apache:main with commit 56bb7b0 Mar 15, 2023
chunshao90 pushed a commit to chunshao90/ceresdb that referenced this pull request May 15, 2023
* feat: http api for server config

* fix clippy warning

* Serialize config

* display config in toml format
@ShiKaiWi ShiKaiWi deleted the feat-http-config branch May 29, 2023 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Http debug api for showing the config used by server
3 participants