Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds /config endpoint to status server that returns the tikv config #4997

Merged

Conversation

Projects
None yet
6 participants
@jlerche
Copy link
Contributor

commented Jun 30, 2019

What have you changed? (mandatory)

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • Adds a /config endpoint to StatusServer that returns the TiKvConfig as json
  • Adds a unit test that checks if a StatusServer created with a default TiKvConfig returns a json string that is the same as serializing a default TiKvConfig to a json string directly.

What are the type of the changes? (mandatory)

The currently defined types are listed below, please pick one of the types for this PR by removing the others:

  • New feature (change which adds functionality)

How has this PR been tested? (mandatory)

  • Manual testing (curl http://status.server.addr:20180/config)
  • Unit test

Refer to a related PR or issue link (optional)

#4901

@jlerche

This comment has been minimized.

Copy link
Contributor Author

commented Jun 30, 2019

@liufuyang please take a look

I'm not sure if my approach was the best way, I tried a variety of approaches and settled on this to minimize changes outside of status_server.rs. I did some slight restructuring to satisfy lifetime requirements for the closures in build.serve and service_fn when passing in config.

Show resolved Hide resolved src/server/status_server.rs Outdated
@siddontang

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2019

how about the output?

@jlerche

This comment has been minimized.

Copy link
Contributor Author

commented Jul 1, 2019

using httpie for pretty printing

$ http http://127.0.0.1:20180/config
HTTP/1.1 200 OK
content-length: 11956
content-type: application/json
date: Mon, 01 Jul 2019 01:26:00 GMT

{
    "coprocessor": {
        "batch-split-limit": 10,
        "region-max-keys": 1440000,
        "region-max-size": "144MiB",
        "region-split-keys": 960000,
        "region-split-size": "96MiB",
        "split-region-on-table": true
    },
    "import": {
        "import-dir": "/tmp/tikv/import",
        "max-open-engines": 8,
        "max-prepare-duration": "5m",
        "min-available-ratio": 0.05,
        "num-import-jobs": 8,
        "num-import-sst-jobs": 2,
        "num-threads": 8,
        "region-split-size": "512MiB",
        "stream-channel-window": 128,
        "upload-speed-limit": "512MiB"
    },
    "log-file": "",
    "log-level": "info",
    "log-rotation-timespan": "1d",
    "metric": {
        "address": "",
        "interval": "15s",
        "job": "tikv"
    },
    "panic-when-unexpected-key-or-data": false,
    "pd": {
        "endpoints": [
            "127.0.0.1:2379"
        ],
        "retry-interval": "300ms",
        "retry-log-every": 10,
        "retry-max-count": 9223372036854775807
    },
    "pessimistic-txn": {
        "enabled": true,
        "monitor-membership-interval": 3000,
        "wait-for-lock-timeout": 3000,
        "wake-up-delay-duration": 1
    },
    "raftdb": {
        "allow-concurrent-memtable-write": false,
        "bytes-per-sync": "1MiB",
        "compaction-readahead-size": "0KiB",
        "create-if-missing": true,
        "defaultcf": {
            "block-based-bloom-filter": false,
            "block-cache-size": "643MiB",
            "block-size": "64KiB",
            "bloom-filter-bits-per-key": 10,
            "cache-index-and-filter-blocks": true,
            "compaction-pri": 0,
            "compaction-style": 0,
            "compression-per-level": [
                "no",
                "no",
                "lz4",
                "lz4",
                "lz4",
                "zstd",
                "zstd"
            ],
            "disable-auto-compactions": false,
            "disable-block-cache": false,
            "dynamic-level-bytes": true,
            "hard-pending-compaction-bytes-limit": "256GiB",
            "level0-file-num-compaction-trigger": 4,
            "level0-slowdown-writes-trigger": 20,
            "level0-stop-writes-trigger": 36,
            "max-bytes-for-level-base": "512MiB",
            "max-bytes-for-level-multiplier": 10,
            "max-compaction-bytes": "2GiB",
            "max-write-buffer-number": 5,
            "min-write-buffer-number-to-merge": 1,
            "num-levels": 7,
            "optimize-filters-for-hits": true,
            "pin-l0-filter-and-index-blocks": true,
            "prop-keys-index-distance": 40960,
            "prop-size-index-distance": 4194304,
            "read-amp-bytes-per-bit": 0,
            "soft-pending-compaction-bytes-limit": "64GiB",
            "target-file-size-base": "8MiB",
            "titan": {
                "blob-cache-size": "0KiB",
                "blob-file-compression": "lz4",
                "blob-run-mode": "normal",
                "discardable-ratio": 0.5,
                "max-gc-batch-size": "64MiB",
                "merge-small-file-threshold": "8MiB",
                "min-blob-size": 1024,
                "min-gc-batch-size": "16MiB",
                "sample-ratio": 0.1
            },
            "use-bloom-filter": false,
            "whole-key-filtering": true,
            "write-buffer-size": "128MiB"
        },
        "enable-pipelined-write": true,
        "enable-statistics": true,
        "info-log-dir": "",
        "info-log-keep-log-file-num": 10,
        "info-log-max-size": "1GiB",
        "info-log-roll-time": "0s",
        "max-background-jobs": 4,
        "max-manifest-file-size": "20MiB",
        "max-open-files": 40960,
        "max-sub-compactions": 2,
        "max-total-wal-size": "4GiB",
        "stats-dump-period": "10m",
        "use-direct-io-for-flush-and-compaction": false,
        "wal-bytes-per-sync": "512KiB",
        "wal-dir": "",
        "wal-recovery-mode": 2,
        "wal-size-limit": "0KiB",
        "wal-ttl-seconds": 0,
        "writable-file-max-buffer-size": "1MiB"
    },
    "raftstore": {
        "abnormal-leader-missing-duration": "10m",
        "allow-remove-leader": false,
        "apply-max-batch-size": 1024,
        "apply-pool-size": 2,
        "capacity": "0KiB",
        "clean-stale-peer-delay": "11m",
        "cleanup-import-sst-interval": "10m",
        "consistency-check-interval": "0s",
        "future-poll-size": 1,
        "hibernate-regions": false,
        "leader-transfer-max-log-lag": 10,
        "local-read-batch-size": 1024,
        "lock-cf-compact-bytes-threshold": "256MiB",
        "lock-cf-compact-interval": "10m",
        "max-leader-missing-duration": "2h",
        "max-peer-down-duration": "5m",
        "merge-check-tick-interval": "10s",
        "merge-max-log-gap": 10,
        "messages-per-tick": 4096,
        "notify-capacity": 40960,
        "pd-heartbeat-tick-interval": "1m",
        "pd-store-heartbeat-tick-interval": "10s",
        "peer-stale-state-check-interval": "5m",
        "prevote": true,
        "raft-base-tick-interval": "1s",
        "raft-election-timeout-ticks": 10,
        "raft-entry-cache-life-time": "30s",
        "raft-entry-max-size": "8MiB",
        "raft-heartbeat-ticks": 2,
        "raft-log-gc-count-limit": 73728,
        "raft-log-gc-size-limit": "72MiB",
        "raft-log-gc-threshold": 50,
        "raft-log-gc-tick-interval": "10s",
        "raft-max-election-timeout-ticks": 20,
        "raft-max-inflight-msgs": 256,
        "raft-max-size-per-msg": "1MiB",
        "raft-min-election-timeout-ticks": 10,
        "raft-reject-transfer-leader-duration": "3s",
        "raft-store-max-leader-lease": "9s",
        "raftdb-path": "/home/lerche/programming_projects/rust/jlerche_tikv/raft",
        "region-compact-check-interval": "5m",
        "region-compact-check-step": 100,
        "region-compact-min-tombstones": 10000,
        "region-compact-tombstones-percent": 30,
        "region-split-check-diff": "6MiB",
        "report-region-flow-interval": "1m",
        "right-derive-when-split": true,
        "snap-apply-batch-size": "10MiB",
        "snap-gc-timeout": "4h",
        "snap-mgr-gc-tick-interval": "1m",
        "split-region-check-tick-interval": "10s",
        "store-max-batch-size": 1024,
        "store-pool-size": 2,
        "sync-log": true,
        "use-delete-range": false
    },
    "readpool": {
        "coprocessor": {
            "high-concurrency": 12,
            "low-concurrency": 12,
            "max-tasks-per-worker-high": 2000,
            "max-tasks-per-worker-low": 2000,
            "max-tasks-per-worker-normal": 2000,
            "normal-concurrency": 12,
            "stack-size": "10MiB"
        },
        "storage": {
            "high-concurrency": 4,
            "low-concurrency": 4,
            "max-tasks-per-worker-high": 2000,
            "max-tasks-per-worker-low": 2000,
            "max-tasks-per-worker-normal": 2000,
            "normal-concurrency": 4,
            "stack-size": "10MiB"
        }
    },
    "rocksdb": {
        "auto-tuned": false,
        "bytes-per-sync": "1MiB",
        "compaction-readahead-size": "0KiB",
        "create-if-missing": true,
        "defaultcf": {
            "block-based-bloom-filter": false,
            "block-cache-size": "8041MiB",
            "block-size": "64KiB",
            "bloom-filter-bits-per-key": 10,
            "cache-index-and-filter-blocks": true,
            "compaction-pri": 3,
            "compaction-style": 0,
            "compression-per-level": [
                "no",
                "no",
                "lz4",
                "lz4",
                "lz4",
                "zstd",
                "zstd"
            ],
            "disable-auto-compactions": false,
            "disable-block-cache": false,
            "dynamic-level-bytes": true,
            "hard-pending-compaction-bytes-limit": "256GiB",
            "level0-file-num-compaction-trigger": 4,
            "level0-slowdown-writes-trigger": 20,
            "level0-stop-writes-trigger": 36,
            "max-bytes-for-level-base": "512MiB",
            "max-bytes-for-level-multiplier": 10,
            "max-compaction-bytes": "2GiB",
            "max-write-buffer-number": 5,
            "min-write-buffer-number-to-merge": 1,
            "num-levels": 7,
            "optimize-filters-for-hits": true,
            "pin-l0-filter-and-index-blocks": true,
            "prop-keys-index-distance": 40960,
            "prop-size-index-distance": 4194304,
            "read-amp-bytes-per-bit": 0,
            "soft-pending-compaction-bytes-limit": "64GiB",
            "target-file-size-base": "8MiB",
            "titan": {
                "blob-cache-size": "0KiB",
                "blob-file-compression": "lz4",
                "blob-run-mode": "normal",
                "discardable-ratio": 0.5,
                "max-gc-batch-size": "64MiB",
                "merge-small-file-threshold": "8MiB",
                "min-blob-size": 1024,
                "min-gc-batch-size": "16MiB",
                "sample-ratio": 0.1
            },
            "use-bloom-filter": true,
            "whole-key-filtering": true,
            "write-buffer-size": "128MiB"
        },
        "enable-pipelined-write": true,
        "enable-statistics": true,
        "info-log-dir": "",
        "info-log-keep-log-file-num": 10,
        "info-log-max-size": "1GiB",
        "info-log-roll-time": "0s",
        "lockcf": {
            "block-based-bloom-filter": false,
            "block-cache-size": "643MiB",
            "block-size": "16KiB",
            "bloom-filter-bits-per-key": 10,
            "cache-index-and-filter-blocks": true,
            "compaction-pri": 0,
            "compaction-style": 0,
            "compression-per-level": [
                "no",
                "no",
                "no",
                "no",
                "no",
                "no",
                "no"
            ],
            "disable-auto-compactions": false,
            "disable-block-cache": false,
            "dynamic-level-bytes": true,
            "hard-pending-compaction-bytes-limit": "256GiB",
            "level0-file-num-compaction-trigger": 1,
            "level0-slowdown-writes-trigger": 20,
            "level0-stop-writes-trigger": 36,
            "max-bytes-for-level-base": "128MiB",
            "max-bytes-for-level-multiplier": 10,
            "max-compaction-bytes": "2GiB",
            "max-write-buffer-number": 5,
            "min-write-buffer-number-to-merge": 1,
            "num-levels": 7,
            "optimize-filters-for-hits": false,
            "pin-l0-filter-and-index-blocks": true,
            "prop-keys-index-distance": 40960,
            "prop-size-index-distance": 4194304,
            "read-amp-bytes-per-bit": 0,
            "soft-pending-compaction-bytes-limit": "64GiB",
            "target-file-size-base": "8MiB",
            "titan": {
                "blob-cache-size": "0KiB",
                "blob-file-compression": "lz4",
                "blob-run-mode": "normal",
                "discardable-ratio": 0.5,
                "max-gc-batch-size": "64MiB",
                "merge-small-file-threshold": "8MiB",
                "min-blob-size": 4294967296,
                "min-gc-batch-size": "16MiB",
                "sample-ratio": 0.1
            },
            "use-bloom-filter": true,
            "whole-key-filtering": true,
            "write-buffer-size": "128MiB"
        },
        "max-background-jobs": 6,
        "max-manifest-file-size": "128MiB",
        "max-open-files": 40960,
        "max-sub-compactions": 1,
        "max-total-wal-size": "4GiB",
        "raftcf": {
            "block-based-bloom-filter": false,
            "block-cache-size": "128MiB",
            "block-size": "16KiB",
            "bloom-filter-bits-per-key": 10,
            "cache-index-and-filter-blocks": true,
            "compaction-pri": 0,
            "compaction-style": 0,
            "compression-per-level": [
                "no",
                "no",
                "no",
                "no",
                "no",
                "no",
                "no"
            ],
            "disable-auto-compactions": false,
            "disable-block-cache": false,
            "dynamic-level-bytes": true,
            "hard-pending-compaction-bytes-limit": "256GiB",
            "level0-file-num-compaction-trigger": 1,
            "level0-slowdown-writes-trigger": 20,
            "level0-stop-writes-trigger": 36,
            "max-bytes-for-level-base": "128MiB",
            "max-bytes-for-level-multiplier": 10,
            "max-compaction-bytes": "2GiB",
            "max-write-buffer-number": 5,
            "min-write-buffer-number-to-merge": 1,
            "num-levels": 7,
            "optimize-filters-for-hits": true,
            "pin-l0-filter-and-index-blocks": true,
            "prop-keys-index-distance": 40960,
            "prop-size-index-distance": 4194304,
            "read-amp-bytes-per-bit": 0,
            "soft-pending-compaction-bytes-limit": "64GiB",
            "target-file-size-base": "8MiB",
            "titan": {
                "blob-cache-size": "0KiB",
                "blob-file-compression": "lz4",
                "blob-run-mode": "normal",
                "discardable-ratio": 0.5,
                "max-gc-batch-size": "64MiB",
                "merge-small-file-threshold": "8MiB",
                "min-blob-size": 4294967296,
                "min-gc-batch-size": "16MiB",
                "sample-ratio": 0.1
            },
            "use-bloom-filter": true,
            "whole-key-filtering": true,
            "write-buffer-size": "128MiB"
        },
        "rate-bytes-per-sec": "0KiB",
        "rate-limiter-mode": 2,
        "stats-dump-period": "10m",
        "titan": {
            "dirname": "",
            "disable-gc": false,
            "enabled": false,
            "max-background-gc": 1
        },
        "use-direct-io-for-flush-and-compaction": false,
        "wal-bytes-per-sync": "512KiB",
        "wal-dir": "",
        "wal-recovery-mode": 2,
        "wal-size-limit": "0KiB",
        "wal-ttl-seconds": 0,
        "writable-file-max-buffer-size": "1MiB",
        "writecf": {
            "block-based-bloom-filter": false,
            "block-cache-size": "4825MiB",
            "block-size": "64KiB",
            "bloom-filter-bits-per-key": 10,
            "cache-index-and-filter-blocks": true,
            "compaction-pri": 3,
            "compaction-style": 0,
            "compression-per-level": [
                "no",
                "no",
                "lz4",
                "lz4",
                "lz4",
                "zstd",
                "zstd"
            ],
            "disable-auto-compactions": false,
            "disable-block-cache": false,
            "dynamic-level-bytes": true,
            "hard-pending-compaction-bytes-limit": "256GiB",
            "level0-file-num-compaction-trigger": 4,
            "level0-slowdown-writes-trigger": 20,
            "level0-stop-writes-trigger": 36,
            "max-bytes-for-level-base": "512MiB",
            "max-bytes-for-level-multiplier": 10,
            "max-compaction-bytes": "2GiB",
            "max-write-buffer-number": 5,
            "min-write-buffer-number-to-merge": 1,
            "num-levels": 7,
            "optimize-filters-for-hits": false,
            "pin-l0-filter-and-index-blocks": true,
            "prop-keys-index-distance": 40960,
            "prop-size-index-distance": 4194304,
            "read-amp-bytes-per-bit": 0,
            "soft-pending-compaction-bytes-limit": "64GiB",
            "target-file-size-base": "8MiB",
            "titan": {
                "blob-cache-size": "0KiB",
                "blob-file-compression": "lz4",
                "blob-run-mode": "normal",
                "discardable-ratio": 0.5,
                "max-gc-batch-size": "64MiB",
                "merge-small-file-threshold": "8MiB",
                "min-blob-size": 4294967296,
                "min-gc-batch-size": "16MiB",
                "sample-ratio": 0.1
            },
            "use-bloom-filter": true,
            "whole-key-filtering": false,
            "write-buffer-size": "128MiB"
        }
    },
    "security": {
        "ca-path": "",
        "cert-path": "",
        "cipher-file": "",
        "key-path": ""
    },
    "server": {
        "addr": "127.0.0.1:20160",
        "advertise-addr": "127.0.0.1:20160",
        "concurrent-recv-snap-limit": 32,
        "concurrent-send-snap-limit": 32,
        "end-point-batch-row-limit": 64,
        "end-point-enable-batch-if-possible": true,
        "end-point-recursion-limit": 1000,
        "end-point-request-max-handle-duration": "1m",
        "end-point-stream-batch-row-limit": 128,
        "end-point-stream-channel-size": 8,
        "grpc-compression-type": "none",
        "grpc-concurrency": 4,
        "grpc-concurrent-stream": 1024,
        "grpc-keepalive-time": "10s",
        "grpc-keepalive-timeout": "3s",
        "grpc-raft-conn-num": 1,
        "grpc-stream-initial-window-size": "2MiB",
        "heavy-load-threshold": 300,
        "heavy-load-wait-duration": "1ms",
        "labels": {},
        "snap-max-total-size": "0KiB",
        "snap-max-write-bytes-per-sec": "100MiB",
        "stats-concurrency": 1,
        "status-addr": "127.0.0.1:20180",
        "status-thread-pool-size": 1
    },
    "storage": {
        "block-cache": {
            "capacity": "14152MiB",
            "high-pri-pool-ratio": 0.0,
            "num-shard-bits": 6,
            "shared": true,
            "strict-capacity-limit": false
        },
        "data-dir": "./",
        "gc-ratio-threshold": 1.1,
        "max-key-size": 4096,
        "scheduler-concurrency": 2048000,
        "scheduler-notify-capacity": 10240,
        "scheduler-pending-write-threshold": "100MiB",
        "scheduler-worker-pool-size": 8
    }
}
@liufuyang

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2019

firstly I think you need to do something like cargo fmt then commit and push again :)
Also use -s option while git commit, so to sign-off your commit.

@siddontang

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2019

oh, we have so many configurations!!!!!!

Seem we must reduce many in Q3.

@liufuyang

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2019

@siddontang the "no-fail" compile option is not be able to show in the config list (above output) for now?

@@ -344,7 +344,8 @@ fn run_raft_server(pd_client: RpcClient, cfg: &TiKvConfig, security_mgr: Arc<Sec
let mut status_enabled = cfg.metric.address.is_empty() && !server_cfg.status_addr.is_empty();

// Create a status server.
let mut status_server = StatusServer::new(server_cfg.status_thread_pool_size);
// TODO: How to keep cfg updated?

This comment has been minimized.

Copy link
@liufuyang

liufuyang Jul 1, 2019

Contributor

@siddontang Does tikv support internal config update without needing to restart itself?

This comment has been minimized.

Copy link
@breeswish

breeswish Jul 2, 2019

Member

This feature will be added soon.

This comment has been minimized.

Copy link
@Hoverbear

Hoverbear Jul 2, 2019

Member

Is there any planned way to handle replicating config changes through the modules? Destroy/recreate the status server seems a bit extreme.

This comment has been minimized.

Copy link
@breeswish

breeswish Jul 3, 2019

Member

I think the solution would be a (global) Config repository that provides config access and mutation interfaces, maybe change listeners as well.

This comment has been minimized.

Copy link
@breeswish

breeswish Jul 3, 2019

Member

I think the solution would be a (global) Config repository that provides config access and mutation interfaces, maybe change listeners as well.

@liufuyang
Copy link
Contributor

left a comment

Looks good to me. I also tried to play with your code locally, besides some effort on trying to fight with the compilers I don't think I found any better implementation methods. Let's see what others would give input on this. Perhaps @breeswish would give a look here?

@breeswish
Copy link
Member

left a comment

Thanks a lot. This PR looks fine, but I'm afraid that it will be refactored when we want dynamic configs.

@@ -251,7 +277,7 @@ impl StatusServer {
#[cfg(not(feature = "no-fail"))]
fn handle_fail_points_request(
req: Request<Body>,
) -> Box<dyn Future<Item = Response<Body>, Error = hyper::Error> + Send> {
) -> Box<dyn Future<Item=Response<Body>, Error=hyper::Error> + Send> {

This comment has been minimized.

Copy link
@breeswish

breeswish Jul 2, 2019

Member

I believe this won't pass rustfmt check.

@@ -344,7 +344,8 @@ fn run_raft_server(pd_client: RpcClient, cfg: &TiKvConfig, security_mgr: Arc<Sec
let mut status_enabled = cfg.metric.address.is_empty() && !server_cfg.status_addr.is_empty();

// Create a status server.
let mut status_server = StatusServer::new(server_cfg.status_thread_pool_size);
// TODO: How to keep cfg updated?

This comment has been minimized.

Copy link
@breeswish

breeswish Jul 2, 2019

Member

This feature will be added soon.

@breeswish

This comment has been minimized.

Copy link
Member

commented Jul 2, 2019

Please run make dev to format and run tests, and then sign-off all of your commits. Thanks!

@siddontang

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

@jlerche

Please sign off your all commits. See https://github.com/tikv/tikv/blob/master/CONTRIBUTING.md#signing-off-the-commit

If you have already pushed some commits before, you can use git rebase -i master --signoff to resign them off.

jlerche added some commits Jun 30, 2019

Adds /config endpoint to status server that returns the tikv config
Signed-off-by: Jacob Lerche <jacob.lerche@gmail.com>
Removes commented out code
Signed-off-by: Jacob Lerche <jacob.lerche@gmail.com>
rustfmt pass and rebase
Signed-off-by: Jacob Lerche <jacob.lerche@gmail.com>

@jlerche jlerche force-pushed the jlerche:tikv-4901_http_api_handle_configuration branch from 57baca8 to 11029d9 Jul 2, 2019

@jlerche

This comment has been minimized.

Copy link
Contributor Author

commented Jul 2, 2019

Sorry about the formatting, looks like that's how CLion does it. I ran make dev and rebased commits with sign off 👍

@siddontang

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

CI still failed @jlerche

Make CI pass
Signed-off-by: Jacob Lerche <jacob.lerche@gmail.com>
@breeswish
Copy link
Member

left a comment

Great job!

@Hoverbear
Copy link
Member

left a comment

Thanks for this PR @jlerche :)

@Hoverbear Hoverbear merged commit 153c2f3 into tikv:master Jul 3, 2019

2 checks passed

DCO All commits are signed off!
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.