ybase: fix some tserver web UI buttons on empty pgsql_proxy_bind_address #9489
Assignees
Labels
Projects
Comments
|
To clarify, tserver web UI is on port 9000 by default. The buttons should be in the utils page. |
thepinetree
pushed a commit
that referenced
this issue
Sep 24, 2021
…c bind address Summary: #8580: Index backfill would fail if the `pgsql_proxy_bind_address` flag was unset due to an inconsistency between the default Postgres listen address and the default flag value read by the tablet server. #9489: Tablet server UI buttons for YSQL failed if the `pgsql_proxy_bind_address` flag was unset due to problematic treatment of empty string values and links would be set without host. Both issues are solved once the `pgsql_proxy_bind_address` receives a reasonable default value. Previously, when the gflag was unset, the default empty value would result in an empty host: "". In the former issue, the `PgProcessConf` instead had a separate default host of `0.0.0.0` in the case the flag was empty, while the tablet server treated the empty host as-is, leading to a discrepancy in index backfill. Similarly, when `pgsql_proxy_bind_address` defaulted to the empty host, the host for `GetDynamicUrlTile` was empty (with the appropriate port, in this case, 13000), but an empty host is notably not a wildcard address, causing URLs to be generated to `http://:13000`. This diff inherits a default value from the value of `rpc_bind_addresses` for `pgsql_proxy_bind_address`, as was done previously for `cql` and `redis` bind addresses. This new default flag value for `pgsql_proxy_bind_address` is then utilized both by the tablet server and `PgProcessConf` as the flag is defined both before the tablet server is initialized on `Init()` and before the Postgres process is initialized with `CreateValidateAndRunInitDb(...)`, solving the first issue. Additionally, the default host for `rpc_bind_addresses` is `0.0.0.0`, matching the Postgres value even if the `rpc_bind_addresses` gflag is unset. The new default flag value for `pgsql_proxy_bind_address` is also used in the `GetDynamicUrlTile` function, now with a valid host. Test Plan: Jenkins: skip For both of the below testing methods, remove the lines which set `--pgsql_proxy_bind_address` in `get_tserver_only_flags(...)` from the `yb-ctl` script to illustrate functionality in the lack of an explicit setting. **#8580** 1. Run `./bin/yb-ctl start --tserver_flags "vmodule=tablet=1` to start up local cluster with tserver logging extended 2. Run `./bin/ysqlsh --command 'create table t (i int)'; ./bin/ysqlsh --command 'create index on t (i)'` and observe result: ``` CREATE TABLE CREATE INDEX ``` after change and error: ``` ERROR: Aborted: backfill connection to DB failed: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/tmp/.yb.0/.s.PGSQL.5433"? ``` before diff **#9489** 1. Run `./bin/yb-ctl` start to start up local cluster 2. Open `http://127.0.0.1:9000/`, navigate to `Utilities`, and observe that `YSQL Live Ops` and `YSQL All Ops` route to valid hosts (`http://127.0.0.1:13000/(rpcz,statements)`) after change, vs invalid host (`http://:13000/(rpcz,statements)`) before change. **Exhibiting expected functionality even when `rpc_bind_addresses` is left unset (defaults to `0.0.0.0`)** 1. Start up master with the `rpc_bind_addresses` flag unset (which is set by default in `yb-ctl`): ``` ./build/latest/bin/yb-master --fs_data_dirs $PATH_TO_YUGABYTE_DATA/node-1/disk-1 --webserver_interface 127.0.0.1 --v 0 --version_file_json_path=$PATH_TO_YUGABYTE/build/debug-clang-dynamic-ninja --replication_factor=1 --yb_num_shards_per_tserver 2 --ysql_num_shards_per_tserver=2 --default_memory_limit_to_ram_ratio=0.35 --master_addresses 127.0.0.1:7100 --enable_ysql=true ``` 2. Start up the tserver without the `rpc_bind_addresses` flag set: ``` ./build/latest/bin/yb-tserver --fs_data_dirs $PATH_TO_YUGABYTE_DATA/node-1/disk-1 --webserver_interface 127.0.0.1 --v 0 --version_file_json_path= $PATH_TO_YUGABYTE/build/debug-clang-dynamic-ninja --tserver_master_addrs=127.0.0.1:7100 --yb_num_shards_per_tserver=2 --redis_proxy_bind_address=127.0.0.1:6379 --cql_proxy_bind_address=127.0.0.1:9042 --local_ip_for_outbound_sockets=127.0.0.1 --use_cassandra_authentication=false --ysql_num_shards_per_tserver=2 --default_memory_limit_to_ram_ratio=0.65 --enable_ysql=true --vmodule=tablet=1 ``` 3. Run `./bin/ysqlsh --command 'create table t (i int)'; ./bin/ysqlsh --command 'create index on t (i)'` and observe result: ``` CREATE TABLE CREATE INDEX ``` 4. Open `http://127.0.0.1:9000/`, navigate to `Utilities`, and observe that `YSQL Live Ops` and `YSQL All Ops` route to valid hosts (`http://127.0.0.1:13000/(rpcz,statements)`). Reviewers: jason, sanketh Reviewed By: jason, sanketh Subscribers: bogdan, sanketh, yql Differential Revision: https://phabricator.dev.yugabyte.com/D13136
thepinetree
pushed a commit
that referenced
this issue
Sep 27, 2021
…ddress from rpc bind address Summary: #8580: Index backfill would fail if the `pgsql_proxy_bind_address` flag was unset due to an inconsistency between the default Postgres listen address and the default flag value read by the tablet server. #9489: Tablet server UI buttons for YSQL failed if the `pgsql_proxy_bind_address` flag was unset due to problematic treatment of empty string values and links would be set without host. Both issues are solved once the `pgsql_proxy_bind_address` receives a reasonable default value. Previously, when the gflag was unset, the default empty value would result in an empty host: "". In the former issue, the `PgProcessConf` instead had a separate default host of `0.0.0.0` in the case the flag was empty, while the tablet server treated the empty host as-is, leading to a discrepancy in index backfill. Similarly, when `pgsql_proxy_bind_address` defaulted to the empty host, the host for `GetDynamicUrlTile` was empty (with the appropriate port, in this case, 13000), but an empty host is notably not a wildcard address, causing URLs to be generated to `http://:13000`. This diff inherits a default value from the value of `rpc_bind_addresses` for `pgsql_proxy_bind_address`, as was done previously for `cql` and `redis` bind addresses. This new default flag value for `pgsql_proxy_bind_address` is then utilized both by the tablet server and `PgProcessConf` as the flag is defined both before the tablet server is initialized on `Init()` and before the Postgres process is initialized with `CreateValidateAndRunInitDb(...)`, solving the first issue. Additionally, the default host for `rpc_bind_addresses` is `0.0.0.0`, matching the Postgres value even if the `rpc_bind_addresses` gflag is unset. The new default flag value for `pgsql_proxy_bind_address` is also used in the `GetDynamicUrlTile` function, now with a valid host. Original Commit: 3f1e218 Original Diff: https://phabricator.dev.yugabyte.com/D13136 Test Plan: Jenkins: rebase: 2.6 For both of the below testing methods, remove the lines which set `--pgsql_proxy_bind_address` in `get_tserver_only_flags(...)` from the `yb-ctl` script to illustrate functionality in the lack of an explicit setting. **#8580** 1. Run `./bin/yb-ctl start --tserver_flags "vmodule=tablet=1` to start up local cluster with tserver logging extended 2. Run `./bin/ysqlsh --command 'create table t (i int)'; ./bin/ysqlsh --command 'create index on t (i)'` and observe result: ``` CREATE TABLE CREATE INDEX ``` after change and error: ``` ERROR: Aborted: backfill connection to DB failed: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/tmp/.yb.0/.s.PGSQL.5433"? ``` before diff **#9489** 1. Run `./bin/yb-ctl` start to start up local cluster 2. Open `http://127.0.0.1:9000/`, navigate to `Utilities`, and observe that `YSQL Live Ops` and `YSQL All Ops` route to valid hosts (`http://127.0.0.1:13000/(rpcz,statements)`) after change, vs invalid host (`http://:13000/(rpcz,statements)`) before change. **Exhibiting expected functionality even when `rpc_bind_addresses` is left unset (defaults to `0.0.0.0`)** 1. Start up master with the `rpc_bind_addresses` flag unset (which is set by default in `yb-ctl`): ``` ./build/latest/bin/yb-master --fs_data_dirs $PATH_TO_YUGABYTE_DATA/node-1/disk-1 --webserver_interface 127.0.0.1 --v 0 --version_file_json_path=$PATH_TO_YUGABYTE/build/debug-clang-dynamic-ninja --replication_factor=1 --yb_num_shards_per_tserver 2 --ysql_num_shards_per_tserver=2 --default_memory_limit_to_ram_ratio=0.35 --master_addresses 127.0.0.1:7100 --enable_ysql=true ``` 2. Start up the tserver without the `rpc_bind_addresses` flag set: ``` ./build/latest/bin/yb-tserver --fs_data_dirs $PATH_TO_YUGABYTE_DATA/node-1/disk-1 --webserver_interface 127.0.0.1 --v 0 --version_file_json_path= $PATH_TO_YUGABYTE/build/debug-clang-dynamic-ninja --tserver_master_addrs=127.0.0.1:7100 --yb_num_shards_per_tserver=2 --redis_proxy_bind_address=127.0.0.1:6379 --cql_proxy_bind_address=127.0.0.1:9042 --local_ip_for_outbound_sockets=127.0.0.1 --use_cassandra_authentication=false --ysql_num_shards_per_tserver=2 --default_memory_limit_to_ram_ratio=0.65 --enable_ysql=true --vmodule=tablet=1 ``` 3. Run `./bin/ysqlsh --command 'create table t (i int)'; ./bin/ysqlsh --command 'create index on t (i)'` and observe result: ``` CREATE TABLE CREATE INDEX ``` 4. Open `http://127.0.0.1:9000/`, navigate to `Utilities`, and observe that `YSQL Live Ops` and `YSQL All Ops` route to valid hosts (`http://127.0.0.1:13000/(rpcz,statements)`). Reviewers: sanketh, jason Reviewed By: jason Subscribers: yql, sanketh, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D13184
thepinetree
pushed a commit
that referenced
this issue
Oct 11, 2021
…ddress from rpc bind address Summary: #8580: Index backfill would fail if the `pgsql_proxy_bind_address` flag was unset due to an inconsistency between the default Postgres listen address and the default flag value read by the tablet server. #9489: Tablet server UI buttons for YSQL failed if the `pgsql_proxy_bind_address` flag was unset due to problematic treatment of empty string values and links would be set without host. Both issues are solved once the `pgsql_proxy_bind_address` receives a reasonable default value. Previously, when the gflag was unset, the default empty value would result in an empty host: "". In the former issue, the `PgProcessConf` instead had a separate default host of `0.0.0.0` in the case the flag was empty, while the tablet server treated the empty host as-is, leading to a discrepancy in index backfill. Similarly, when `pgsql_proxy_bind_address` defaulted to the empty host, the host for `GetDynamicUrlTile` was empty (with the appropriate port, in this case, 13000), but an empty host is notably not a wildcard address, causing URLs to be generated to `http://:13000`. This diff inherits a default value from the value of `rpc_bind_addresses` for `pgsql_proxy_bind_address`, as was done previously for `cql` and `redis` bind addresses. This new default flag value for `pgsql_proxy_bind_address` is then utilized both by the tablet server and `PgProcessConf` as the flag is defined both before the tablet server is initialized on `Init()` and before the Postgres process is initialized with `CreateValidateAndRunInitDb(...)`, solving the first issue. Additionally, the default host for `rpc_bind_addresses` is `0.0.0.0`, matching the Postgres value even if the `rpc_bind_addresses` gflag is unset. The new default flag value for `pgsql_proxy_bind_address` is also used in the `GetDynamicUrlTile` function, now with a valid host. Original Commit: 3f1e218 Original Diff: https://phabricator.dev.yugabyte.com/D13136 Test Plan: Jenkins: rebase: 2.4 For both of the below testing methods, remove the lines which set `--pgsql_proxy_bind_address` in `get_tserver_only_flags(...)` from the `yb-ctl` script to illustrate functionality in the lack of an explicit setting. **#8580** 1. Run `./bin/yb-ctl start --tserver_flags "vmodule=tablet=1` to start up local cluster with tserver logging extended 2. Run `./bin/ysqlsh --command 'create table t (i int)'; ./bin/ysqlsh --command 'create index on t (i)'` and observe result: ``` CREATE TABLE CREATE INDEX ``` after change and error: ``` ERROR: Aborted: backfill connection to DB failed: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/tmp/.yb.0/.s.PGSQL.5433"? ``` before diff **#9489** 1. Run `./bin/yb-ctl` start to start up local cluster 2. Open `http://127.0.0.1:9000/`, navigate to `Utilities`, and observe that `YSQL Live Ops` and `YSQL All Ops` route to valid hosts (`http://127.0.0.1:13000/(rpcz,statements)`) after change, vs invalid host (`http://:13000/(rpcz,statements)`) before change. **Exhibiting expected functionality even when `rpc_bind_addresses` is left unset (defaults to `0.0.0.0`)** 1. Start up master with the `rpc_bind_addresses` flag unset (which is set by default in `yb-ctl`): ``` ./build/latest/bin/yb-master --fs_data_dirs $PATH_TO_YUGABYTE_DATA/node-1/disk-1 --webserver_interface 127.0.0.1 --v 0 --version_file_json_path=$PATH_TO_YUGABYTE/build/debug-clang-dynamic-ninja --replication_factor=1 --yb_num_shards_per_tserver 2 --ysql_num_shards_per_tserver=2 --default_memory_limit_to_ram_ratio=0.35 --master_addresses 127.0.0.1:7100 --enable_ysql=true ``` 2. Start up the tserver without the `rpc_bind_addresses` flag set: ``` ./build/latest/bin/yb-tserver --fs_data_dirs $PATH_TO_YUGABYTE_DATA/node-1/disk-1 --webserver_interface 127.0.0.1 --v 0 --version_file_json_path= $PATH_TO_YUGABYTE/build/debug-clang-dynamic-ninja --tserver_master_addrs=127.0.0.1:7100 --yb_num_shards_per_tserver=2 --redis_proxy_bind_address=127.0.0.1:6379 --cql_proxy_bind_address=127.0.0.1:9042 --local_ip_for_outbound_sockets=127.0.0.1 --use_cassandra_authentication=false --ysql_num_shards_per_tserver=2 --default_memory_limit_to_ram_ratio=0.65 --enable_ysql=true --vmodule=tablet=1 ``` 3. Run `./bin/ysqlsh --command 'create table t (i int)'; ./bin/ysqlsh --command 'create index on t (i)'` and observe result: ``` CREATE TABLE CREATE INDEX ``` 4. Open `http://127.0.0.1:9000/`, navigate to `Utilities`, and observe that `YSQL Live Ops` and `YSQL All Ops` route to valid hosts (`http://127.0.0.1:13000/(rpcz,statements)`). Reviewers: sanketh, jason Reviewed By: jason Subscribers: yql, sanketh, bogdan Differential Revision: https://phabricator.dev.yugabyte.com/D13183
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
There are problems when yb-tserver flag
pgsql_proxy_bind_addressis not set (or is set to empty string or just spaces). One problem is discussed in issue #8580. There is a second one I found with some links in tserver web UI.When
FLAGS_pgsql_proxy_bind_addressis unset, the buttons don't work. The divs havehref="http://:13000/rpcz", so clicking them doesn't do anything. Use the first part of my repro in issue #8580 to easily reproduce this.If
pgsql_proxy_bind_addressis unset, you can assume postgres will listen on 0.0.0.0. I think usingrpc_bind_addressesis a safe choice, but I'm not too familiar. When using the above yb-ctl repro, I could use nearly any 127.0.0.0/8 address.As a side note, I don't think there are any other issues related to unset
pgsql_proxy_bind_addressbesides this and issue #8580, having grepped for pgsql_proxy_bind_address in src/yb.The text was updated successfully, but these errors were encountered: