Issue 43666: Add skip_unavailable_shards as a setting for Distributed table. #57218

tntnatbry · 2023-11-25T02:44:12Z

This setting, when enabled (disabled by default), allows ClickHouse to silently skip unavailable shards of a Distributed table during a query execution, instead of throwing an exception to the client.

Changelog category:

Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add skip_unavailable_shards as a setting for Distributed tables that is similar to the corresponding query-level setting. Closes #43666.

CLAassistant · 2023-11-25T02:44:18Z

All committers have signed the CLA.

tntnatbry · 2023-11-25T03:00:39Z

Setup: 3-Shard (with 1 replica each) ClickHouse cluster in Docker.

On Shard 1:

d1da863ce311 :) select * from system.clusters;

SELECT *
FROM system.clusters

Query id: a9449f11-21ea-40d2-be64-0fcb062b8f04

┌─cluster────────────┬─shard_num─┬─shard_weight─┬─internal_replication─┬─replica_num─┬─host_name───┬─host_address──┬─port─┬─is_local─┬─user────┬─default_database─┬─errors_count─┬─slowdowns_count─┬─estimated_recovery_time─┬─database_shard_name─┬─database_replica_name─┬─is_active─┐
│ replicated_cluster │         1 │            1 │                    0 │           1 │ clickhouse1 │ 192.168.112.4 │ 9000 │        1 │ default │                  │            0 │               0 │                       0 │                     │                       │      ᴺᵁᴸᴸ │
│ replicated_cluster │         2 │            1 │                    0 │           1 │ clickhouse2 │ 192.168.112.3 │ 9000 │        0 │ default │                  │            0 │               0 │                       0 │                     │                       │      ᴺᵁᴸᴸ │
│ replicated_cluster │         3 │            1 │                    0 │           1 │ clickhouse3 │ 192.168.112.5 │ 9000 │        0 │ default │                  │            0 │               0 │                       0 │                     │                       │      ᴺᵁᴸᴸ │
└────────────────────┴───────────┴──────────────┴──────────────────────┴─────────────┴─────────────┴───────────────┴──────┴──────────┴─────────┴──────────────────┴──────────────┴─────────────────┴─────────────────────────┴─────────────────────┴───────────────────────┴───────────┘

3 rows in set. Elapsed: 0.035 sec.

d1da863ce311 :) show create table t1;

SHOW CREATE TABLE t1

Query id: ce5bdb07-c2a3-4bc1-968a-8d4ef46f0396

┌─statement────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ CREATE TABLE default.t1
(
    `ID` UInt32,
    `Name` String
)
ENGINE = MergeTree
ORDER BY ID
SETTINGS index_granularity = 8192 │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

1 row in set. Elapsed: 0.012 sec.

Add one row to t1 individually on each shard:
On Shard 1:

d1da863ce311 :) select * from t1;

SELECT *
FROM t1

Query id: 249ca169-1c6b-45e1-ace4-4cc33d5d90d3

┌─ID─┬─Name─┐
│  1 │ a    │
└────┴──────┘

1 row in set. Elapsed: 0.012 sec.

On Shard 2:

6377136da30b :) select * from t1;

SELECT *
FROM t1

Query id: 54deffa0-f52a-4ae5-9a54-03377775830f

┌─ID─┬─Name─┐
│  2 │ b    │
└────┴──────┘

1 row in set. Elapsed: 0.015 sec.

On Shard 3:

91dc18ab6188 :) select * from t1;

SELECT *
FROM t1

Query id: e0c54baa-9cc1-4e93-b591-c1d331233bbe

┌─ID─┬─Name─┐
│  3 │ c    │
└────┴──────┘

1 row in set. Elapsed: 0.011 sec.

Now, create the Distributed version of t1 on Shard 1:

d1da863ce311 :) CREATE TABLE t1_distributed
(
    ID UInt32,
    Name String
) 
ENGINE = Distributed('replicated_cluster', 'default', t1, rand());

CREATE TABLE t1_distributed
(
    `ID` UInt32,
    `Name` String
)
ENGINE = Distributed('replicated_cluster', 'default', t1, rand())

Query id: 8f1b8e11-7d63-4718-8b08-ea0e5af983b4

Ok.

0 rows in set. Elapsed: 0.009 sec.


d1da863ce311 :) select * from t1_distributed;

SELECT *
FROM t1_distributed

Query id: 96e99458-e47a-4065-931f-70a15df84540

┌─ID─┬─Name─┐
│  1 │ a    │
└────┴──────┘
┌─ID─┬─Name─┐
│  2 │ b    │
└────┴──────┘
┌─ID─┬─Name─┐
│  3 │ c    │
└────┴──────┘

3 rows in set. Elapsed: 0.052 sec.

As expected, all 3 rows are returned as all 3 shards are online.

Now, stop Shard 2. On Shard 1:

d1da863ce311 :) select * from t1_distributed;

SELECT *
FROM t1_distributed

Query id: 76f34040-4577-409c-8b2a-b9bbe7c413a0

┌─ID─┬─Name─┐
│  1 │ a    │
└────┴──────┘
┌─ID─┬─Name─┐
│  3 │ c    │
└────┴──────┘
↗ Progress: 2.00 rows, 28.00 B (3.67 rows/s., 51.36 B/s.)  99%
2 rows in set. Elapsed: 0.545 sec. 

Received exception from server (version 23.11.1):
Code: 279. DB::Exception: Received from localhost:9000. DB::NetException. DB::NetException: All connection tries failed. Log: 

Code: 32. DB::Exception: Attempt to read after eof. (ATTEMPT_TO_READ_AFTER_EOF) (version 23.11.1.1)
Code: 210. DB::NetException: Connection refused (clickhouse2:9000). (NETWORK_ERROR) (version 23.11.1.1)
Code: 210. DB::NetException: Connection refused (clickhouse2:9000). (NETWORK_ERROR) (version 23.11.1.1)

: While executing Remote. (ALL_CONNECTION_TRIES_FAILED)

As expected, exception is thrown to the client since Shard 2 is unreachable.

Now, re-create t1_distributed on Shard 1 with the new setting:

d1da863ce311 :) drop table t1_distributed;

DROP TABLE t1_distributed

Query id: f242a3cf-ab62-4eef-a2c5-07a7591178c6

Ok.

0 rows in set. Elapsed: 0.004 sec. 

d1da863ce311 :) CREATE TABLE t1_distributed
(
    ID UInt32,
    Name String
) 
ENGINE = Distributed('replicated_cluster', 'default', t1, rand())
SETTINGS skip_unavailable_shards=true;

CREATE TABLE t1_distributed
(
    `ID` UInt32,
    `Name` String
)
ENGINE = Distributed('replicated_cluster', 'default', t1, rand())
SETTINGS skip_unavailable_shards = 1

Query id: 14aebde8-f4bf-4631-b29e-2ae97da5ef36

Ok.

0 rows in set. Elapsed: 0.009 sec. 

d1da863ce311 :) select * from t1_distributed;

SELECT *
FROM t1_distributed

Query id: 217b2347-9924-4d1e-92fa-72150da484b1

┌─ID─┬─Name─┐
│  1 │ a    │
└────┴──────┘
┌─ID─┬─Name─┐
│  3 │ c    │
└────┴──────┘

2 rows in set. Elapsed: 0.476 sec.

As expected, ClickHouse now silently skips the unavailable Shard 2.

Confirm that the query-level setting overrides the table-level setting:

d1da863ce311 :) select * from t1_distributed SETTINGS skip_unavailable_shards=false;

SELECT *
FROM t1_distributed
SETTINGS skip_unavailable_shards = 0

Query id: 6b09dc96-9a4b-4670-b41f-1ead48998d96

┌─ID─┬─Name─┐
│  1 │ a    │
└────┴──────┘
┌─ID─┬─Name─┐
│  3 │ c    │
└────┴──────┘
← Progress: 2.00 rows, 28.00 B (2.31 rows/s., 32.39 B/s.)  99%
2 rows in set. Elapsed: 0.864 sec. 

Received exception from server (version 23.11.1):
Code: 279. DB::Exception: Received from localhost:9000. DB::NetException. DB::NetException: All connection tries failed. Log: 

Code: 210. DB::NetException: Connection refused (clickhouse2:9000). (NETWORK_ERROR) (version 23.11.1.1)
Code: 210. DB::NetException: Connection refused (clickhouse2:9000). (NETWORK_ERROR) (version 23.11.1.1)
Code: 210. DB::NetException: Connection refused (clickhouse2:9000). (NETWORK_ERROR) (version 23.11.1.1)

: While executing Remote. (ALL_CONNECTION_TRIES_FAILED)

I have also confirmed that the user-level setting also overrides the table-level setting as expected.

xiedeyantu · 2023-11-25T21:16:41Z

Why do we need to do this feature? It can lead to incorrect data.

alexey-milovidov · 2023-11-26T01:23:57Z

src/Storages/Distributed/DistributedSettings.h

@@ -17,6 +17,8 @@ class ASTStorage;
 #define LIST_OF_DISTRIBUTED_SETTINGS(M, ALIAS) \
    M(Bool, fsync_after_insert, false, "Do fsync for every inserted. Will decreases performance of inserts (only for background INSERT, i.e. distributed_foreground_insert=false)", 0) \
    M(Bool, fsync_directories, false, "Do fsync for temporary directory (that is used for background INSERT only) after all part operations (writes, renames, etc.).", 0) \
+    /** This is the distributed version of the skip_unavailable_shards setting available in src/Core/Settings.h */ \
+    M(Bool, skip_unavailable_shards, false, "If true, ClickHouse silently skips unavailable shards and nodes unresolvable through DNS. Shard is marked as unavailable when none of the replicas can be reached.", 0) \


The comment is ambiguous. It could be read as only DNS failures are skipped. But connection failures and even shards with non-existent tables are skipped as well.

If the original description has the same problem, it should be fixed as well.

I have updated the comment. Please take a look and let me know if the new description is OK, or if you need this further updated. I have also updated the comment in the original description in src/Core/Settings.h.

alexey-milovidov · 2023-11-26T01:25:11Z

Need to add a test. It can be a functional test using the test clusters. You can take a similar test for skip_unavailable_shards as an example.

alexey-milovidov · 2023-11-26T01:25:28Z

Missing changelog entry.

robot-ch-test-poll1 · 2023-11-26T01:26:05Z

This is an automated comment for commit bae58fe with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Successful checks

Check name	Description	Status
AST fuzzer	Runs randomly generated queries to catch program errors. The build type is optionally given in parenthesis. If it fails, ask a maintainer for help	✅ success
CI running	A meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR	✅ success
ClickHouse build check	Builds ClickHouse in various configurations for use in further steps. You have to fix the builds that fail. Build logs often has enough information to fix the error, but you might have to reproduce the failure locally. The cmake options can be found in the build log, grepping for cmake. Use these options and follow the general build process	✅ success
Compatibility check	Checks that clickhouse binary runs on distributions with old libc versions. If it fails, ask a maintainer for help	✅ success
Docker image for servers	The check to build and optionally push the mentioned image to docker hub	✅ success
Fast test	Normally this is the first check that is ran for a PR. It builds ClickHouse and runs most of stateless functional tests, omitting some. If it fails, further checks are not started until it is fixed. Look at the report to see which tests fail, then reproduce the failure locally as described here	✅ success
Flaky tests	Checks if new added or modified tests are flaky by running them repeatedly, in parallel, with more randomization. Functional tests are run 100 times with address sanitizer, and additional randomization of thread scheduling. Integrational tests are run up to 10 times. If at least once a new test has failed, or was too long, this check will be red. We don't allow flaky tests, read the doc	✅ success
Install packages	Checks that the built packages are installable in a clear environment	✅ success
Mergeable Check	Checks if all other necessary checks are successful	✅ success
Performance Comparison	Measure changes in query performance. The performance test report is described in detail here. In square brackets are the optional part/total tests	✅ success
Push to Dockerhub	The check for building and pushing the CI related docker images to docker hub	✅ success
SQLTest	There's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS	✅ success
SQLancer	Fuzzing tests that detect logical bugs with SQLancer tool	✅ success
Sqllogic	Run clickhouse on the sqllogic test set against sqlite and checks that all statements are passed	✅ success
Stateful tests	Runs stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	✅ success
Stateless tests	Runs stateless functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc	✅ success
Stress test	Runs stateless functional tests concurrently from several clients to detect concurrency-related errors	✅ success
Style Check	Runs a set of checks to keep the code style clean. If some of tests failed, see the related log from the report	✅ success
Unit tests	Runs the unit tests for different release types	✅ success
Upgrade check	Runs stress tests on server version from last release and then tries to upgrade it to the version from the PR. It checks if the new server can successfully startup without any errors, crashes or sanitizer asserts	✅ success

Check name	Description	Status
Integration tests	The integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests	❌ failure

tntnatbry · 2023-11-29T03:12:41Z

@alexey-milovidov I have added a functional test. Also added the changelog entry. Please review.

tntnatbry · 2023-12-05T19:48:31Z

It looks like the test I added has turned flaky: https://s3.amazonaws.com/clickhouse-test-reports/57218/946845a6425ea01514ca456af96447967029433d/stateless_tests_flaky_check__asan_.html

This is due to this line in the test case for testing the query-level override of the setting:

SELECT *, _shard_num FROM table_02916_distributed SETTINGS skip_unavailable_shards=0; -- { serverError ALL_CONNECTION_TRIES_FAILED }

I don't see any other tests in the testsuite which has ALL_CONNECTION_TRIES_FAILED as the -- {serverError}.

We have two options:

Add the long tag to the test and see if that helps.
Remove this line altogether from the test and skip testing the query-level override.

What do you think?

tntnatbry · 2023-12-05T19:56:20Z

I added the long tag to the test. Let's see how this looks now. Can you please re-trigger the CI checks?

… table. This setting, when enabled (disabled by default), allows ClickHouse to silently skip unavailable shards of a Distributed table during a query execution, instead of throwing an exception to the client.

tntnatbry · 2023-12-09T19:15:23Z

@alexey-milovidov The CI is now green, except for the Docs Check.

I had to comment out the query-level override test statement in the test file:

--SELECT *, _shard_num FROM table_02916_distributed SETTINGS skip_unavailable_shards=0; -- { serverError ALL_CONNECTION_TRIES_FAILED }

For some reason, if I remove this line instead of commenting out, the test turns flaky. It doesn't look like an issue with the code itself but it's probably a bug in the testing framework.

Please review.

alexey-milovidov · 2023-12-09T22:59:05Z

Ok. The docs check asks for adding documentation. I will change the category to "Imrovement" to not have this requirement.

alexey-milovidov · 2023-12-09T23:02:30Z

Flaky check also introduces random sleeps, and slowdowns the test many times, and then checks that it fits in 60 seconds. This could be the reason for what you see. Usually, we merge if the slow-down is still reasonable.

alexey-milovidov

Everything is perfect, let's check the CI run, and we can merge...

tntnatbry · 2023-12-10T19:53:56Z

@alexey-milovidov It looks like you updated the test case and uncommented this line:

--SELECT *, _shard_num FROM table_02916_distributed SETTINGS skip_unavailable_shards=0; -- { serverError ALL_CONNECTION_TRIES_FAILED }

The test has again turned flaky due to this change. If you read my previous comment, I had to comment out this line in the test to make it pass the CI. This is most likely an issue in the testing framework that needs a separate investigation/bug ticket.

You will have to revert your commit.

alexey-milovidov · 2023-12-10T20:06:58Z

"Flaky check" runs the test in parallel with itself; there might be a problem with that. I will check.

alexey-milovidov · 2023-12-10T20:07:38Z

But the test is, in fact, flaky: https://s3.amazonaws.com/clickhouse-test-reports/57218/d4e8c0558302b2a006cac17976558e2617ddd982/stateless_tests__asan__[1_4].html

alexey-milovidov · 2023-12-10T20:17:00Z

@tntnatbry We have settings randomization in tests to cover more edge cases in a randomized fashion.
The test is failing when --prefer_localhost_replica 1.

The reason - when it is set, and the address is localhost, the query goes to localhost without network communication, regardless of the address's port.

You can change the cluster definition for test_unavailable_shard to use something other than localhost or modify your test and SET prefer_localhost_replica = 0 to avoid randomization.

alexey-milovidov · 2023-12-10T20:19:05Z

Good article: https://clickhouse.com/blog/decorating-a-christmas-tree-with-the-help-of-flaky-tests

alexey-milovidov · 2023-12-10T22:09:33Z

Still, there is something.

tntnatbry · 2023-12-11T05:21:58Z

Reference file also needed to be updated. Just pushed another commit. Let's see how this looks now.

Thanks for sharing the article, almost that time of the year again!

tntnatbry · 2023-12-13T20:13:16Z

@alexey-milovidov Test is no longer flaky now. I needed to SET send_logs_level='fatal'; in the test case (similar to the original 00183_skip_unavailable_shards test). So we are good here. Please take a look.

I am now seeing an integration test failing: https://s3.amazonaws.com/clickhouse-test-reports/57218/bae58febf3b2573a745243ae5ed016aae13b3d49/integration_tests__release__[4_4].html

Out of these 2, only test_delayed_replica_failover is remotely related to this patch. If you look at the logs, the test is failing for the below query (which is not using the skip_unavailable_shards query override setting):

        # If we forbid stale replicas, the query must fail. But sometimes we must have bigger timeouts.
        for _ in range(20):
            try:
                instance_with_dist_table.query(
                    """
SELECT count() FROM distributed SETTINGS
    load_balancing='in_order',
    max_replica_delay_for_distributed_queries=1,
    fallback_to_stale_replicas_for_distributed_queries=0
"""
                )
                time.sleep(0.5)
            except:
                break
        else:
            raise Exception("Didn't raise when stale replicas are not allowed")

When I run this integration test locally on my machine, it is passing:

root@tntnatbry:/home/tntnatbry/git-projects/ClickHouse/tests/integration# ./runner 'test_delayed_replica_failover'
2023-12-13 14:56:19,783 [ 445878 ] INFO : ClickHouse root is not set. Will use /home/tntnatbry/git-projects/ClickHouse (runner:42, check_args_and_update_paths)
2023-12-13 14:56:19,784 [ 445878 ] INFO : Cases dir is not set. Will use /home/tntnatbry/git-projects/ClickHouse/tests/integration (runner:90, check_args_and_update_paths)
2023-12-13 14:56:19,784 [ 445878 ] INFO : utils dir is not set. Will use /home/tntnatbry/git-projects/ClickHouse/utils (runner:101, check_args_and_update_paths)
2023-12-13 14:56:19,784 [ 445878 ] INFO : base_configs_dir: /home/tntnatbry/git-projects/ClickHouse/programs/server, binary: /home/tntnatbry/git-projects/ClickHouse/build/programs/clickhouse-server, cases_dir: /home/tntnatbry/git-projects/ClickHouse/tests/integration  (runner:103, check_args_and_update_paths)
clickhouse_integration_tests_volume
Running pytest container as: 'docker run --net=host -it --rm --name clickhouse_integration_tests_1pn11o --privileged --dns-search='.' --volume=/home/tntnatbry/git-projects/ClickHouse/build/programs/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/tntnatbry/git-projects/ClickHouse/build/programs/clickhouse-server:/clickhouse --volume=/home/tntnatbry/git-projects/ClickHouse/build/programs/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/tntnatbry/git-projects/ClickHouse/programs/server:/clickhouse-config --volume=/home/tntnatbry/git-projects/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/tntnatbry/git-projects/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/tntnatbry/git-projects/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker   -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600  -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS=" test_delayed_replica_failover   -vvv" clickhouse/integration-tests-runner:latest '.
Start tests
========================================================================================================================================================================================================================================== test session starts ==========================================================================================================================================================================================================================================
platform linux -- Python 3.10.12, pytest-7.4.3, pluggy-1.3.0 -- /usr/bin/python3
cachedir: .pytest_cache
rootdir: /ClickHouse/tests/integration
configfile: pytest.ini
plugins: xdist-3.5.0, repeat-0.9.3, order-1.0.0, random-0.2, timeout-2.2.0
timeout: 900.0s
timeout method: signal
timeout func_only: False
collected 1 item                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        

test_delayed_replica_failover/test.py::test PASSED                                                                                                                                                                                                                                                                                                                                                                                                                                                [100%]

===================================================================================================================================================================================================================================== 1 passed in 63.37s (0:01:03) ======================================================================================================================================================================================================================================

Do you know what the issue here is?

tntnatbry mentioned this pull request Nov 25, 2023

Add skip_unavailable_shards as a setting for Distributed table. #43666

Closed

alexey-milovidov added the can be tested Allows running workflows for external contributors label Nov 26, 2023

alexey-milovidov reviewed Nov 26, 2023

View reviewed changes

alexey-milovidov removed the can be tested Allows running workflows for external contributors label Nov 26, 2023

alexey-milovidov self-assigned this Nov 26, 2023

robot-ch-test-poll1 added the pr-feature Pull request with new product feature label Nov 26, 2023

tntnatbry force-pushed the issue-43666 branch from 58a8700 to 946845a Compare November 29, 2023 03:01

alexey-milovidov added the can be tested Allows running workflows for external contributors label Dec 4, 2023

tntnatbry force-pushed the issue-43666 branch from 946845a to 932327a Compare December 5, 2023 19:50

tntnatbry force-pushed the issue-43666 branch from 932327a to 3ba3c93 Compare December 7, 2023 02:47

Issue 43666: Add skip_unavailable_shards as a setting for Distributed…

e547db0

… table. This setting, when enabled (disabled by default), allows ClickHouse to silently skip unavailable shards of a Distributed table during a query execution, instead of throwing an exception to the client.

tntnatbry force-pushed the issue-43666 branch from 3ba3c93 to e547db0 Compare December 8, 2023 20:45

Update 02916_distributed_skip_unavailable_shards.sql

d4e8c05

alexey-milovidov approved these changes Dec 10, 2023

View reviewed changes

robot-ch-test-poll added pr-improvement Pull request with some product improvements and removed pr-feature Pull request with new product feature labels Dec 10, 2023

Update 02916_distributed_skip_unavailable_shards.sql

ec05335

Update 02916_distributed_skip_unavailable_shards.reference.

229a044

Update 02916_distributed_skip_unavailable_shards.sql/.reference files.

bae58fe

alexey-milovidov merged commit 58396c5 into ClickHouse:master Dec 18, 2023
335 of 337 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 43666: Add skip_unavailable_shards as a setting for Distributed table. #57218

Issue 43666: Add skip_unavailable_shards as a setting for Distributed table. #57218

tntnatbry commented Nov 25, 2023 •

edited by alexey-milovidov

CLAassistant commented Nov 25, 2023 •

edited

tntnatbry commented Nov 25, 2023

xiedeyantu commented Nov 25, 2023

alexey-milovidov Nov 26, 2023

alexey-milovidov Nov 26, 2023

tntnatbry Nov 29, 2023

alexey-milovidov commented Nov 26, 2023

alexey-milovidov commented Nov 26, 2023

robot-ch-test-poll1 commented Nov 26, 2023 •

edited by robot-ch-test-poll2

tntnatbry commented Nov 29, 2023

tntnatbry commented Dec 5, 2023

tntnatbry commented Dec 5, 2023

tntnatbry commented Dec 9, 2023

alexey-milovidov commented Dec 9, 2023

alexey-milovidov commented Dec 9, 2023

alexey-milovidov left a comment

tntnatbry commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

tntnatbry commented Dec 11, 2023

tntnatbry commented Dec 13, 2023

Issue 43666: Add skip_unavailable_shards as a setting for Distributed table. #57218

Issue 43666: Add skip_unavailable_shards as a setting for Distributed table. #57218

Conversation

tntnatbry commented Nov 25, 2023 • edited by alexey-milovidov

Changelog category:

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

CLAassistant commented Nov 25, 2023 • edited

tntnatbry commented Nov 25, 2023

xiedeyantu commented Nov 25, 2023

alexey-milovidov Nov 26, 2023

Choose a reason for hiding this comment

alexey-milovidov Nov 26, 2023

Choose a reason for hiding this comment

tntnatbry Nov 29, 2023

Choose a reason for hiding this comment

alexey-milovidov commented Nov 26, 2023

alexey-milovidov commented Nov 26, 2023

robot-ch-test-poll1 commented Nov 26, 2023 • edited by robot-ch-test-poll2

tntnatbry commented Nov 29, 2023

tntnatbry commented Dec 5, 2023

tntnatbry commented Dec 5, 2023

tntnatbry commented Dec 9, 2023

alexey-milovidov commented Dec 9, 2023

alexey-milovidov commented Dec 9, 2023

alexey-milovidov left a comment

Choose a reason for hiding this comment

tntnatbry commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

alexey-milovidov commented Dec 10, 2023

tntnatbry commented Dec 11, 2023

tntnatbry commented Dec 13, 2023

tntnatbry commented Nov 25, 2023 •

edited by alexey-milovidov

CLAassistant commented Nov 25, 2023 •

edited

robot-ch-test-poll1 commented Nov 26, 2023 •

edited by robot-ch-test-poll2