Skip to content

2024.2.4.0-b82

@timothy-e timothy-e tagged this 27 Jun 13:28
Summary:
Original commit: abf33bac51c890d4bd2949630f09849cd6c62b1f / D44455

== Changes for Backport ==
JSON types weren't supported by libpq_utils until 4805e915e84d8c863d8e679f223e4992f56daec6 / D43156, which wasn't backported to 2024.2. Modify the utils to support JSON.
Also, the insert was failing due to a too large RPC error. Split it into batches.

== Original Summary ==

If multiple batched request are sent to multiple nodes in parallel, their results are combined into single response. This single response can exceed size limits.

There are two ways to fix it.
1. set the fetch size limit to ~ `rpc_max_message_size * max_buffer_size_to_rpc_limit_ratio / send_count`
2. modify the batching behaviour

If the request has no size limit (yb_fetch_size_limit = 0) or a size limit greater than the max allowed, set the size limit to `rpc_max_message_size * max_buffer_size_to_rpc_limit_ratio / send_count`.
Jira: DB-16688

Test Plan:
```lang=sql
CREATE TABLE tbl1(col0 TEXT) SPLIT INTO 10 TABLETS;
INSERT INTO tbl1 SELECT repeat('a', 30000) FROM generate_series(0, 30000);
ANALYZE tbl1; -- fails before this diff
```

```lang=sh
./yb_build.sh --cxx-test pg_tablet_split-test --gtest_filter PgManyTabletsSelect.AnalyzeTableWithLargeRows -n 20
```
Note that this test uses a separate test class because RPC_max_message_size is a non-runtime flag, and setting it at runtime can result in flaky failures as some sections of the code are aware of the new limit and other's aren't.

Reviewers: amartsinchyk

Reviewed By: amartsinchyk

Subscribers: yql

Differential Revision: https://phorge.dev.yugabyte.com/D44985
Assets 2
Loading