YQ-3157 batch script result fetch by size #4219

GrigoriyPA · 2024-05-02T09:07:40Z

Changelog entry

Added script result batching by size

Changelog category

Improvement

Additional information

Added script result batching by size
Fixed forget operation, update ttl instead of delete

github-actions · 2024-05-02T09:11:04Z

⚪ 2024-05-02 09:11:03 UTC Pre-commit check for f82ad85 has started.
⚪ 2024-05-02 09:11:07 UTC Build linux-x86_64-release-clang14 is running...
⚫ 2024-05-02 09:30:58 UTC Check cancelled

github-actions · 2024-05-02T09:11:05Z

⚪ 2024-05-02 09:11:05 UTC Pre-commit check for f82ad85 has started.
⚪ 2024-05-02 09:11:08 UTC Build linux-x86_64-relwithdebinfo is running...
⚫ 2024-05-02 09:30:58 UTC Check cancelled

github-actions · 2024-05-02T09:11:11Z

⚪ 2024-05-02 09:11:11 UTC Pre-commit check for f82ad85 has started.
⚪ 2024-05-02 09:11:14 UTC Build linux-x86_64-release-asan is running...
⚫ 2024-05-02 09:30:58 UTC Check cancelled

github-actions · 2024-05-02T09:32:11Z

⚪ 2024-05-02 09:32:11 UTC Pre-commit check for 8bff3db has started.
⚪ 2024-05-02 09:32:12 UTC Build linux-x86_64-release-asan is running...
🔴 2024-05-02 09:53:30 UTC Build failed. see the build logs.
🔴 2024-05-02 09:55:10 UTC Tests run skipped.

github-actions · 2024-05-02T09:32:12Z

⚪ 2024-05-02 09:32:11 UTC Pre-commit check for 8bff3db has started.
⚪ 2024-05-02 09:32:13 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-05-02 09:47:15 UTC Build failed. see the build logs.

github-actions · 2024-05-02T09:32:33Z

⚪ 2024-05-02 09:32:33 UTC Pre-commit check for 8bff3db has started.
⚪ 2024-05-02 09:32:34 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-05-02 09:50:41 UTC Build failed. see the build logs.
🔴 2024-05-02 09:52:03 UTC Tests run skipped.

github-actions · 2024-05-02T10:40:46Z

⚪ 2024-05-02 10:40:46 UTC Pre-commit check for 4b34fa0 has started.
⚪ 2024-05-02 10:40:48 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-02 11:16:14 UTC Build successful.

github-actions · 2024-05-02T10:41:01Z

⚪ 2024-05-02 10:41:01 UTC Pre-commit check for 4b34fa0 has started.
⚪ 2024-05-02 10:41:03 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-02 11:20:37 UTC Build successful.
⚪ 2024-05-02 11:22:27 UTC Tests are running...
🔴 2024-05-02 13:11:27 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10275	10107	0	96	56	16

github-actions · 2024-05-02T10:41:03Z

⚪ 2024-05-02 10:41:02 UTC Pre-commit check for 4b34fa0 has started.
⚪ 2024-05-02 10:41:05 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-02 11:18:58 UTC Build successful.
⚪ 2024-05-02 11:20:46 UTC Tests are running...
🔴 2024-05-02 12:57:46 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
37537	32038	0	254	5232	13

github-actions · 2024-05-03T19:13:16Z

⚪ 2024-05-03 19:13:16 UTC Pre-commit check for 3c7f6a3 has started.
⚪ 2024-05-03 19:13:18 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-03 19:53:55 UTC Build successful.

github-actions · 2024-05-03T19:13:43Z

⚪ 2024-05-03 19:13:42 UTC Pre-commit check for 3c7f6a3 has started.
⚪ 2024-05-03 19:13:45 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-03 19:52:54 UTC Build successful.
⚪ 2024-05-03 19:54:46 UTC Tests are running...
🔴 2024-05-03 21:42:26 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10299	10166	0	27	98	8

github-actions · 2024-05-03T19:15:33Z

⚪ 2024-05-03 19:15:33 UTC Pre-commit check for 3c7f6a3 has started.
⚪ 2024-05-03 19:15:35 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-03 19:51:42 UTC Build successful.
⚪ 2024-05-03 19:53:28 UTC Tests are running...
🔴 2024-05-03 21:24:49 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
37566	32306	0	5	5235	20

github-actions · 2024-05-05T08:49:01Z

⚪ 2024-05-05 08:49:01 UTC Pre-commit check for 5640c3d has started.
⚪ 2024-05-05 08:49:04 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-05 09:25:02 UTC Build successful.

github-actions · 2024-05-05T08:49:04Z

⚪ 2024-05-05 08:49:03 UTC Pre-commit check for 5640c3d has started.
⚪ 2024-05-05 08:49:06 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-05 09:27:58 UTC Build successful.
⚪ 2024-05-05 09:29:53 UTC Tests are running...
🔴 2024-05-05 11:10:38 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
37571	32318	0	4	5232	17

github-actions · 2024-05-05T08:49:25Z

⚪ 2024-05-05 08:49:25 UTC Pre-commit check for 5640c3d has started.
⚪ 2024-05-05 08:49:28 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-05 09:28:56 UTC Build successful.
⚪ 2024-05-05 09:30:58 UTC Tests are running...
🔴 2024-05-05 11:17:57 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10298	10167	0	27	94	10

uzhastik · 2024-05-14T15:00:18Z

ydb/core/kqp/ut/federated_query/s3/kqp_federated_query_ut.cpp

@@ -1768,10 +1765,10 @@ Y_UNIT_TEST_SUITE(KqpFederatedQuery) {

            if (status.GetStatus() == NYdb::EStatus::CLIENT_DEADLINE_EXCEEDED) {
                // Wait until last forget is not finished
-                Sleep(TDuration::Seconds(1));
+                Sleep(TDuration::Seconds(30));


давай оставим 1 сек. оно же не потребляет ресурсы, зато тест шустрее может выйти

Так при локальной сборке типа debug тест перестаёт работать, так как Forget не успевает завершиться за 6 секунд и идут TLI

github-actions · 2024-05-21T13:16:36Z

⚪ 2024-05-21 13:16:35 UTC Pre-commit check for 231c27a has started.
⚪ 2024-05-21 13:16:37 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-21 13:52:52 UTC Build successful.
⚪ 2024-05-21 13:54:33 UTC Tests are running...
🔴 2024-05-21 15:22:58 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38118	33320	0	5	4785	8

github-actions · 2024-05-21T13:17:15Z

⚪ 2024-05-21 13:17:15 UTC Pre-commit check for 231c27a has started.
⚪ 2024-05-21 13:17:17 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-21 13:52:47 UTC Build successful.

github-actions · 2024-05-21T13:23:09Z

⚪ 2024-05-21 13:23:08 UTC Pre-commit check for 231c27a has started.
⚪ 2024-05-21 13:23:10 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-21 13:59:30 UTC Build successful.
⚪ 2024-05-21 14:01:08 UTC Tests are running...
🔴 2024-05-21 15:52:16 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10659	10587	0	28	28	16

github-actions · 2024-05-23T15:52:09Z

⚪ 2024-05-23 15:52:08 UTC Pre-commit check for e03355f has started.
⚪ 2024-05-23 15:54:45 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-23 15:56:24 UTC Build successful.
⚪ 2024-05-23 15:56:36 UTC Tests are running...
🔴 2024-05-23 17:53:20 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
10839	10744	0	42	39	14

github-actions · 2024-05-23T15:52:46Z

⚪ 2024-05-23 15:52:46 UTC Pre-commit check for e03355f has started.
⚪ 2024-05-23 15:55:21 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-23 15:56:51 UTC Build successful.

github-actions · 2024-05-23T15:57:03Z

⚪ 2024-05-23 15:57:03 UTC Pre-commit check for e03355f has started.
⚪ 2024-05-23 16:00:06 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-23 16:02:21 UTC Build successful.
⚪ 2024-05-23 16:02:32 UTC Tests are running...
🔴 2024-05-23 17:39:29 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
38311	33503	0	2	4796	10

GrigoriyPA requested a review from a team as a code owner May 2, 2024 09:07

github-actions bot added the new-feature label May 2, 2024

GrigoriyPA requested review from Hor911 and dorooleg May 2, 2024 09:09

github-actions bot added the improvement label May 2, 2024

uzhastik reviewed May 14, 2024

View reviewed changes

GrigoriyPA added 5 commits May 20, 2024 10:41

Added results batching by size

e065b95

Fixed build

650194e

Added unit test

0e76fd0

Fixed forget operation

9470575

Fixed result writer actor

47a56d2

GrigoriyPA force-pushed the YQ-3157-batch-script-result-fetch-by-size branch from 23682ef to 47a56d2 Compare May 21, 2024 13:14

GrigoriyPA dismissed Hor911’s stale review via aea4a05 May 23, 2024 15:49

dorooleg approved these changes May 23, 2024

View reviewed changes

GrigoriyPA merged commit 69d3866 into ydb-platform:main May 24, 2024
3 of 5 checks passed

GrigoriyPA deleted the YQ-3157-batch-script-result-fetch-by-size branch May 24, 2024 13:09

This was referenced May 27, 2024

Cli changes for autoscaling #4871

Closed

Fix getting mid of range for autoscaling #4884

Closed

Fix getting mid range #4885

Merged

Autoscaling logs #4976

Merged

This was referenced May 29, 2024

YQ-3292 fixed kqprun ResourceBrokerConfig passing #4994

Merged

Added ExecutionId for script execution logs #5014

Merged

niksaveliev mentioned this pull request May 30, 2024

Forbid autoscaling with size retention #5054

Merged

StekPerepolnen mentioned this pull request May 31, 2024

vdisk evict #5093

Merged

This was referenced Jun 6, 2024

Mute split by load ut #5264

Closed

Mute split by load ut #5267

Closed

Mute split by load ut #5272

Merged

Fix build #5310

Merged

serbel324 mentioned this pull request Jun 7, 2024

Don't mark slow disks if there is only one #5340

Merged

niksaveliev mentioned this pull request Jun 10, 2024

Fix split by load ut #5383

Merged

This was referenced Jun 11, 2024

metadata cache keeps merged issues #5461

Merged

json_hc metadata cache #5465

Merged

json tenantinfo status metadata cache #5543

Merged

json query transaction mode #5574

Merged

niksaveliev mentioned this pull request Jun 17, 2024

Fix win build #5622

Closed

This was referenced Jun 19, 2024

hc api doc #5703

Merged

move time difference issue under database level #5859

Merged

metadata requests flags fix #5860

Merged

StekPerepolnen mentioned this pull request Jun 28, 2024

oidc-proxy new iam 2 #6059

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YQ-3157 batch script result fetch by size #4219

YQ-3157 batch script result fetch by size #4219

GrigoriyPA commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 5, 2024 •

edited

Loading

github-actions bot commented May 5, 2024 •

edited

Loading

github-actions bot commented May 5, 2024 •

edited

Loading

uzhastik May 14, 2024

GrigoriyPA May 14, 2024

github-actions bot commented May 21, 2024 •

edited

Loading

github-actions bot commented May 21, 2024 •

edited

Loading

github-actions bot commented May 21, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

YQ-3157 batch script result fetch by size #4219

YQ-3157 batch script result fetch by size #4219

Conversation

GrigoriyPA commented May 2, 2024 • edited Loading

Changelog entry

Changelog category

Additional information

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 2, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 3, 2024 • edited Loading

github-actions bot commented May 5, 2024 • edited Loading

github-actions bot commented May 5, 2024 • edited Loading

github-actions bot commented May 5, 2024 • edited Loading

uzhastik May 14, 2024

Choose a reason for hiding this comment

GrigoriyPA May 14, 2024

Choose a reason for hiding this comment

github-actions bot commented May 21, 2024 • edited Loading

github-actions bot commented May 21, 2024 • edited Loading

github-actions bot commented May 21, 2024 • edited Loading

github-actions bot commented May 23, 2024 • edited Loading

github-actions bot commented May 23, 2024 • edited Loading

github-actions bot commented May 23, 2024 • edited Loading

GrigoriyPA commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 2, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 3, 2024 •

edited

Loading

github-actions bot commented May 5, 2024 •

edited

Loading

github-actions bot commented May 5, 2024 •

edited

Loading

github-actions bot commented May 5, 2024 •

edited

Loading

github-actions bot commented May 21, 2024 •

edited

Loading

github-actions bot commented May 21, 2024 •

edited

Loading

github-actions bot commented May 21, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading

github-actions bot commented May 23, 2024 •

edited

Loading