Yql-18391 gracejoin refactoring for spilling #4559

lll-phill-lll · 2024-05-15T14:02:07Z

Changelog entry

Small refactoring that makes its easier to add spilling to grace join

Changelog category

Not for changelog (changelog entry is not required)

Additional information

...

github-actions · 2024-05-15T14:03:28Z

⚪ 2024-05-15 14:03:27 UTC Pre-commit check for 594b374 has started.
⚪ 2024-05-15 14:03:29 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-15 14:07:20 UTC Build successful.

github-actions · 2024-05-15T14:04:32Z

⚪ 2024-05-15 14:04:32 UTC Pre-commit check for 594b374 has started.
⚪ 2024-05-15 14:04:33 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-15 14:07:43 UTC Build successful.
⚪ 2024-05-15 14:09:21 UTC Tests are running...
🔴 2024-05-15 16:05:32 UTC Test run completed, no test results found for commit 35a1f44. Please check build logs.
⚫ 2024-05-15 16:05:35 UTC Check cancelled

github-actions · 2024-05-15T14:05:26Z

⚪ 2024-05-15 14:05:26 UTC Pre-commit check for 594b374 has started.
⚪ 2024-05-15 14:05:28 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-15 14:08:12 UTC Build successful.
⚪ 2024-05-15 14:10:06 UTC Tests are running...
🔴 2024-05-15 15:52:45 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11698	11583	0	37	69	9

github-actions · 2024-05-15T16:09:04Z

⚪ 2024-05-15 16:09:04 UTC Pre-commit check for 36d3763 has started.
⚪ 2024-05-15 16:09:06 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-15 16:11:54 UTC Build successful.
⚪ 2024-05-15 16:13:52 UTC Tests are running...
🔴 2024-05-15 16:47:43 UTC Test run completed, no test results found for commit a77f3f4. Please check build logs.
⚫ 2024-05-15 16:47:46 UTC Check cancelled

github-actions · 2024-05-15T16:09:09Z

⚪ 2024-05-15 16:09:09 UTC Pre-commit check for 36d3763 has started.
⚪ 2024-05-15 16:09:11 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-15 16:12:35 UTC Build successful.

github-actions · 2024-05-15T16:09:18Z

⚪ 2024-05-15 16:09:18 UTC Pre-commit check for 36d3763 has started.
⚪ 2024-05-15 16:09:20 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-15 16:12:34 UTC Build successful.
⚪ 2024-05-15 16:14:27 UTC Tests are running...
🔴 2024-05-15 16:47:55 UTC Test run completed, no test results found for commit a77f3f4. Please check build logs.
⚫ 2024-05-15 16:47:59 UTC Check cancelled

github-actions · 2024-05-15T16:49:18Z

⚪ 2024-05-15 16:49:18 UTC Pre-commit check for 4bfdde1 has started.
⚪ 2024-05-15 16:49:19 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-15 16:52:58 UTC Build successful.

github-actions · 2024-05-15T16:51:28Z

⚪ 2024-05-15 16:51:28 UTC Pre-commit check for 4bfdde1 has started.
⚪ 2024-05-15 16:51:31 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-15 16:54:22 UTC Build successful.
⚪ 2024-05-15 16:56:20 UTC Tests are running...
🔴 2024-05-15 18:42:47 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11699	11601	0	33	56	9

github-actions · 2024-05-15T16:52:34Z

⚪ 2024-05-15 16:52:34 UTC Pre-commit check for 4bfdde1 has started.
⚪ 2024-05-15 16:52:37 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-15 16:55:55 UTC Build successful.
⚪ 2024-05-15 16:57:52 UTC Tests are running...
🔴 2024-05-15 18:57:02 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
70526	57661	0	203	12654	8

github-actions · 2024-05-16T08:30:53Z

⚪ 2024-05-16 08:30:53 UTC Pre-commit check for 1dbdfcb has started.
⚪ 2024-05-16 08:30:55 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-16 08:34:17 UTC Build successful.

github-actions · 2024-05-16T08:30:54Z

⚪ 2024-05-16 08:30:54 UTC Pre-commit check for 1dbdfcb has started.
⚪ 2024-05-16 08:30:57 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-16 08:33:50 UTC Build successful.
⚪ 2024-05-16 08:35:51 UTC Tests are running...
🔴 2024-05-16 10:23:08 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11699	11617	0	29	44	9

github-actions · 2024-05-16T08:31:02Z

⚪ 2024-05-16 08:31:02 UTC Pre-commit check for 1dbdfcb has started.
⚪ 2024-05-16 08:31:04 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-16 08:34:22 UTC Build successful.
⚪ 2024-05-16 08:36:17 UTC Tests are running...
🔴 2024-05-16 10:33:37 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
70530	57654	0	212	12654	10

github-actions · 2024-05-16T13:32:54Z

⚪ 2024-05-16 13:32:54 UTC Pre-commit check for 309b8b7 has started.
⚪ 2024-05-16 13:32:56 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-16 13:35:51 UTC Build successful.
⚪ 2024-05-16 13:37:45 UTC Tests are running...
🔴 2024-05-16 15:21:39 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11710	11463	0	47	193	7

lll-phill-lll · 2024-05-16T13:34:38Z

ydb/library/yql/minikql/comp_nodes/mkql_grace_join_imp.cpp


    if (NumberOfStringColumns || NumberOfIColumns ) {
-        stringsOffsets.push_back(offset); // Adding offset to tuple in keyIntVals vector
-        stringsOffsets.push_back(stringVals.size());  // Adding offset to string values


these 2 sizes will be less than full vector size because of spilling. Vectors partly will be spilled.

github-actions · 2024-05-16T13:52:48Z

⚪ 2024-05-16 13:52:48 UTC Pre-commit check for 309b8b7 has started.
⚪ 2024-05-16 13:52:50 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-16 13:55:56 UTC Build successful.

github-actions · 2024-05-16T13:55:51Z

⚪ 2024-05-16 13:55:50 UTC Pre-commit check for 309b8b7 has started.
⚪ 2024-05-16 13:55:52 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-16 13:58:45 UTC Build successful.
⚪ 2024-05-16 14:00:24 UTC Tests are running...
🔴 2024-05-16 15:47:30 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
70532	57646	0	208	12671	7

github-actions · 2024-05-17T06:22:18Z

⚪ 2024-05-17 06:22:18 UTC Pre-commit check for 9bd5332 has started.
⚪ 2024-05-17 06:22:21 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-17 06:58:17 UTC Build successful.

github-actions · 2024-05-17T06:22:22Z

⚪ 2024-05-17 10:25:49 UTC Pre-commit check for 9bd5332 has started.
⚪ 2024-05-17 10:25:53 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-17 10:27:07 UTC Build successful.
⚪ 2024-05-17 10:29:07 UTC Tests are running...
🔴 2024-05-17 11:25:46 UTC Test run completed, no test results found for commit 64cc097. Please check build logs.
⚫ 2024-05-17 11:25:49 UTC Check cancelled

github-actions · 2024-05-17T06:22:24Z

⚪ 2024-05-17 10:25:52 UTC Pre-commit check for 9bd5332 has started.
⚪ 2024-05-17 10:25:55 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-17 10:27:08 UTC Build successful.
⚪ 2024-05-17 10:29:06 UTC Tests are running...
🔴 2024-05-17 11:25:45 UTC Test run completed, no test results found for commit 64cc097. Please check build logs.
⚫ 2024-05-17 11:25:49 UTC Check cancelled

ydb/library/yql/minikql/comp_nodes/mkql_grace_join_imp.cpp

github-actions · 2024-05-17T11:27:10Z

⚪ 2024-05-17 11:27:09 UTC Pre-commit check for 33fa0a3 has started.
⚪ 2024-05-17 11:27:11 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-17 11:39:17 UTC Build successful.
⚪ 2024-05-17 11:41:03 UTC Tests are running...
🔴 2024-05-17 13:42:54 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11757	11413	0	65	267	12

github-actions · 2024-05-17T11:30:42Z

✅ Documentation build

Revision built successfully
Revision preview link

github-actions · 2024-05-17T11:35:05Z

⚪ 2024-05-17 11:35:05 UTC Pre-commit check for 33fa0a3 has started.
⚪ 2024-05-17 11:35:06 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-17 11:48:05 UTC Build successful.

github-actions · 2024-05-17T11:38:58Z

⚪ 2024-05-17 11:38:57 UTC Pre-commit check for 33fa0a3 has started.
⚪ 2024-05-17 11:38:59 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-17 11:42:00 UTC Build successful.
⚪ 2024-05-17 11:43:44 UTC Tests are running...
🔴 2024-05-17 13:37:10 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
70573	57871	0	12	12681	9

github-actions · 2024-05-17T17:20:37Z

⚪ 2024-05-17 17:20:37 UTC Pre-commit check for ca9ac8b has started.
⚪ 2024-05-17 17:20:40 UTC Build linux-x86_64-release-asan is running...
🟢 2024-05-17 17:21:49 UTC Build successful.
⚪ 2024-05-17 17:23:36 UTC Tests are running...
🔴 2024-05-17 19:20:40 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
11767	11506	0	45	192	24

github-actions · 2024-05-17T17:20:48Z

⚪ 2024-05-17 17:20:48 UTC Pre-commit check for ca9ac8b has started.
⚪ 2024-05-17 17:20:50 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-05-17 17:22:00 UTC Build successful.

github-actions · 2024-05-17T17:22:40Z

⚪ 2024-05-17 17:22:39 UTC Pre-commit check for ca9ac8b has started.
⚪ 2024-05-17 17:22:41 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-05-17 17:23:40 UTC Build successful.
⚪ 2024-05-17 17:25:15 UTC Tests are running...
🔴 2024-05-17 19:14:34 UTC Some tests failed, follow the links below.

Test history

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
70603	57922	0	2	12670	9

lll-phill-lll requested a review from a team as a code owner May 15, 2024 14:02

github-actions bot added the not-for-changelog label May 15, 2024

lll-phill-lll force-pushed the YQL-18391-gracejoin-refactoring-for-spilling branch from badd59a to 994685b Compare May 16, 2024 08:27

lll-phill-lll commented May 16, 2024

View reviewed changes

lll-phill-lll requested a review from Darych May 16, 2024 13:59

Darych reviewed May 17, 2024

View reviewed changes

ydb/library/yql/minikql/comp_nodes/mkql_grace_join_imp.cpp Show resolved Hide resolved

Darych previously approved these changes May 17, 2024

View reviewed changes

lll-phill-lll force-pushed the YQL-18391-gracejoin-refactoring-for-spilling branch from 64cc097 to 2a7a3a8 Compare May 17, 2024 11:25

lll-phill-lll added 9 commits May 17, 2024 20:17

fixed tests

9694e14

YQL-18391 added spilling for grace join

e000420

use global vectors sizes as offsets

488e00c

fixup

0efb3c6

fixup

eaf2dda

fixup

0d2d4e6

fixup

fb029e8

fixup

a52bb64

fixup

74f7d9b

lll-phill-lll force-pushed the YQL-18391-gracejoin-refactoring-for-spilling branch from 78199b2 to 74f7d9b Compare May 17, 2024 17:17

Darych approved these changes May 17, 2024

View reviewed changes

lll-phill-lll merged commit 70713cc into ydb-platform:main May 18, 2024
3 of 5 checks passed

serbel324 mentioned this pull request May 24, 2024

Add description of new ICB controls for CMS #4818

Merged

This was referenced May 27, 2024

Cli changes for autoscaling #4871

Closed

Fix getting mid of range for autoscaling #4884

Closed

Fix getting mid range #4885

Merged

MrLolthe1st pushed a commit to MrLolthe1st/ydb that referenced this pull request May 28, 2024

Yql-18391 gracejoin refactoring for spilling (ydb-platform#4559)

ac2bc9f

niksaveliev mentioned this pull request May 29, 2024

Autoscaling logs #4976

Merged

This was referenced May 29, 2024

YQ-3292 fixed kqprun ResourceBrokerConfig passing #4994

Merged

Added ExecutionId for script execution logs #5014

Merged

This was referenced May 30, 2024

Forbid autoscaling with size retention #5054

Merged

Mute split by load ut #5264

Closed

Mute split by load ut #5267

Closed

Mute split by load ut #5272

Merged

This was referenced Jun 7, 2024

Fix build #5310

Merged

Fix split by load ut #5383

Merged

niksaveliev mentioned this pull request Jun 17, 2024

Fix win build #5622

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Yql-18391 gracejoin refactoring for spilling #4559

Yql-18391 gracejoin refactoring for spilling #4559

lll-phill-lll commented May 15, 2024

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

lll-phill-lll May 16, 2024

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

Yql-18391 gracejoin refactoring for spilling #4559

Yql-18391 gracejoin refactoring for spilling #4559

Conversation

lll-phill-lll commented May 15, 2024

Changelog entry

Changelog category

Additional information

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 15, 2024 • edited Loading

github-actions bot commented May 16, 2024 • edited Loading

github-actions bot commented May 16, 2024 • edited Loading

github-actions bot commented May 16, 2024 • edited Loading

github-actions bot commented May 16, 2024 • edited Loading

lll-phill-lll May 16, 2024

Choose a reason for hiding this comment

github-actions bot commented May 16, 2024 • edited Loading

github-actions bot commented May 16, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024

✅ Documentation build

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 17, 2024 • edited Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 15, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 16, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading

github-actions bot commented May 17, 2024 •

edited

Loading