Support fulltext index on SchemeShard #23739

kunga · 2025-08-28T16:01:51Z

Changelog entry

...

Changelog category

Not for changelog (changelog entry is not required)

Description for reviewers

Not only add a new index type but also switch-case index type so the next one can be also easily added

github-actions · 2025-08-28T16:05:20Z

⚪ 2025-08-28 16:05:19 UTC Pre-commit check linux-x86_64-release-asan for 9044a54 has started.
⚪ 2025-08-28 16:05:34 UTC Artifacts will be uploaded here
⚪ 2025-08-28 16:09:26 UTC ya make is running...
🟡 2025-08-28 18:59:12 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17819	17122	0	300	356	41

🟢 2025-08-28 19:00:50 UTC Build successful.

github-actions · 2025-08-28T16:07:03Z

⚪ 2025-08-28 16:07:02 UTC Pre-commit check linux-x86_64-relwithdebinfo for 9044a54 has started.
⚪ 2025-08-28 16:07:16 UTC Artifacts will be uploaded here
⚪ 2025-08-28 16:11:06 UTC ya make is running...
🟡 2025-08-28 18:19:58 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40528	37275	0	489	2718	46

⚪ 2025-08-28 18:24:01 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-08-28 18:42:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1543 (only retried tests)	1028	0	485	0	30

⚪ 2025-08-28 18:43:09 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-08-28 18:58:15 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
919 (only retried tests)	410	0	485	0	24

🟢 2025-08-28 18:58:26 UTC Build successful.

github-actions · 2025-09-01T12:06:53Z

⚪ 2025-09-01 12:06:52 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6dff6a7 has started.
⚪ 2025-09-01 12:07:07 UTC Artifacts will be uploaded here
⚪ 2025-09-01 12:10:57 UTC ya make is running...
🟡 2025-09-01 14:18:09 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40581	37444	0	376	2715	46

⚪ 2025-09-01 14:21:49 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-09-01 14:41:22 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1012 (only retried tests)	608	0	375	0	29

⚪ 2025-09-01 14:41:36 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-09-01 14:55:34 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
819 (only retried tests)	420	0	374	0	25

🟢 2025-09-01 14:55:45 UTC Build successful.

github-actions · 2025-09-01T12:06:57Z

⚪ 2025-09-01 12:06:56 UTC Pre-commit check linux-x86_64-release-asan for 6dff6a7 has started.
⚪ 2025-09-01 12:07:12 UTC Artifacts will be uploaded here
⚪ 2025-09-01 12:11:22 UTC ya make is running...

github-actions · 2025-09-02T14:48:51Z

⚪ 2025-09-02 14:48:51 UTC Pre-commit check linux-x86_64-release-asan for 123a552 has started.
⚪ 2025-09-02 14:49:04 UTC Artifacts will be uploaded here
🔴 2025-09-02 14:51:04 UTC Graph compare failed, see the logs.

github-actions · 2025-09-02T14:51:01Z

⚪ 2025-09-02 14:51:01 UTC Pre-commit check linux-x86_64-relwithdebinfo for 123a552 has started.
⚪ 2025-09-02 14:51:14 UTC Artifacts will be uploaded here
🔴 2025-09-02 14:53:07 UTC Graph compare failed, see the logs.

github-actions · 2025-09-02T16:49:18Z

⚪ 2025-09-02 16:49:17 UTC Pre-commit check linux-x86_64-relwithdebinfo for 15c9cab has started.
⚪ 2025-09-02 16:49:32 UTC Artifacts will be uploaded here
⚪ 2025-09-02 16:53:31 UTC ya make is running...
🟡 2025-09-02 19:00:54 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40588	37393	0	426	2728	41

⚪ 2025-09-02 19:05:05 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-09-02 19:21:16 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1481 (only retried tests)	1029	0	422	6	24

⚪ 2025-09-02 19:21:31 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-09-02 19:37:09 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
846 (only retried tests)	398	0	422	3	23

🟢 2025-09-02 19:37:21 UTC Build successful.

github-actions · 2025-09-02T16:49:36Z

⚪ 2025-09-02 16:49:35 UTC Pre-commit check linux-x86_64-release-asan for 15c9cab has started.
⚪ 2025-09-02 16:49:50 UTC Artifacts will be uploaded here
⚪ 2025-09-02 16:53:45 UTC ya make is running...
🟡 2025-09-02 19:33:37 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17874	17292	0	215	343	24

🟢 2025-09-02 19:35:08 UTC Build successful.

github-actions · 2025-09-03T15:56:22Z

⚪ 2025-09-03 15:56:21 UTC Pre-commit check linux-x86_64-release-asan for b104f8f has started.
⚪ 2025-09-03 15:56:37 UTC Artifacts will be uploaded here
⚪ 2025-09-03 16:00:38 UTC ya make is running...
🟡 2025-09-03 18:27:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17874	17310	0	206	331	27

🟢 2025-09-03 18:29:21 UTC Build successful.

github-actions · 2025-09-03T16:20:22Z

⚪ 2025-09-03 16:20:21 UTC Pre-commit check linux-x86_64-relwithdebinfo for b104f8f has started.
⚪ 2025-09-03 16:20:35 UTC Artifacts will be uploaded here
⚪ 2025-09-03 16:24:25 UTC ya make is running...
🟡 2025-09-03 18:08:30 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40588	37451	0	379	2719	39

⚪ 2025-09-03 18:12:04 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-09-03 18:27:36 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
961 (only retried tests)	563	0	374	0	24

⚪ 2025-09-03 18:27:51 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-09-03 18:43:23 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
690 (only retried tests)	292	0	374	0	24

🟢 2025-09-03 18:43:33 UTC Build successful.

github-actions · 2025-09-03T21:18:02Z

⚪ 2025-09-03 21:18:01 UTC Pre-commit check linux-x86_64-release-asan for d9b0058 has started.
⚪ 2025-09-03 21:18:05 UTC Artifacts will be uploaded here
⚪ 2025-09-03 21:21:57 UTC ya make is running...
🟡 2025-09-03 23:37:33 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17874	17452	0	106	287	29

🟢 2025-09-03 23:39:04 UTC Build successful.

github-actions · 2025-09-03T21:19:34Z

⚪ 2025-09-03 21:19:34 UTC Pre-commit check linux-x86_64-relwithdebinfo for d9b0058 has started.
⚪ 2025-09-03 21:19:38 UTC Artifacts will be uploaded here
⚪ 2025-09-03 21:23:28 UTC ya make is running...
🟡 2025-09-03 23:03:25 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40588	37532	0	297	2722	37

⚪ 2025-09-03 23:06:57 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-09-03 23:26:59 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
1057 (only retried tests)	728	0	293	9	27

⚪ 2025-09-03 23:27:15 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-09-03 23:42:30 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
651 (only retried tests)	322	0	293	9	27

🟢 2025-09-03 23:42:40 UTC Build successful.

github-actions · 2025-09-04T10:47:49Z

⚪ 2025-09-04 10:47:48 UTC Pre-commit check linux-x86_64-release-asan for b912428 has started.
⚪ 2025-09-04 10:48:01 UTC Artifacts will be uploaded here
⚪ 2025-09-04 10:51:53 UTC ya make is running...
🟡 2025-09-04 13:35:31 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17958	17456	0	152	324	26

🟢 2025-09-04 13:37:10 UTC Build successful.

github-actions · 2025-09-04T10:50:47Z

⚪ 2025-09-04 10:50:47 UTC Pre-commit check linux-x86_64-relwithdebinfo for b912428 has started.
⚪ 2025-09-04 10:51:00 UTC Artifacts will be uploaded here
⚪ 2025-09-04 10:55:08 UTC ya make is running...
🟡 2025-09-04 13:00:47 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40695	37648	0	294	2717	36

⚪ 2025-09-04 13:04:25 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-09-04 13:24:22 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
733 (only retried tests)	414	0	294	0	25

⚪ 2025-09-04 13:24:36 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-09-04 13:38:57 UTC Some tests failed, follow the links below.

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
589 (only retried tests)	271	0	295	0	23

🟢 2025-09-04 13:39:08 UTC Build successful.

github-actions · 2025-09-04T20:45:51Z

⚪ 2025-09-04 20:45:50 UTC Pre-commit check linux-x86_64-release-asan for fa6f27f has started.
⚪ 2025-09-04 20:46:19 UTC Artifacts will be uploaded here
⚪ 2025-09-04 20:50:53 UTC ya make is running...
⚫ 2025-09-04 21:11:43 UTC Check cancelled

github-actions · 2025-09-04T20:48:28Z

⚪ 2025-09-04 20:48:28 UTC Pre-commit check linux-x86_64-relwithdebinfo for fa6f27f has started.
⚪ 2025-09-04 20:48:42 UTC Artifacts will be uploaded here
⚪ 2025-09-04 20:52:46 UTC ya make is running...
⚫ 2025-09-04 21:11:36 UTC Check cancelled

github-actions · 2025-09-04T21:12:47Z

⚪ 2025-09-04 21:12:46 UTC Pre-commit check linux-x86_64-release-asan for dbf2826 has started.
⚪ 2025-09-04 21:15:33 UTC Artifacts will be uploaded here
⚪ 2025-09-04 21:20:09 UTC ya make is running...
🟡 2025-09-04 23:21:48 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
17958	17476	0	137	317	28

🟢 2025-09-04 23:23:26 UTC Build successful.

github-actions · 2025-09-04T21:15:22Z

⚪ 2025-09-04 21:15:21 UTC Pre-commit check linux-x86_64-relwithdebinfo for dbf2826 has started.
⚪ 2025-09-04 21:15:35 UTC Artifacts will be uploaded here
⚪ 2025-09-04 21:19:38 UTC ya make is running...
🟡 2025-09-04 22:45:12 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
40695	37933	0	2	2721	39

⚪ 2025-09-04 22:48:43 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-09-04 23:02:22 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED^?
527 (only retried tests)	498	0	0	3	26

🟢 2025-09-04 23:02:31 UTC Build successful.

CyberROFL · 2025-09-05T13:51:41Z

ydb/core/tx/schemeshard/ut_helpers/ls_checks.cpp

+            break;
+        }
+
+        UNIT_ASSERT_VALUES_EQUAL(actual, proto);


Suggested change

UNIT_ASSERT_VALUES_EQUAL(actual, proto);

UNIT_ASSERT_NO_DIFF(actual, proto);

Или вообще перейти на MessageDifferencer.

CyberROFL · 2025-09-05T13:51:57Z

ydb/core/tx/schemeshard/ut_helpers/ls_checks.cpp

+        TString actual;
+        switch (record.GetPathDescription().GetTableIndex().GetSpecializedIndexDescriptionCase()) {
+        case NKikimrSchemeOp::TIndexDescription::kVectorIndexKmeansTreeDescription:
+            actual = record.GetPathDescription().GetTableIndex().GetVectorIndexKmeansTreeDescription().GetSettings().ShortDebugString();


Почему ShortDebugString, а не просто DebugString?

там переводы строк появляются и неудобно передавать

в целом можно парсить строчку протобуфную из expected и сравнивать протобуфы, но не думаю что это важно

kunga mentioned this pull request Aug 28, 2025

Fulltext index: Initial version #23384

Open

12 tasks

kunga self-assigned this Aug 29, 2025

kunga force-pushed the dev-fulltext branch from 0db5bfe to 6f7c846 Compare September 1, 2025 11:43

kunga force-pushed the fulltext-scheme branch from 2276e78 to 0fd0d7e Compare September 1, 2025 12:05

kunga force-pushed the dev-fulltext branch from 6f7c846 to bd93100 Compare September 2, 2025 14:43

kunga force-pushed the fulltext-scheme branch from 0fd0d7e to f47840e Compare September 2, 2025 14:46

kunga force-pushed the dev-fulltext branch from bd93100 to e3081ad Compare September 4, 2025 10:46

kunga added 11 commits September 4, 2025 13:46

switch case new index

8c1b184

fix ttl check on create

b9d650a

better asserts

2a65512

NFulltext

b38b98c

wip test

e2e40b4

fix

3b00eec

fix

ba21d06

add tests

cc5f487

fix break

c6a8b92

fixes

92469e5

fixes

b96714a

kunga added 4 commits September 4, 2025 13:46

EIndexTypeInvalid -> default

4dfe913

fix build

9e5bf9c

fix GetIndexType

c758018

fix using GetIndexType

4ab2fdf

kunga force-pushed the fulltext-scheme branch from 6b74983 to 4ab2fdf Compare September 4, 2025 10:46

kunga added 4 commits September 4, 2025 14:45

fix DoesIndexSupportTTL

f455418

disable backup tests

af3cc35

fix

ca15bbe

fix not invalid type

4182afd

kunga added 2 commits September 4, 2025 23:53

better comment

a1a252d

verify index type more relaxed

a0eed96

kunga marked this pull request as ready for review September 5, 2025 08:15

kunga requested a review from a team as a code owner September 5, 2025 08:15

kunga requested review from CyberROFL and vitalif September 8, 2025 09:15

snaury approved these changes Sep 8, 2025

View reviewed changes

vitalif approved these changes Sep 8, 2025

View reviewed changes

CyberROFL reviewed Sep 8, 2025

View reviewed changes

CyberROFL approved these changes Sep 8, 2025

View reviewed changes

kunga merged commit bbcdbc4 into ydb-platform:dev-fulltext Sep 8, 2025
17 checks passed

kunga added a commit that referenced this pull request Sep 9, 2025

Support fulltext index on SchemeShard (#23739)

919dc2d

kunga added a commit that referenced this pull request Sep 19, 2025

Support fulltext index on SchemeShard (#23739)

d900d1a

kunga added a commit that referenced this pull request Sep 23, 2025

Support fulltext index on SchemeShard (#23739)

4e76cbc

AlexanderShprot pushed a commit to AlexanderShprot/ydb that referenced this pull request Sep 25, 2025

Support fulltext index on SchemeShard (ydb-platform#23739)

815675b

	UNIT_ASSERT_VALUES_EQUAL(actual, proto);
	UNIT_ASSERT_NO_DIFF(actual, proto);

Support fulltext index on SchemeShard #23739

Support fulltext index on SchemeShard #23739

Uh oh!

Conversation

kunga commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog entry

Changelog category

Description for reviewers

Uh oh!

github-actions bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CyberROFL Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

CyberROFL Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

CyberROFL Sep 5, 2025

Choose a reason for hiding this comment

Uh oh!

kunga Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kunga commented Aug 28, 2025 •

edited

Loading

github-actions bot commented Aug 28, 2025 •

edited

Loading

github-actions bot commented Aug 28, 2025 •

edited

Loading

github-actions bot commented Sep 1, 2025 •

edited

Loading

github-actions bot commented Sep 1, 2025 •

edited

Loading

github-actions bot commented Sep 2, 2025 •

edited

Loading

github-actions bot commented Sep 2, 2025 •

edited

Loading

github-actions bot commented Sep 2, 2025 •

edited

Loading

github-actions bot commented Sep 2, 2025 •

edited

Loading

github-actions bot commented Sep 3, 2025 •

edited

Loading

github-actions bot commented Sep 3, 2025 •

edited

Loading

github-actions bot commented Sep 3, 2025 •

edited

Loading

github-actions bot commented Sep 3, 2025 •

edited

Loading

github-actions bot commented Sep 4, 2025 •

edited

Loading

github-actions bot commented Sep 4, 2025 •

edited

Loading

github-actions bot commented Sep 4, 2025 •

edited

Loading

github-actions bot commented Sep 4, 2025 •

edited

Loading

github-actions bot commented Sep 4, 2025 •

edited

Loading

github-actions bot commented Sep 4, 2025 •

edited

Loading