Skip to content

Conversation

@stebloev
Copy link
Member

Add snowball stemming filter for fulltext index

Now use_filter_snowball fulltext index option is supported. It requires language option to be set. List of supported languages can be found here

Changelog category

  • New feature

Description for reviewers

  • Add use_filter_snowball option and relevant validation
  • Add snowball stemming into index building process
  • Add unit tests for analyzer and in kqp

@github-actions
Copy link

🔴 2025-11-13 11:57:15 UTC The validation of the Pull Request description has failed. Please update the description.

The changelog entry is less than 20 characters or missing.

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 11:57:56 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1f82ea2 has started.
2025-11-13 11:58:14 UTC Artifacts will be uploaded here
2025-11-13 12:00:32 UTC ya make is running...
2025-11-13 13:23:28 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 11:57:57 UTC Pre-commit check linux-x86_64-release-asan for 1f82ea2 has started.
2025-11-13 11:58:15 UTC Artifacts will be uploaded here
2025-11-13 12:00:31 UTC ya make is running...
2025-11-13 13:23:26 UTC Check cancelled

@stebloev stebloev mentioned this pull request Nov 13, 2025
13 tasks
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 13:24:07 UTC Pre-commit check linux-x86_64-release-asan for 6c32f3f has started.
2025-11-13 13:24:40 UTC Artifacts will be uploaded here
2025-11-13 13:26:06 UTC ya make is running...
2025-11-13 13:33:21 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 13:25:07 UTC Pre-commit check linux-x86_64-relwithdebinfo for 6c32f3f has started.
2025-11-13 13:25:25 UTC Artifacts will be uploaded here
2025-11-13 13:27:50 UTC ya make is running...
2025-11-13 13:33:17 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 13:35:41 UTC Pre-commit check linux-x86_64-relwithdebinfo for 860b84d has started.
2025-11-13 13:35:57 UTC Artifacts will be uploaded here
2025-11-13 13:38:15 UTC ya make is running...
2025-11-13 15:02:47 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 13:35:42 UTC Pre-commit check linux-x86_64-release-asan for 860b84d has started.
2025-11-13 13:35:59 UTC Artifacts will be uploaded here
2025-11-13 13:38:13 UTC ya make is running...
2025-11-13 15:02:47 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 15:05:17 UTC Pre-commit check linux-x86_64-relwithdebinfo for ea2c2d9 has started.
2025-11-13 15:05:39 UTC Artifacts will be uploaded here
2025-11-13 15:07:57 UTC ya make is running...
2025-11-13 15:19:28 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 15:05:17 UTC Pre-commit check linux-x86_64-release-asan for ea2c2d9 has started.
2025-11-13 15:05:34 UTC Artifacts will be uploaded here
2025-11-13 15:07:47 UTC ya make is running...
2025-11-13 15:19:28 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 15:19:57 UTC Pre-commit check linux-x86_64-release-asan for 3783e12 has started.
2025-11-13 15:20:18 UTC Artifacts will be uploaded here
2025-11-13 15:21:43 UTC ya make is running...
🟡 2025-11-13 17:56:25 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19040 18616 0 173 225 26

🟢 2025-11-13 17:56:36 UTC Build successful.
🔴 2025-11-13 17:57:02 UTC ydbd size 3.9 GiB changed* by +2.9 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 4d51368 merge: 3783e12 diff diff %
ydbd size 4 135 321 368 Bytes 4 138 357 584 Bytes +2.9 MiB +0.073%
ydbd stripped size 1 522 578 120 Bytes 1 524 859 560 Bytes +2.2 MiB +0.150%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stebloev stebloev marked this pull request as ready for review November 13, 2025 15:21
@stebloev stebloev requested a review from a team as a code owner November 13, 2025 15:21
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 15:23:00 UTC Pre-commit check linux-x86_64-relwithdebinfo for 3783e12 has started.
2025-11-13 15:23:17 UTC Artifacts will be uploaded here
2025-11-13 15:25:36 UTC ya make is running...
🟡 2025-11-13 17:32:51 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
42569 39749 0 3 2791 26

2025-11-13 17:33:05 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-11-13 17:42:22 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
50 (only retried tests) 36 0 0 0 14

🟢 2025-11-13 17:42:29 UTC Build successful.
🟡 2025-11-13 17:42:48 UTC ydbd size 2.3 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 4d51368 merge: 3783e12 diff diff %
ydbd size 2 479 232 200 Bytes 2 480 404 752 Bytes +1.1 MiB +0.047%
ydbd stripped size 520 990 064 Bytes 521 565 680 Bytes +562.1 KiB +0.110%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stebloev stebloev self-assigned this Nov 14, 2025
@stebloev stebloev requested a review from azevaykin November 14, 2025 12:44
@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 11:11:55 UTC Pre-commit check linux-x86_64-release-asan for bf1bca5 has started.
2025-11-19 11:12:35 UTC Artifacts will be uploaded here
2025-11-19 11:14:40 UTC ya make is running...
🟡 2025-11-19 13:12:48 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14729 14618 0 92 7 12

🟢 2025-11-19 13:12:57 UTC Build successful.
🔴 2025-11-19 13:13:21 UTC ydbd size 3.8 GiB changed* by +2.9 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 3986e8b merge: bf1bca5 diff diff %
ydbd size 4 096 192 152 Bytes 4 099 235 424 Bytes +2.9 MiB +0.074%
ydbd stripped size 1 520 505 552 Bytes 1 522 792 304 Bytes +2.2 MiB +0.150%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 11:13:11 UTC Pre-commit check linux-x86_64-relwithdebinfo for bf1bca5 has started.
2025-11-19 11:13:29 UTC Artifacts will be uploaded here
2025-11-19 11:15:47 UTC ya make is running...
2025-11-19 13:31:32 UTC Check cancelled

@stebloev stebloev requested a review from azevaykin November 19, 2025 12:24
@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 13:33:19 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8b2d767 has started.
2025-11-19 13:33:36 UTC Artifacts will be uploaded here
2025-11-19 13:35:50 UTC ya make is running...
2025-11-19 14:07:45 UTC Check cancelled

azevaykin
azevaykin previously approved these changes Nov 19, 2025
@azevaykin azevaykin requested a review from Copilot November 19, 2025 13:34
@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 13:36:08 UTC Pre-commit check linux-x86_64-release-asan for 8b2d767 has started.
2025-11-19 13:36:26 UTC Artifacts will be uploaded here
2025-11-19 13:38:35 UTC ya make is running...
2025-11-19 14:07:45 UTC Check cancelled

Copilot finished reviewing on behalf of azevaykin November 19, 2025 13:39
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 14:08:16 UTC Pre-commit check linux-x86_64-release-asan for 0dca931 has started.
2025-11-19 14:08:37 UTC Artifacts will be uploaded here
2025-11-19 14:09:57 UTC ya make is running...
2025-11-19 14:24:28 UTC Check cancelled

@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 14:09:12 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0dca931 has started.
2025-11-19 14:09:28 UTC Artifacts will be uploaded here
2025-11-19 14:11:35 UTC ya make is running...
2025-11-19 14:24:27 UTC Check cancelled

@stebloev stebloev requested a review from Copilot November 19, 2025 14:09
Copilot finished reviewing on behalf of stebloev November 19, 2025 14:12
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 14:26:20 UTC Pre-commit check linux-x86_64-relwithdebinfo for 42f46ca has started.
2025-11-19 14:26:37 UTC Artifacts will be uploaded here
2025-11-19 14:28:48 UTC ya make is running...
🟡 2025-11-19 16:46:01 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41976 39105 0 4 2839 28

2025-11-19 16:46:15 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-11-19 17:50:33 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
63 (only retried tests) 43 0 1 0 19

2025-11-19 17:50:41 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-11-19 18:02:45 UTC Tests successful.

Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
38 (only retried tests) 22 0 0 0 16

🟢 2025-11-19 18:02:53 UTC Build successful.
🟡 2025-11-19 18:03:17 UTC ydbd size 2.3 GiB changed* by +1.1 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 2bb8a9f merge: 42f46ca diff diff %
ydbd size 2 448 587 224 Bytes 2 449 763 176 Bytes +1.1 MiB +0.048%
ydbd stripped size 520 464 504 Bytes 521 041 048 Bytes +563.0 KiB +0.111%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 19, 2025

2025-11-19 14:26:36 UTC Pre-commit check linux-x86_64-release-asan for 42f46ca has started.
2025-11-19 14:26:42 UTC Artifacts will be uploaded here
2025-11-19 14:28:48 UTC ya make is running...
🟡 2025-11-19 16:32:48 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14692 14615 0 49 14 14

🟢 2025-11-19 16:32:57 UTC Build successful.
🔴 2025-11-19 16:33:25 UTC ydbd size 3.8 GiB changed* by +2.9 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: 419aa07 merge: 42f46ca diff diff %
ydbd size 4 099 165 608 Bytes 4 102 211 056 Bytes +2.9 MiB +0.074%
ydbd stripped size 1 521 570 224 Bytes 1 523 858 000 Bytes +2.2 MiB +0.150%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stebloev stebloev merged commit 0f75010 into ydb-platform:main Nov 20, 2025
10 checks passed
@stebloev stebloev deleted the fulltext-add-snowball-filter branch November 20, 2025 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants