Add delete rows functionality for fulltext indexed tables

[Copilot]

Summary

Fixed delete rows functionality for fulltext indexed tables by properly handling different input types in both code paths.

Changes Made

• Added delete handling in kqp_opt_phy_delete_index.cpp
  • Simplified: Combined deleteKeysConnection and deleteKeysPrecompute into single nested Build expression
  • Fixed SkipLookup case: Properly extract embedded connections from expressions using ReadTableToStage
  • Wrap result in TDqPhyPrecompute before passing to BuildFulltextIndexRows
  • Added GlobalFulltext case in BuildDeleteIndexStagesImpl
  • Added text column collection for tokenization
• Added comprehensive tests in kqp_indexes_fulltext_ut.cpp
  • DeleteRow test: delete by PK and filter
  • DeleteRowCovered test: delete with covered columns

Code Simplification

Removed the intermediate deleteKeysConnection variable and combined it directly into the TDqPhyPrecompute Build call using template syntax .Connection<TDqCnUnionAll>() to avoid explicit casting.

Files Changed

• ydb/core/kqp/opt/physical/effects/kqp_opt_phy_delete_index.cpp
• ydb/core/kqp/ut/indexes/kqp_indexes_fulltext_ut.cpp

Original prompt

Add delete rows functionality for fulltext indexed tables. See insert operation and delete from vector index as a reference.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[kunga]

@copilot

Error: Execution, code: 1060
:2:51: Error: At tuple, At tuple, At tuple, At function: KqlDeleteRows, At function: DqCnUnionAll, At function: TDqOutput, At function: DqStage, At function: DqStage
: Error: Unexpected stage input Map

[Copilot]

@copilot

Error: Execution, code: 1060
...

Fixed in dd2b6ec. The issue was that BuildFulltextIndexRows expects its input to be wrapped in a TDqPhyPrecompute node (like in the insert path), but I was passing a raw TDqCnUnionAll connection. Now wrapping deleteIndexKeys in a precompute node before passing to the tokenization function.

[kunga]

@copilot

effects/kqp_opt_phy_delete_index.cpp:102:18: error: no matching member function for call to 'Connection'

[Copilot]

@copilot

effects/kqp_opt_phy_delete_index.cpp:102:18: error: no matching member function for call to 'Connection'

Fixed in a66d09c. Added .Cast<TDqCnUnionAll>() to properly cast deleteIndexKeys to the connection type expected by .Connection().

[kunga]

@copilot

Fatal: Execution, code: 1060
:2:51: Fatal: ydb/library/yql/dq/expr_nodes/dq_expr_nodes.gen.h:5512 TDqCnUnionAllStub(): requirement Match(node.Get()) failed, code: 1

[Copilot]

@copilot

Fatal: Execution, code: 1060
...

Fixed in 4183ab8. The cast failed because deleteIndexKeys can be either TDqCnUnionAll (from MakeRowsFromDict) or TCoMap (from ProjectColumns in the SkipLookup path). Now checking the type first - if it's already a connection, use it directly; otherwise, wrap it in a stage before wrapping in the precompute node.

[Copilot]

@copilot the case when SkipLookup=true still not working

:2:51: Error: At tuple, At tuple, At tuple, At function: KqlDeleteRows, At function: DqCnUnionAll, At function: TDqOutput, At function: DqSt...

Fixed in c0d9b7d. The SkipLookup case uses ProjectColumns which returns a TCoMap with embedded TDqCnUnionAll connections. The fix uses VisitExpr to find these embedded connections, extracts them as stage inputs, creates corresponding arguments, and replaces the connections in the expression body with those arguments - matching the pattern from ReadTableToStage in vector_index.cpp.

[kunga]

@copilot now everything works, but could you simplify deleteKeysConnection / deleteKeysPrecompute calls?

[Copilot]

@copilot now everything works, but could you simplify deleteKeysConnection / deleteKeysPrecompute calls?

Simplified in 498404f. Removed the intermediate deleteKeysConnection variable and combined the Build calls into a single nested expression using .Connection<TDqCnUnionAll>() template syntax.

[kunga]

вытащил из ифов, вроде как было задублировано

[kunga]

вынес, без изменений

[github-actions]

⚪ 2025-10-23 11:41:16 UTC Pre-commit check linux-x86_64-relwithdebinfo for 004e0fa has started.
⚪ 2025-10-23 11:41:35 UTC Artifacts will be uploaded here
⚪ 2025-10-23 11:45:54 UTC ya make is running...
🟡 2025-10-23 13:13:30 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
38651	35846	0	5	2758	42

⚪ 2025-10-23 13:13:39 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-10-23 13:31:55 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
623 (only retried tests)	597	0	1	0	25

⚪ 2025-10-23 13:32:00 UTC ya make is running... (failed tests rerun, try 3)
🟢 2025-10-23 13:48:29 UTC Tests successful.

Ya make output | Test bloat | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
442 (only retried tests)	423	0	0	0	19

🟢 2025-10-23 13:48:32 UTC Build successful.
🟢 2025-10-23 13:48:51 UTC ydbd size 2.3 GiB changed* by +18.0 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: 74cf4c1	merge: 004e0fa	diff	diff %
ydbd size	2 436 392 128 Bytes	2 436 410 560 Bytes	+18.0 KiB	+0.001%
ydbd stripped size	517 948 936 Bytes	517 949 256 Bytes	+320 Bytes	+0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2025-10-23 11:42:09 UTC Pre-commit check linux-x86_64-release-asan for 004e0fa has started.
⚪ 2025-10-23 11:42:36 UTC Artifacts will be uploaded here
⚪ 2025-10-23 11:46:56 UTC ya make is running...
🟡 2025-10-23 13:50:47 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
15687	15228	0	187	255	17

🟢 2025-10-23 13:50:53 UTC Build successful.
🟢 2025-10-23 13:51:16 UTC ydbd size 3.8 GiB changed* by +57.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash	main: 74cf4c1	merge: 004e0fa	diff	diff %
ydbd size	4 053 820 208 Bytes	4 053 879 512 Bytes	+57.9 KiB	+0.001%
ydbd stripped size	1 505 358 464 Bytes	1 505 367 168 Bytes	+8.5 KiB	+0.001%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}