YQ-4736 optimized json parser memory usage

[GrigoriyPA]

Changelog entry

Optimized json parser memory usage

Changelog category

• Not for changelog (changelog entry is not required)

Description for reviewers

[Copilot]

Pull Request Overview

This PR optimizes JSON parser memory usage by refactoring data structures to use contiguous memory layouts and reducing memory allocations.

• Replaced pointer-based column storage with std::span for better memory locality
• Changed row indexing from 64-bit to 16-bit integers with fixed upper limits
• Consolidated memory buffers to reduce fragmentation and allocation overhead

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
topic_parser_ut.cpp	Updated test fixtures to use std::span instead of pointer-based column access
topic_filter_ut.cpp	Modified test helper methods to return std::span instead of vector pointers
raw_parser.cpp	Changed GetParsedColumn to return std::span instead of vector pointer
parser_abstract.h	Updated interface to use std::span for parsed column data
json_parser.h	Removed memory complexity comment from buffer configuration
json_parser.cpp	Implemented core memory optimizations with consolidated buffers and 16-bit indexing
format_handler.cpp	Updated data handling to work with std::span instead of vector pointers
purecalc_filter.h	Changed filter interface to accept std::span instead of vector pointers
purecalc_filter.cpp	Updated filter implementation to work with std::span data access
filters_set.h	Modified filter set interface to use std::span for data processing
filters_set.cpp	Updated filter processing logic to handle std::span data structures

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[github-actions]

🟢 2025-09-26 21:04:56 UTC The validation of the Pull Request description is successful.

[github-actions]

⚪ 2025-09-26 21:05:04 UTC Pre-commit check linux-x86_64-relwithdebinfo for 87106bf has started.
⚪ 2025-09-26 21:05:20 UTC Artifacts will be uploaded here
⚪ 2025-09-26 21:08:53 UTC ya make is running...
🟡 2025-09-26 22:33:54 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
38060	35176	0	2	2848	34

⚪ 2025-09-26 22:34:02 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-09-26 22:48:51 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
518 (only retried tests)	387	0	0	108	23

🟢 2025-09-26 22:48:53 UTC Build successful.
🟢 2025-09-26 22:49:09 UTC ydbd size 2.2 GiB changed* by -11.8 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: 592d4dc	merge: 87106bf	diff	diff %
ydbd size	2 397 030 096 Bytes	2 397 018 064 Bytes	-11.8 KiB	-0.001%
ydbd stripped size	512 486 504 Bytes	512 484 264 Bytes	-2.2 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2025-09-26 21:05:08 UTC Pre-commit check linux-x86_64-release-asan for 87106bf has started.
⚪ 2025-09-26 21:05:23 UTC Artifacts will be uploaded here
⚪ 2025-09-26 21:08:59 UTC ya make is running...
🟡 2025-09-26 23:08:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
15443	14898	0	166	343	36

🟢 2025-09-26 23:08:48 UTC Build successful.
🟢 2025-09-26 23:09:12 UTC ydbd size 3.7 GiB changed* by -4.1 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: 592d4dc	merge: 87106bf	diff	diff %
ydbd size	3 986 696 168 Bytes	3 986 691 968 Bytes	-4.1 KiB	-0.000%
ydbd stripped size	1 485 978 080 Bytes	1 485 976 928 Bytes	-1.1 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2025-09-29 15:30:11 UTC Pre-commit check linux-x86_64-release-asan for c912402 has started.
⚪ 2025-09-29 15:30:17 UTC Artifacts will be uploaded here
⚪ 2025-09-29 15:33:59 UTC ya make is running...
🟡 2025-09-29 17:30:05 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
15446	14932	0	213	272	29

🟢 2025-09-29 17:30:10 UTC Build successful.
🟢 2025-09-29 17:30:38 UTC ydbd size 3.7 GiB changed* by -4.1 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: b0061c8	merge: c912402	diff	diff %
ydbd size	3 987 302 968 Bytes	3 987 298 744 Bytes	-4.1 KiB	-0.000%
ydbd stripped size	1 486 194 528 Bytes	1 486 193 376 Bytes	-1.1 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}

[github-actions]

⚪ 2025-09-29 15:30:33 UTC Pre-commit check linux-x86_64-relwithdebinfo for c912402 has started.
⚪ 2025-09-29 15:30:37 UTC Artifacts will be uploaded here
⚪ 2025-09-29 15:34:23 UTC ya make is running...
🟡 2025-09-29 17:02:01 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
38077	35186	0	6	2852	33

⚪ 2025-09-29 17:02:10 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-09-29 17:18:03 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS	PASSED	ERRORS	FAILED	SKIPPED	MUTED?
454 (only retried tests)	318	0	0	108	28

🟢 2025-09-29 17:18:06 UTC Build successful.
🟢 2025-09-29 17:18:24 UTC ydbd size 2.2 GiB changed* by -11.6 KiB, which is <= 0 Bytes vs main: OK

ydbd size dash	main: b0061c8	merge: c912402	diff	diff %
ydbd size	2 397 354 008 Bytes	2 397 342 152 Bytes	-11.6 KiB	-0.000%
ydbd stripped size	512 554 408 Bytes	512 552 296 Bytes	-2.1 KiB	-0.000%

^{*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation}