Skip to content

Conversation

dcherednik
Copy link
Member

@dcherednik dcherednik commented Apr 14, 2025

Changelog entry

Support for MSG_ZEROCOPY feature for XDC stream.

Linux has features which allows to perform send in to socket without copy during send call. MSG_ZEROCOPY is a one of possibility to achieve it.

without ZC
image

with ZC
image

Changelog category

  • New feature
  • Experimental feature
  • Improvement
  • Performance improvement
  • User Interface
  • Bugfix
  • Backward incompatible change
  • Documentation (changelog entry is not required)
  • Not for changelog (changelog entry is not required)

Description for reviewers

...

@dcherednik dcherednik marked this pull request as draft April 14, 2025 15:39
Copy link

github-actions bot commented Apr 14, 2025

🔴 2025-04-22 13:07:24 UTC The validation of the Pull Request description has failed. Please update the description.

Only one category can be selected at a time.

Copy link

github-actions bot commented Apr 14, 2025

2025-04-14 15:40:46 UTC Pre-commit check linux-x86_64-release-asan for 16154c0 has started.
2025-04-14 15:41:01 UTC Artifacts will be uploaded here
2025-04-14 15:44:13 UTC ya make is running...
🟡 2025-04-14 17:32:34 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14630 14491 0 67 36 36

2025-04-14 17:33:48 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-14 17:47:03 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
190 (only retried tests) 110 0 37 10 33

2025-04-14 17:47:12 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-04-14 17:59:06 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
114 (only retried tests) 41 0 35 5 33

🟢 2025-04-14 17:59:14 UTC Build successful.
🟡 2025-04-14 17:59:44 UTC ydbd size 3.9 GiB changed* by +247.3 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6c87d31 merge: 16154c0 diff diff %
ydbd size 4 141 764 720 Bytes 4 142 017 976 Bytes +247.3 KiB +0.006%
ydbd stripped size 1 430 153 480 Bytes 1 430 210 984 Bytes +56.2 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Apr 14, 2025

2025-04-14 15:40:59 UTC Pre-commit check linux-x86_64-relwithdebinfo for 16154c0 has started.
2025-04-14 15:41:16 UTC Artifacts will be uploaded here
2025-04-14 15:44:35 UTC ya make is running...
🟡 2025-04-14 17:33:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
29388 26685 0 5 2576 122

2025-04-14 17:35:37 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-04-14 17:46:46 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
201 (only retried tests) 94 0 0 0 107

🟢 2025-04-14 17:46:53 UTC Build successful.
🟡 2025-04-14 17:47:14 UTC ydbd size 2.2 GiB changed* by +152.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 6c87d31 merge: 16154c0 diff diff %
ydbd size 2 360 246 144 Bytes 2 360 401 800 Bytes +152.0 KiB +0.007%
ydbd stripped size 493 203 808 Bytes 493 223 584 Bytes +19.3 KiB +0.004%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

dcherednik and others added 6 commits April 22, 2025 14:04
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>
Copy link

github-actions bot commented Apr 22, 2025

2025-04-22 12:45:47 UTC Pre-commit check linux-x86_64-release-asan for 34db1de has started.
2025-04-22 12:45:53 UTC Artifacts will be uploaded here
2025-04-22 12:49:09 UTC ya make is running...
2025-04-22 14:06:34 UTC Check cancelled

Copy link

github-actions bot commented Apr 22, 2025

2025-04-22 12:46:51 UTC Pre-commit check linux-x86_64-relwithdebinfo for 34db1de has started.
2025-04-22 12:46:58 UTC Artifacts will be uploaded here
2025-04-22 12:50:07 UTC ya make is running...
2025-04-22 14:06:34 UTC Check cancelled

@dcherednik dcherednik marked this pull request as ready for review April 22, 2025 12:56
Copy link
Member

@va-kuznecov va-kuznecov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link

github-actions bot commented Apr 22, 2025

2025-04-22 14:10:01 UTC Pre-commit check linux-x86_64-relwithdebinfo for 0e454f8 has started.
2025-04-22 14:10:17 UTC Artifacts will be uploaded here
2025-04-22 14:13:27 UTC ya make is running...
🟡 2025-04-22 16:10:48 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
29833 27122 0 7 2593 111

2025-04-22 16:13:17 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-04-22 16:25:15 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
192 (only retried tests) 95 0 0 0 97

🟢 2025-04-22 16:25:22 UTC Build successful.
🟡 2025-04-22 16:25:44 UTC ydbd size 2.2 GiB changed* by +208.8 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: f600bdf merge: 0e454f8 diff diff %
ydbd size 2 352 326 696 Bytes 2 352 540 456 Bytes +208.8 KiB +0.009%
ydbd stripped size 493 847 520 Bytes 493 872 992 Bytes +24.9 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Apr 22, 2025

2025-04-22 14:10:08 UTC Pre-commit check linux-x86_64-release-asan for 0e454f8 has started.
2025-04-22 14:10:24 UTC Artifacts will be uploaded here
2025-04-22 14:13:38 UTC ya make is running...
🟡 2025-04-22 16:07:45 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
14964 14824 0 75 37 28

2025-04-22 16:09:02 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-04-22 16:22:19 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet Going to retry failed tests...

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
176 (only retried tests) 107 0 38 7 24

2025-04-22 16:22:28 UTC ya make is running... (failed tests rerun, try 3)
🟡 2025-04-22 16:35:50 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
94 (only retried tests) 37 0 34 2 21

🟢 2025-04-22 16:36:00 UTC Build successful.
🟡 2025-04-22 16:36:31 UTC ydbd size 3.8 GiB changed* by +338.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: d7f39a1 merge: 0e454f8 diff diff %
ydbd size 4 132 405 632 Bytes 4 132 752 440 Bytes +338.7 KiB +0.008%
ydbd stripped size 1 430 873 688 Bytes 1 430 959 800 Bytes +84.1 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

result.EventDelay = TDuration::MicroSeconds(config.GetEventDelayMicrosec());
}

if (config.HasSocketSendOptimization()) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this "if", really. It defaults to IC_SO_DISABLED, and result.SocketSendOptimization defaults to DISABLED.

@dcherednik dcherednik merged commit 08dc695 into ydb-platform:main Apr 24, 2025
15 checks passed
dcherednik added a commit to dcherednik/ydb that referenced this pull request Sep 26, 2025
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>

Support for MSG_ZEROCOPY feature for XDC stream.

Linux has features which allows to perform send in to socket without copy during send call. MSG_ZEROCOPY is a one of possibility to achieve it.

To use it we must configure net.core.optmem_max kernel option, default values usually not enought. Also process should have capability to lock memory, so consider settings ulimit -l or IPC_LOCK capability.

Note0: MSG_ZEROCOPY causes hiden copy inside kernel when traffic goes between process on the same host. The implementation handle it  and disable using MSG_ZEROCOPY for such sessions.
Note1: Some processes using PACKET sockets breaks MSG_ZEROCOPY and also causes hidden copy inside kernel. So consider output `ss -0` in case of non succcess.
Note2: Hardware must support scatter-gather

ydb-platform#17181 (comment)
dcherednik added a commit that referenced this pull request Sep 29, 2025
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>

Support for MSG_ZEROCOPY feature for XDC stream.

Linux has features which allows to perform send in to socket without copy during send call. MSG_ZEROCOPY is a one of possibility to achieve it.

To use it we must configure net.core.optmem_max kernel option, default values usually not enought. Also process should have capability to lock memory, so consider settings ulimit -l or IPC_LOCK capability.

Note0: MSG_ZEROCOPY causes hiden copy inside kernel when traffic goes between process on the same host. The implementation handle it  and disable using MSG_ZEROCOPY for such sessions.
Note1: Some processes using PACKET sockets breaks MSG_ZEROCOPY and also causes hidden copy inside kernel. So consider output `ss -0` in case of non succcess.
Note2: Hardware must support scatter-gather

#17181 (comment)
dcherednik added a commit to dcherednik/ydb that referenced this pull request Sep 30, 2025
Co-authored-by: Vlad Kuznetsov <va.kuznecov@physics.msu.ru>

Support for MSG_ZEROCOPY feature for XDC stream.

Linux has features which allows to perform send in to socket without copy during send call. MSG_ZEROCOPY is a one of possibility to achieve it.

To use it we must configure net.core.optmem_max kernel option, default values usually not enought. Also process should have capability to lock memory, so consider settings ulimit -l or IPC_LOCK capability.

Note0: MSG_ZEROCOPY causes hiden copy inside kernel when traffic goes between process on the same host. The implementation handle it  and disable using MSG_ZEROCOPY for such sessions.
Note1: Some processes using PACKET sockets breaks MSG_ZEROCOPY and also causes hidden copy inside kernel. So consider output `ss -0` in case of non succcess.
Note2: Hardware must support scatter-gather

ydb-platform#17181 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants