Skip to content

Conversation

@stanislav-shchetinin
Copy link
Collaborator

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

...

Copilot AI review requested due to automatic review settings November 6, 2025 14:42
@github-actions

This comment was marked as outdated.

@github-actions
Copy link

github-actions bot commented Nov 6, 2025

🟢 2025-11-06 14:45:39 UTC The validation of the Pull Request description is successful.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR extends the file system import/export functionality in the YDB API to support encryption and more flexible path handling, bringing it to feature parity with the S3 import/export capabilities.

Key changes:

  • Added encryption support for FS import/export operations with encryption_settings field
  • Made source_path and destination_path fields optional in Item messages to support automatic path resolution
  • Added global source_path and destination_path fields for more flexible path configuration

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
ydb/public/api/protos/ydb_import.proto Added encryption support, made Item fields optional, and added global destination_path for FS imports
ydb/public/api/protos/ydb_export.proto Added encryption support, made Item.destination_path optional, and added global source_path for FS exports

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions

This comment was marked as outdated.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

@stanislav-shchetinin stanislav-shchetinin linked an issue Nov 7, 2025 that may be closed by this pull request
1 task
@UgnineSirdis
Copy link
Collaborator

Заметил вот тут отличие от других методов этого сервиса: https://github.com/stanislav-shchetinin/ydb/blob/7c83b4ae4e1e1870b050fe49ca057fbc1427d06c/ydb/public/api/grpc/ydb_export_v1.proto#L20
Наверное, для консистентности лучше будет написать rpc ExportToFs(Export.ExportToFsRequest) returns (Export.ExportToFsResponse);

Copy link
Collaborator

@UgnineSirdis UgnineSirdis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

А нам актуален параметр number_of_retries тут?

Copy link
Collaborator

@UgnineSirdis UgnineSirdis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Касательно EncryptionSettings вроде всё просто. Но эти настройки по требованиям идут рука об руку с идеей о том, что экспорт должен представлять собой нечто целое. Я бы подумал, не объявить ли это основным подходом. В таком случае некоторые из "путевых" параметров можно никак не добавлять в API

@stanislav-shchetinin
Copy link
Collaborator Author

stanislav-shchetinin commented Nov 10, 2025

А нам актуален параметр number_of_retries тут?

Мы хотим через этот вызов делать экспорт в том числе и в NFS, а там при мягком монтировании нужно указать количество ретраев. @UgnineSirdis

@github-actions

This comment was marked as outdated.

@github-actions

This comment was marked as outdated.

UgnineSirdis
UgnineSirdis previously approved these changes Nov 11, 2025
CyberROFL
CyberROFL previously approved these changes Nov 11, 2025
@github-actions
Copy link

github-actions bot commented Nov 11, 2025

2025-11-11 16:43:48 UTC Pre-commit check linux-x86_64-release-asan for 8dfc223 has started.
2025-11-11 16:44:04 UTC Artifacts will be uploaded here
2025-11-11 16:46:02 UTC ya make is running...
🟡 2025-11-11 19:28:28 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
18950 18579 0 145 197 29

🟢 2025-11-11 19:28:38 UTC Build successful.
🟡 2025-11-11 19:29:04 UTC ydbd size 3.8 GiB changed* by +311.6 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ed7c647 merge: 8dfc223 diff diff %
ydbd size 4 084 372 304 Bytes 4 084 691 424 Bytes +311.6 KiB +0.008%
ydbd stripped size 1 516 379 880 Bytes 1 516 514 312 Bytes +131.3 KiB +0.009%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

CyberROFL
CyberROFL previously approved these changes Nov 11, 2025
@github-actions
Copy link

github-actions bot commented Nov 11, 2025

2025-11-11 16:46:11 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8dfc223 has started.
2025-11-11 16:46:29 UTC Artifacts will be uploaded here
2025-11-11 16:48:48 UTC ya make is running...
🟡 2025-11-11 19:06:10 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
42477 39652 0 4 2793 28

2025-11-11 19:06:24 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-11-11 19:15:41 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
60 (only retried tests) 44 0 1 0 15

2025-11-11 19:15:47 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-11-11 19:23:37 UTC Some tests failed, follow the links below.

Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
32 (only retried tests) 17 0 1 0 14

🟢 2025-11-11 19:23:44 UTC Build successful.
🟡 2025-11-11 19:24:05 UTC ydbd size 2.3 GiB changed* by +160.9 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: ed7c647 merge: 8dfc223 diff diff %
ydbd size 2 439 429 336 Bytes 2 439 594 048 Bytes +160.9 KiB +0.007%
ydbd stripped size 518 928 848 Bytes 518 961 296 Bytes +31.7 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stanislav-shchetinin stanislav-shchetinin self-assigned this Nov 12, 2025
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 08:59:55 UTC Pre-commit check linux-x86_64-relwithdebinfo for 08648d2 has started.
2025-11-13 09:00:13 UTC Artifacts will be uploaded here
2025-11-13 09:02:34 UTC ya make is running...
🟡 2025-11-13 11:24:45 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
42552 39726 0 3 2791 32

2025-11-13 11:25:00 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-11-13 11:34:58 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
64 (only retried tests) 47 0 1 0 16

2025-11-13 11:35:06 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-11-13 11:43:14 UTC Some tests failed, follow the links below.

Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
34 (only retried tests) 18 0 1 0 15

🟢 2025-11-13 11:43:22 UTC Build successful.
🟡 2025-11-13 11:43:45 UTC ydbd size 2.3 GiB changed* by +161.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 80a142b merge: 08648d2 diff diff %
ydbd size 2 442 222 968 Bytes 2 442 387 872 Bytes +161.0 KiB +0.007%
ydbd stripped size 519 623 568 Bytes 519 656 208 Bytes +31.9 KiB +0.006%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 09:02:53 UTC Pre-commit check linux-x86_64-release-asan for 08648d2 has started.
2025-11-13 09:03:09 UTC Artifacts will be uploaded here
2025-11-13 09:05:25 UTC ya make is running...
🟡 2025-11-13 11:49:55 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19024 18595 0 195 206 28

🟢 2025-11-13 11:50:05 UTC Build successful.
🟡 2025-11-13 11:50:32 UTC ydbd size 3.8 GiB changed* by +311.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 80a142b merge: 08648d2 diff diff %
ydbd size 4 089 115 400 Bytes 4 089 434 616 Bytes +311.7 KiB +0.008%
ydbd stripped size 1 518 496 712 Bytes 1 518 631 208 Bytes +131.3 KiB +0.009%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@stanislav-shchetinin stanislav-shchetinin added the rebase-and-check Rebase PR with the current base branch and check label Nov 13, 2025
@github-actions github-actions bot removed the rebase-and-check Rebase PR with the current base branch and check label Nov 13, 2025
@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 12:35:39 UTC Pre-commit check linux-x86_64-relwithdebinfo for 286095e has started.
2025-11-13 12:35:43 UTC Artifacts will be uploaded here
2025-11-13 12:37:08 UTC ya make is running...
🟡 2025-11-13 14:48:47 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
42554 39729 0 1 2792 32

2025-11-13 14:49:02 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-11-13 14:58:19 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
60 (only retried tests) 43 0 0 0 17

🟢 2025-11-13 14:58:26 UTC Build successful.
🟡 2025-11-13 14:58:47 UTC ydbd size 2.3 GiB changed* by +157.0 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 501d619 merge: 286095e diff diff %
ydbd size 2 479 158 304 Bytes 2 479 319 056 Bytes +157.0 KiB +0.006%
ydbd stripped size 520 975 280 Bytes 521 003 760 Bytes +27.8 KiB +0.005%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 13, 2025

2025-11-13 12:35:56 UTC Pre-commit check linux-x86_64-release-asan for 286095e has started.
2025-11-13 12:36:00 UTC Artifacts will be uploaded here
2025-11-13 12:37:30 UTC ya make is running...
🟡 2025-11-13 15:29:42 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
19026 18472 0 261 254 39

🟢 2025-11-13 15:29:52 UTC Build successful.
🟡 2025-11-13 15:30:20 UTC ydbd size 3.9 GiB changed* by +307.7 KiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 501d619 merge: 286095e diff diff %
ydbd size 4 135 191 208 Bytes 4 135 506 328 Bytes +307.7 KiB +0.008%
ydbd stripped size 1 522 525 384 Bytes 1 522 655 784 Bytes +127.3 KiB +0.009%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add protobuf definitions for ExportToFs / ImportFromFs

3 participants