Skip to content

24-2: schemeshard: reject operations with too big local tx commit (#6760) - #6850

Merged
ijon merged 1 commit into
ydb-platform:stable-24-2from
ijon:merge/24-2/fix-schemeshard-reject-too-massive-operations
Jul 22, 2024
Merged

ijon merged 1 commit into
ydb-platform:stable-24-2from
ijon:merge/24-2/fix-schemeshard-reject-too-massive-operations

Conversation

@ijon

@ijon ijon commented Jul 18, 2024

Copy link
Copy Markdown
Contributor

merged ebee36a from main

Add commit redo size check for successfully ignited operations as a precaution measure to avoid infinite loop of schemeshard hitting local tx commit redo size limit, restarting, attempting to propose persisted operation again, hitting commit redo size limit again, restarting and so on.

This could happen with inherently massive operations such as copy-tables used as a starting step of database export/backup.

Coping large number of tables with huge number of partitions can result in so large TTxOperationPropose local transaction that its size would hit the limit imposed by the tablet executor. Tablet violating that limit is considered broken and will be immediately stopped. See ydb/core/tablet_flat/flat_executor.cpp, NTabletFlatExecutor::TExecutor::ExecuteTransaction().

KIKIMR-21751

Changelog entry

...

Changelog category

  • New feature
  • Experimental feature
  • Improvement
  • Performance improvement
  • Bugfix
  • Backward incompatible change
  • Documentation (changelog entry is not required)
  • Not for changelog (changelog entry is not required)

Additional information

...

@ijon
ijon requested a review from a team as a code owner July 18, 2024 20:20
@ijon ijon linked an issue Jul 18, 2024 that may be closed by this pull request
@github-actions

github-actions Bot commented Jul 18, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-18 20:22:37 UTC Pre-commit check for a809519 has started.
⚪ 2024-07-18 20:22:39 UTC Build linux-x86_64-release-asan is running...
🔴 2024-07-18 21:33:33 UTC Build failed. see the build logs.
🔴 2024-07-18 21:35:00 UTC Tests run skipped.

@github-actions

github-actions Bot commented Jul 18, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-18 20:23:48 UTC Pre-commit check for a809519 has started.
⚪ 2024-07-18 20:23:50 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-07-18 21:30:49 UTC Build failed. see the build logs.

@github-actions

github-actions Bot commented Jul 18, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-18 20:24:02 UTC Pre-commit check for a809519 has started.
⚪ 2024-07-18 20:24:05 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-07-18 21:40:26 UTC Build failed. see the build logs.
🔴 2024-07-18 21:41:56 UTC Tests run skipped.

@ijon ijon added the rebase-and-check Rebase PR with the current base branch and check label Jul 20, 2024
@github-actions github-actions Bot removed the rebase-and-check Rebase PR with the current base branch and check label Jul 20, 2024
@github-actions

github-actions Bot commented Jul 20, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-20 12:08:43 UTC Pre-commit check for a809519 has started.
⚪ 2024-07-20 12:08:47 UTC Build linux-x86_64-release-asan is running...
🔴 2024-07-20 13:20:56 UTC Build failed. see the build logs.
🔴 2024-07-20 13:22:24 UTC Tests run skipped.

@github-actions

github-actions Bot commented Jul 20, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-20 12:08:46 UTC Pre-commit check for a809519 has started.
⚪ 2024-07-20 12:08:48 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-07-20 13:16:16 UTC Build failed. see the build logs.

@github-actions

github-actions Bot commented Jul 20, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-20 12:11:20 UTC Pre-commit check for a809519 has started.
⚪ 2024-07-20 12:11:23 UTC Build linux-x86_64-relwithdebinfo is running...
🔴 2024-07-20 13:30:33 UTC Build failed. see the build logs.
🔴 2024-07-20 13:32:05 UTC Tests run skipped.

…form#6760)

Add commit redo size check for successfully ignited operations as a precaution measure to avoid infinite loop of schemeshard hitting local tx commit redo size limit, restarting, attempting to propose persisted operation again, hitting commit redo size limit again, restarting and so on.

This could happen with inherently massive operations such as copy-tables used as a starting step of database export/backup.

Coping large number of tables with huge number of partitions can result in so large TTxOperationPropose local transaction that its size would hit the limit imposed by the tablet executor. Tablet violating that limit is considered broken and will be immediately stopped.
See ydb/core/tablet_flat/flat_executor.cpp, NTabletFlatExecutor::TExecutor::ExecuteTransaction().

KIKIMR-21751
@ijon
ijon force-pushed the merge/24-2/fix-schemeshard-reject-too-massive-operations branch from 954379f to 9262662 Compare July 20, 2024 19:13
@github-actions

github-actions Bot commented Jul 20, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-20 19:16:38 UTC Pre-commit check for 0d1cb58 has started.
⚪ 2024-07-20 19:16:40 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-07-20 20:22:48 UTC Build successful.
⚪ 2024-07-20 20:24:29 UTC Tests are running...
🔴 2024-07-20 21:47:18 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9496 9433 0 5 32 26

@github-actions

github-actions Bot commented Jul 20, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-20 19:16:52 UTC Pre-commit check for 0d1cb58 has started.
⚪ 2024-07-20 19:16:54 UTC Build linux-x86_64-release-asan is running...
🟢 2024-07-20 20:38:20 UTC Build successful.
⚪ 2024-07-20 20:40:09 UTC Tests are running...
🔴 2024-07-20 22:40:44 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9175 9024 0 19 58 74

@github-actions

github-actions Bot commented Jul 20, 2024 •

Copy link
Copy Markdown
Contributor

⚪ 2024-07-20 19:16:52 UTC Pre-commit check for 0d1cb58 has started.
⚪ 2024-07-20 19:16:55 UTC Build linux-x86_64-release-clang14 is running...
🟢 2024-07-20 20:38:08 UTC Build successful.

@ijon
ijon merged commit 80bf6ef into ydb-platform:stable-24-2 Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

schemeshard: reject too massive operations

2 participants