Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c/distributed_kv_stm: prevent invalidated iterators during list #16421

Merged
merged 1 commit into from
Feb 1, 2024

Conversation

rockwotj
Copy link
Contributor

@rockwotj rockwotj commented Feb 1, 2024

The read lock held in list is for snapshots, and doesn't protect
against concurrent writes. If we yield when copying the kv_data, then
re-lookup the position we were at when resuming so that we don't access
an invalidated iterator.

FIXES: #16402

Backports Required

  • none - not a bug fix
  • none - this is a backport
  • none - issue does not exist in previous branches
  • none - papercut/not impactful enough to backport
  • v23.3.x
  • v23.2.x
  • v23.1.x

Release Notes

Bug Fixes

  • Fix invalidate iterator access when yielding control to the scheduler in data transforms debug endpoint.

The read lock held in `list` is for snapshots, and doesn't protect
against concurrent writes. If we yield when copying the kv_data, then
re-lookup the position we were at when resuming so that we don't access
an invalidated iterator.

Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
@rockwotj rockwotj self-assigned this Feb 1, 2024
@rockwotj rockwotj added the area/wasm WASM Data Transforms label Feb 1, 2024
@github-actions github-actions bot removed the area/wasm WASM Data Transforms label Feb 1, 2024
Copy link
Contributor

@bharathv bharathv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat..

@vbotbuildovich
Copy link
Collaborator

new failures in https://buildkite.com/redpanda/redpanda/builds/44572#018d62e8-fb2d-4fca-b622-d401ce1c568d:

"rptest.tests.e2e_shadow_indexing_test.EndToEndShadowIndexingTestWithDisruptions.test_write_with_node_failures.cloud_storage_type=CloudStorageType.ABS"

@rockwotj rockwotj merged commit 697d4a1 into redpanda-data:dev Feb 1, 2024
17 checks passed
@rockwotj rockwotj deleted the yield branch February 1, 2024 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI Failure (NodeCrash) in DataTransformsTest.test_tracked_offsets_cleaned_up
3 participants