Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Race in Concurrent Snapshot Delete and Create #37612

Conversation

Projects
None yet
4 participants
@original-brownbear
Copy link
Member

commented Jan 18, 2019

  • The repo id was determined wrong when the delete picked up on an in progress snapshot
    • NOTE: This solution is still a best-effort fix and there's a slight chance of running into concurrency issues here
      when multiple create and delete requests for the same snapshot name are happening concurrently, but these require a sequence
      of multiple cluster state updates between the changed method reading the genId and submitting its cluster state update task
  • Added test reproduced the issue reliably in about 50% of runs
  • Closes #37581
Fix Race in Concurrent Snapshot Delete and Create
* The repo id was determined wrong when the delete picked up on an in progress snapshot
  * NOTE: This solution is still a best-effort fix and there's a slight chance of running into concurrency issues here
when multiple create and delete requests for the same snapshot name are happening concurrently, but these require a sequence
of multiple cluster state updates between the changed method reading the genId and submitting its cluster state update task
* Added test reproduced the issue reliably in about 50% of runs
* Closes #37581
@elasticmachine

This comment has been minimized.

Copy link

commented Jan 18, 2019

@ywelsch
Copy link
Contributor

left a comment

LGTM. The new snapshots test infrastructure is already paying off.

@original-brownbear

This comment has been minimized.

Copy link
Member Author

commented Jan 21, 2019

@ywelsch thanks!

@original-brownbear original-brownbear merged commit 3a3f5b3 into elastic:master Jan 21, 2019

7 checks passed

CLA Commit author has signed the CLA
Details
elasticsearch-ci-1 Build finished.
Details
elasticsearch-ci-2 Build finished.
Details
elasticsearch-ci/default-distro Build finished.
Details
elasticsearch-ci/docbldesx Build finished.
Details
elasticsearch-ci/oss-distro-docs Build finished.
Details
elasticsearch-ci/packaging-sample Build finished.
Details

@original-brownbear original-brownbear deleted the original-brownbear:fix-snapshot-delete-concurrency branch Jan 21, 2019

@colings86 colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request Feb 28, 2019

Snapshot Stability Fixes
* Backport of various snapshot stability fixes from `master` to `6.7`
* Includes elastic#38368, elastic#38025 and elastic#37612

original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request Feb 28, 2019

Snapshot Stability Fixes
* Backport of various snapshot stability fixes from `master` to `6.7`
* Includes elastic#38368, elastic#38025 and elastic#37612

original-brownbear added a commit that referenced this pull request Mar 1, 2019

Snapshot Stability Fixes (#39502)
* Snapshot Stability Fixes

* Backport of various snapshot stability fixes from `master` to `6.7`
* Includes #38368, #38025 and #37612

original-brownbear added a commit that referenced this pull request Mar 4, 2019

Snapshot Stability Fixes (#39550)
* Backport of various snapshot stability fixes from `master` to `6.7` making the snapshot logic in `6.7` equivalent to that in `master` functionally
* Includes #38368, #38025 and #37612
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.