Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Do not check for object existence when deleting repository index files #31680

Merged
merged 1 commit into from
Jun 29, 2018
Merged

Do not check for object existence when deleting repository index files #31680

merged 1 commit into from
Jun 29, 2018

Conversation

tlrx
Copy link
Member

@tlrx tlrx commented Jun 29, 2018

Before deleting a repository index generation file, BlobStoreRepository
checks for the existence of the file and then deletes it. We can save
a request here by using BlobContainer.deleteBlobIgnoringIfNotExists()
which ignores error when deleting a file that does not exist.

Since there is no way with S3 to know if a non versioned file existed
before being deleted, this pull request also changes S3BlobContainer so
that it now implements deleteBlobIgnoringIfNotExists(). It will now save
one more request (blobExist?) when appropriate. The tests and fixture
have been modified to conform the S3 API that always returns a 204/NO
CONTENT HTTP response on deletions.

@tlrx
Do not check for object existence when deleting repository index files
472ab56 
Before deleting a repository index generation file, BlobStoreRepository
checks for the existence of the file and then deletes it. We can save
a request here by using BlobContainer.deleteBlobIgnoringIfNotExists()
 which ignores error when deleting a file that does not exist.

Since there is no way with S3 to know if a non versioned file existed
before being deleted, this pull request also changes S3BlobContainer so
that it now implements deleteBlobIgnoringIfNotExists(). It will now save
 one more request (blobExist?) when appropriate. The tests and fixture
 have been modified to conform the S3 API that always returns a 204/NO
 CONTENT HTTP response on deletions.
@tlrx tlrx added >enhancement review :Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs v7.0.0 v6.4.0 labels Jun 29, 2018
@tlrx tlrx requested a review from ywelsch June 29, 2018 09:00
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

ywelsch
ywelsch approved these changes Jun 29, 2018
View reviewed changes
Copy link
Contributor

@ywelsch ywelsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Awesome stuff!

@tlrx tlrx merged commit 8fa0629 into elastic:master Jun 29, 2018
tlrx added a commit that referenced this pull request Jun 29, 2018
@tlrx
Do not check for object existence when deleting repository index files (
e057f1a 
#31680)

Before deleting a repository index generation file, BlobStoreRepository
checks for the existence of the file and then deletes it. We can save
a request here by using BlobContainer.deleteBlobIgnoringIfNotExists()
 which ignores error when deleting a file that does not exist.

Since there is no way with S3 to know if a non versioned file existed
before being deleted, this pull request also changes S3BlobContainer so
that it now implements deleteBlobIgnoringIfNotExists(). It will now save
 one more request (blobExist?) when appropriate. The tests and fixture
 have been modified to conform the S3 API that always returns a 204/NO
 CONTENT HTTP response on deletions.
@tlrx
Copy link
Member Author

tlrx commented Jun 29, 2018

Thanks @ywelsch

dnhatn added a commit that referenced this pull request Jun 29, 2018
@dnhatn
Merge branch 'master' into ccr
52d9012 
* master:
  Do not check for object existence when deleting repository index files (#31680)
  Remove extra check for object existence in repository-gcs read object (#31661)
  Support multiple system store types (#31650)
  [Test] Clean up some repository-s3 tests (#31601)
  [Docs] Use capital letters in section headings (#31678)
  [DOCS] Add PQL language Plugin (#31237)
  Merge AzureStorageService and AzureStorageServiceImpl and clean up tests (#31607)
  TEST: Fix test task invocation (#31657)
  Revert "[TEST] Mute failing tests in NativeRealmInteg and ReservedRealmInteg"
  Fix RealmInteg test failures
  Extend allowed characters for grok field names (#21745) (#31653)
  [DOCS] Fix licensing API details (#31667)
  [TEST] Mute failing tests in NativeRealmInteg and ReservedRealmInteg
  Fix CreateSnapshotRequestTests Failure (#31630)
  Configurable password hashing algorithm/cost (#31234)
  [TEST] Mute failing NamingConventionsTaskIT tests
  [DOCS] Replace CONFIG_DIR with ES_PATH_CONF (#31635)
  Core: Require all actions have a Task (#31627)
dnhatn added a commit that referenced this pull request Jun 29, 2018
@dnhatn
Merge branch '6.x' into ccr-6.x
71cc05d 
* 6.x:
  Do not check for object existence when deleting repository index files (#31680)
  Remove extra check for object existence in repository-gcs read object (#31661)
  [Test] Clean up some repository-s3 tests (#31601)
  Merge AzureStorageService and AzureStorageServiceImpl and clean up tests (#31607)
  [Docs] Use capital letters in section headings (#31678)
  [DOCS] Add PQL language Plugin (#31237)
  [DOCS] Fix licensing API details (#31667)
  Fix CreateSnapshotRequestTests Failure (#31630)
  Add Create Snapshot to High-Level Rest Client (#31215)
  Add MultiSearchTemplate support to High Level Rest client (#31662)
  SQL: Refactor package names of sql-proto and sql-shared-proto projects (#31622)
  [TEST] Mute failing NamingConventionsTaskIT tests
  [DOCS] Replace CONFIG_DIR with ES_PATH_CONF (#31635)
  Add test for low-level client round-robin behaviour (#31616)
  HLRest: Fix test for explain API
  Add explain API to high-level REST client (#31387)
jasontedor added a commit to martijnvg/elasticsearch that referenced this pull request Jul 1, 2018
@jasontedor
Merge remote-tracking branch 'elastic/ccr' into pr/31581
5cd19ad 
* elastic/ccr: (30 commits)
  Enable setting client path prefix to / (elastic#30119)
  [DOCS] Secure settings specified per node (elastic#31621)
  has_parent builder: exception message/param fix (elastic#31182)
  TEST: Randomize soft-deletes settings (elastic#31585)
  Mute 'Test typed keys parameter for suggesters' as we await a fix.
  Build test: Thread linger
  Fix gradle4.8 deprecation warnings (elastic#31654)
  Mute FileRealmTests#testAuthenticateCaching with an @AwaitsFix.
  Mute TransportChangePasswordActionTests#testIncorrectPasswordHashingAlgorithm with an @AwaitsFix.
  Build: Fix naming conventions task   (elastic#31681)
  Introduce a Hashing Processor (elastic#31087)
  Do not check for object existence when deleting repository index files (elastic#31680)
  Remove extra check for object existence in repository-gcs read object (elastic#31661)
  Support multiple system store types (elastic#31650)
  [Test] Clean up some repository-s3 tests (elastic#31601)
  [Docs] Use capital letters in section headings (elastic#31678)
  muted tests that will be replaced by the shard follow task refactoring: elastic#31581
  [DOCS] Add PQL language Plugin (elastic#31237)
  Merge AzureStorageService and AzureStorageServiceImpl and clean up tests (elastic#31607)
  TEST: Fix test task invocation (elastic#31657)
  ...
@tlrx tlrx deleted the use-delete-if-not-exist branch September 17, 2018 07:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ywelsch ywelsch ywelsch approved these changes

Assignees
No one assigned
Labels
:Distributed/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >enhancement v6.4.0 v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@tlrx @elasticmachine @ywelsch @colings86