Fixes default chunk size for Azure repositories #22577
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Before, the default chunk size for Azure repositories was -1 bytes, which meant that if the chunk_size was not set on the Azure repository, nor as a node setting, then no data files would get written as part of the snapshot (because the BlobStoreRepository's PartSliceStream does not know how to process negative chunk sizes). This commit fixes the default chunk size for Azure repositories to be the same as the maximum chunk size. This commit also adds tests for both the Azure and Google Cloud repositories to ensure only valid chunk sizes can be set.
abeyad
added
:Plugin Repository Azure
:Distributed/Snapshot/Restore
jasontedor
approved these changes
Jan 12, 2017
| // default chunk size | ||
| AzureRepository azureRepository = azureRepository(Settings.EMPTY); | ||
| assertEquals(AzureStorageService.MAX_CHUNK_SIZE, azureRepository.chunkSize()); | ||
| assertThat(azureRepository.chunkSize().getBytes(), greaterThanOrEqualTo(0L)); |
There was a problem hiding this comment.
I don't understand the value of this assertion given the assertion on the previous line?
There was a problem hiding this comment.
Good catch, this is sort of left over as the assertion is wrong (the chunk size should always be > 0, not >= 0). I'll remove this and the one below
| RepositoryMetaData repositoryMetaData = new RepositoryMetaData("repo", GoogleCloudStorageRepository.TYPE, Settings.EMPTY); | ||
| ByteSizeValue chunkSize = GoogleCloudStorageRepository.getSetting(GoogleCloudStorageRepository.CHUNK_SIZE, repositoryMetaData); | ||
| assertEquals(GoogleCloudStorageRepository.MAX_CHUNK_SIZE, chunkSize); | ||
| assertThat(chunkSize.getBytes(), greaterThanOrEqualTo(0L)); |
|
Thanks @jasontedor |
abeyad
pushed a commit
that referenced
this pull request
Jan 12, 2017
Before, the default chunk size for Azure repositories was -1 bytes, which meant that if the chunk_size was not set on the Azure repository, nor as a node setting, then no data files would get written as part of the snapshot (because the BlobStoreRepository's PartSliceStream does not know how to process negative chunk sizes). This commit fixes the default chunk size for Azure repositories to be the same as the maximum chunk size. This commit also adds tests for both the Azure and Google Cloud repositories to ensure only valid chunk sizes can be set. Closes #22513
abeyad
pushed a commit
that referenced
this pull request
Jan 12, 2017
Before, the default chunk size for Azure repositories was -1 bytes, which meant that if the chunk_size was not set on the Azure repository, nor as a node setting, then no data files would get written as part of the snapshot (because the BlobStoreRepository's PartSliceStream does not know how to process negative chunk sizes). This commit fixes the default chunk size for Azure repositories to be the same as the maximum chunk size. This commit also adds tests for both the Azure and Google Cloud repositories to ensure only valid chunk sizes can be set. Closes #22513
|
@abeyad Can you relabel this as v5.2.0? |
|
@jasontedor done |
clintongormley
added
:Distributed/Snapshot/Restore
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before, the default chunk size for Azure repositories was
-1 bytes, which meant that if the chunk_size was not set on
the Azure repository, nor as a node setting, then no data
files would get written as part of the snapshot (because
the BlobStoreRepository's PartSliceStream does not know
how to process negative chunk sizes).
This commit fixes the default chunk size for Azure repositories
to be the same as the maximum chunk size. This commit also
adds tests for both the Azure and Google Cloud repositories to
ensure only valid chunk sizes can be set.
Closes #22513