-
Notifications
You must be signed in to change notification settings - Fork 405
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue 5460: (SLTS) Add read index virtual block entries to metadata. #5461
Merged
andreipaduroiu
merged 18 commits into
pravega:master
from
sachin-j-joshi:issue-5460-virtual-block-index
Feb 24, 2021
Merged
Issue 5460: (SLTS) Add read index virtual block entries to metadata. #5461
andreipaduroiu
merged 18 commits into
pravega:master
from
sachin-j-joshi:issue-5460-virtual-block-index
Feb 24, 2021
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
… pinned segments as all metadata is in memory already. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
…e-5460-virtual-block-index
andreipaduroiu
requested changes
Jan 21, 2021
...entstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkIterator.java
Outdated
Show resolved
Hide resolved
.../storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorage.java
Outdated
Show resolved
Hide resolved
.../storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorage.java
Outdated
Show resolved
Hide resolved
.../storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorage.java
Outdated
Show resolved
Hide resolved
...ge/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorageConfig.java
Outdated
Show resolved
Hide resolved
...e/storage/src/main/java/io/pravega/segmentstore/storage/metadata/ReadIndexBlockMetadata.java
Outdated
Show resolved
Hide resolved
...age/src/test/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorageTests.java
Outdated
Show resolved
Hide resolved
...age/src/test/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorageTests.java
Show resolved
Hide resolved
.../storage/src/test/java/io/pravega/segmentstore/storage/mocks/InMemorySimpleStorageTests.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
sachin-j-joshi
changed the title
Issue 5460: Add read index virtual block entries to metadata.
Issue 5460: (SLTS) Add read index virtual block entries to metadata.
Feb 3, 2021
…e-5460-virtual-block-index Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> # Conflicts: # segmentstore/storage/src/test/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorageTests.java # segmentstore/storage/src/test/java/io/pravega/segmentstore/storage/mocks/InMemorySimpleStorageTests.java
Codecov Report
@@ Coverage Diff @@
## master #5461 +/- ##
============================================
- Coverage 84.78% 84.78% -0.01%
- Complexity 13905 13925 +20
============================================
Files 917 918 +1
Lines 51517 51635 +118
Branches 5312 5321 +9
============================================
+ Hits 43677 43777 +100
- Misses 4955 4968 +13
- Partials 2885 2890 +5 Continue to review full report at Codecov.
|
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
...re/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/DefragmentOperation.java
Outdated
Show resolved
Hide resolved
...e/storage/src/main/java/io/pravega/segmentstore/storage/metadata/ReadIndexBlockMetadata.java
Show resolved
Hide resolved
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
…e-5460-virtual-block-index
…e-5460-virtual-block-index Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> # Conflicts: # segmentstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkedSegmentStorage.java # segmentstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ConcatOperation.java # segmentstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/DefragmentOperation.java
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
…e-5460-virtual-block-index
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
andreipaduroiu
approved these changes
Feb 24, 2021
sachin-j-joshi
added a commit
to sachin-j-joshi/pravega
that referenced
this pull request
Feb 24, 2021
…ravega#5461) Persistent custom skip list like sparse index entries. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
co-jo
pushed a commit
to co-jo/pravega
that referenced
this pull request
Mar 1, 2021
…ravega#5461) Persistent custom skip list like sparse index entries. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
co-jo
pushed a commit
to co-jo/pravega
that referenced
this pull request
Mar 9, 2021
…ravega#5461) Persistent custom skip list like sparse index entries. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
sachin-j-joshi
added a commit
that referenced
this pull request
Mar 19, 2021
Cherry-picks following changes 0.9 Issue 5518: (SLTS) Fix Integer overflow. (#5520) Issue 5606: BoundedInputStream::markSupported() should always return false.. (#5615) Issue 5597 : ExtendedS3ChunkStorage and ExtendedS3Storage should close S3Client. (#5598) Issue 5535: (SegmentStore) Refactoring ExtendedS3 Test Mocks (#5537) Issue 5456: (SLTS) - Handle possible failure during deletion of metadata keys can cause version mismatch in next attempt (#5457) Issue 5458: (SLTS) For read operation, read all chunks in parallel. (#5459) Issue 4967: (SLTS) Add createWithContent overload that creates object and writes to it in a single call. (#5455) Issue 5664: SLTS - fix possible thread visibility issues.. (#5665) Issue 5475: Eliminate unnecessary pessimistic checks. (#5662) Issue 5570: (SLTS) AsyncBaseChunkStorage latency should not include time spent in waiting to be executed. (#5661) Issue 4967: (SLTS) No need to check last chunk length for claimOwnership for no append mode. (#5663) Issue 5673: (SLTS) Fix wrong GC config (#5674) Issue 5737: (SLTS) Improve metrics (#5746) Issue 5460: (SLTS) Add read index virtual block entries to metadata. (#5461) Issue 5772: (SLTS) - BaseMetadataStore does not evict entries from buffer. (#5773) Issue 5798: (SLTS) Fix close in GarbageCollector, ExtendedS3ChunkStorage and HDFSChunkStorage. (#5800) Issue 5788: (SLTS) Remove ACL operations from ExtendedS3ChunkStorage (#5790) Issue 5808: SLTS - Fix Preconditions checks to include enough information in message. (#5809) Issue 5853: SLTS - BaseMetadataStore.get does not return deep copy when loading from store. #5858 Issue 5866: (SLTS) ReadIndexCache stats not reported. (#5867) Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Sachin Joshi sachin.joshi@emc.com
Change log description
Persistent custom skip list like sparse index entries.
Purpose of the change
Fixes #5460
What the code does
Add custom skip list index metadata using following logic
How to verify it
All tests must pass