-
Notifications
You must be signed in to change notification settings - Fork 406
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue 4967: (SLTS) Add createWithContent overload that creates object and writes to it in a single call. #5455
Issue 4967: (SLTS) Add createWithContent overload that creates object and writes to it in a single call. #5455
Conversation
59e980e
to
370ef37
Compare
…ed by write in a single call. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
370ef37
to
a4c4f81
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…e-4967-SLTS-optimize-chunk-creation
bindings/src/main/java/io/pravega/storage/extendeds3/ExtendedS3ChunkStorage.java
Outdated
Show resolved
Hide resolved
...st/src/test/java/io/pravega/segmentstore/server/host/NonAppendExtendedS3IntegrationTest.java
Outdated
Show resolved
Hide resolved
.../storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/AsyncBaseChunkStorage.java
Show resolved
Hide resolved
.../storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/AsyncBaseChunkStorage.java
Outdated
Show resolved
Hide resolved
...store/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/BaseChunkStorage.java
Outdated
Show resolved
Hide resolved
...store/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/BaseChunkStorage.java
Show resolved
Hide resolved
segmentstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkStorage.java
Outdated
Show resolved
Hide resolved
...entstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/SystemJournal.java
Show resolved
Hide resolved
...entstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/SystemJournal.java
Show resolved
Hide resolved
...entstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/SystemJournal.java
Outdated
Show resolved
Hide resolved
Codecov Report
@@ Coverage Diff @@
## master #5455 +/- ##
============================================
+ Coverage 84.71% 84.76% +0.04%
- Complexity 13870 13886 +16
============================================
Files 917 917
Lines 51359 51407 +48
Branches 5301 5307 +6
============================================
+ Hits 43510 43576 +66
+ Misses 4957 4942 -15
+ Partials 2892 2889 -3 Continue to review full report at Codecov.
|
bindings/src/test/java/io/pravega/storage/extendeds3/S3FileSystemImpl.java
Outdated
Show resolved
Hide resolved
segmentstore/storage/src/main/java/io/pravega/segmentstore/storage/chunklayer/ChunkStorage.java
Outdated
Show resolved
Hide resolved
…e-4967-SLTS-optimize-chunk-creation Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> # Conflicts: # bindings/src/main/java/io/pravega/storage/extendeds3/ExtendedS3ChunkStorage.java # bindings/src/test/java/io/pravega/storage/extendeds3/S3FileSystemImpl.java # bindings/src/test/java/io/pravega/storage/extendeds3/S3ProxyImpl.java
Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
…e-4967-SLTS-optimize-chunk-creation
bindings/src/main/java/io/pravega/storage/extendeds3/ExtendedS3ChunkStorage.java
Show resolved
Hide resolved
...st/src/test/java/io/pravega/segmentstore/server/host/NonAppendExtendedS3IntegrationTest.java
Show resolved
Hide resolved
…e-4967-SLTS-optimize-chunk-creation
…hub.com/sachin-j-joshi/pravega into issue-4967-SLTS-optimize-chunk-creation
… and writes to it in a single call. (pravega#5455) Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call. In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages. Fix mock S3 implementation and add more tests specifically for Extended S3 with no append mode. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
… and writes to it in a single call. (pravega#5455) Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call. In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages. Fix mock S3 implementation and add more tests specifically for Extended S3 with no append mode. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com> Signed-off-by: Tom Kaitchuck <tom.kaitchuck@emc.com>
… and writes to it in a single call. (pravega#5455) Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call. In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages. Fix mock S3 implementation and add more tests specifically for Extended S3 with no append mode. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
… and writes to it in a single call. (pravega#5455) Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call. In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages. Fix mock S3 implementation and add more tests specifically for Extended S3 with no append mode. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
… and writes to it in a single call. (pravega#5455) Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call. In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages. Fix mock S3 implementation and add more tests specifically for Extended S3 with no append mode. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
… and writes to it in a single call. (pravega#5455) Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call. In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages. Fix mock S3 implementation and add more tests specifically for Extended S3 with no append mode. Signed-off-by: Sachin Joshi <sachin.joshi@emc.com> Co-authored-by: Andrei Paduroiu <andreipaduroiu@users.noreply.github.com>
Cherry-picks following changes 0.9 Issue 5518: (SLTS) Fix Integer overflow. (#5520) Issue 5606: BoundedInputStream::markSupported() should always return false.. (#5615) Issue 5597 : ExtendedS3ChunkStorage and ExtendedS3Storage should close S3Client. (#5598) Issue 5535: (SegmentStore) Refactoring ExtendedS3 Test Mocks (#5537) Issue 5456: (SLTS) - Handle possible failure during deletion of metadata keys can cause version mismatch in next attempt (#5457) Issue 5458: (SLTS) For read operation, read all chunks in parallel. (#5459) Issue 4967: (SLTS) Add createWithContent overload that creates object and writes to it in a single call. (#5455) Issue 5664: SLTS - fix possible thread visibility issues.. (#5665) Issue 5475: Eliminate unnecessary pessimistic checks. (#5662) Issue 5570: (SLTS) AsyncBaseChunkStorage latency should not include time spent in waiting to be executed. (#5661) Issue 4967: (SLTS) No need to check last chunk length for claimOwnership for no append mode. (#5663) Issue 5673: (SLTS) Fix wrong GC config (#5674) Issue 5737: (SLTS) Improve metrics (#5746) Issue 5460: (SLTS) Add read index virtual block entries to metadata. (#5461) Issue 5772: (SLTS) - BaseMetadataStore does not evict entries from buffer. (#5773) Issue 5798: (SLTS) Fix close in GarbageCollector, ExtendedS3ChunkStorage and HDFSChunkStorage. (#5800) Issue 5788: (SLTS) Remove ACL operations from ExtendedS3ChunkStorage (#5790) Issue 5808: SLTS - Fix Preconditions checks to include enough information in message. (#5809) Issue 5853: SLTS - BaseMetadataStore.get does not return deep copy when loading from store. #5858 Issue 5866: (SLTS) ReadIndexCache stats not reported. (#5867) Signed-off-by: Sachin Joshi <sachin.joshi@emc.com>
Signed-off-by: Sachin Joshi sachin.joshi@emc.com
Change log description
Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call.
In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages.
Fix mock S3 implementation and add more tests specifically for Extended S3 with no append mode.
Purpose of the change
Fixes #4967
What the code does
Provide another overload to ChunkStorage.create that takes initial data to write and writes data in a single call.
In addition it avoids using Range header for put or post which may not be supported by all S3 compliant storages.
Fix mock S3 implementation accordingly.
How to verify it
Unit tests and other tests with Extended S3 with no append mode should pass.