blob: pass through reader/writer to WriteTo/ReadFrom if available
#3267
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The blob `Reader` and `Writer` implement `WriterTo` and `ReaderFrom`,
respectively, which is meant to avoid intermediary allocations and
copies (passing readers and readers through all the way down).
Unfortunately, the current implementation falls short and implements
those interfaces by making a local copy ala io.Copy.
Instead, reproduce the strategy in newer versions of `io.Copy`: reflect
on the reader/writer provided by the user, and if they implement these
interface, too, then call into that.
Local benchmarks show significant performance improvements:
```
│ /tmp/old │ /tmp/new │
│ sec/op │ sec/op vs base │
Memblob/BenchmarkRead-10 3.449µ ± 15% 1.919µ ± 2% -44.37% (p=0.000 n=10)
```
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Codecov Report
@@ Coverage Diff @@
## master #3267 +/- ##
==========================================
- Coverage 77.47% 77.42% -0.06%
==========================================
Files 103 103
Lines 13603 13607 +4
==========================================
- Hits 10539 10535 -4
- Misses 2333 2340 +7
- Partials 731 732 +1
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The blob
ReaderandWriterimplementWriterToandReaderFrom, respectively, which is meant to avoid intermediary allocations and copies (passing readers and readers through all the way down). Unfortunately, the current implementation falls short and implements those interfaces by making a local copy ala io.Copy.Instead, reproduce the strategy in newer versions of
io.Copy: reflect on the reader/writer provided by the user, and if they implement these interface, too, then call into that.Local benchmarks show significant performance improvements:
Most benchmarks use
ReadAllandWriteAll, which use slices directly and don't benefit from this change. I modified theBenchmarkReadto use abytes.Bufferand a call toio.Copyto exercise theWriteTopath. I believe we should change the remaining benchmarks to do the same. I don't thinkReadAll/WriteAllrepresent how most real-world applications interact with the library (piping readers/writer instead, but that assumption may be wrong!)