Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blob: allow driver.Bucket to upload data by pulling from an io.Reader instead of pushing to an io.Writer #3245

Closed
pgavlin opened this issue May 5, 2023 · 7 comments · Fixed by #3248

Comments

@pgavlin
Copy link

pgavlin commented May 5, 2023

Please use a title starting with the name of the affected package, or "all",
followed by a colon, followed by a short summary of the feature request.
Example: blob/gcsblob: add support for more blobbing.

Is your feature request related to a problem? Please describe.

Unless I am mistaken, the only way to upload data to a driver.Bucket is using driver.Bucket.NewTypedWriter. While this is fine for most scenarios--users can just io.Copy an io.Reader into the driver.Writer--it can cause performance problems in cases where the API of the underlying storage works in terms of io.Reader. In those cases, the driver must use something like io.Pipe to bridge between the driver.Writer interface and the io.Reader expected by the underlying storage API. For an example of the problems this can cause, see #2807, which details allocation volume issues for s3blob.

Describe the solution you'd like

We really need a method whose semantics are "upload the data from this reader" s.t. we can pass the reader directly to the underlying storage. Maybe we could add Upload methods to blob.Bucket and driver.Bucket:

func (b *blob.Bucket) Upload(ctx context.Context, key string, body io.Reader, opts *blob.WriterOptions)
func Upload(ctx context.Context, key string, body io.Reader, opts *driver.WriterOptions)

For parity, we could also add Download methods:

func (b *blob.Bucket) Download(ctx context.Context, w io.Writer, key string, opts *blob.ReaderOptions)
func Download(ctx context.Context, w io.Writer, key string, opts *driver.ReaderOptions)

Describe alternatives you've considered

blob.Bucket.Writer already exposes ReadFrom. My first thought was that we could change the implementation of that method to check whether or not the underlying writer is an io.ReaderFrom and if so call its ReadFrom method. Unfortunately, I don't think that really helps, as I think the semantics of blob.ReadFrom are such that it can be called repeatedly.

Additional context

In order to work around this issue, we've specialized some of out internal code to use s3manager.Uploader directly instead of bucket.NewWriter because profiles indicated that we were spending a disproportionate amount of time zeroing and collecting temporary buffers due to #2807. The workaround isn't too annoying, but we were pretty shocked when we discovered that we were hitting #2807 and that it was costing us so much CPU time.

@vangent
Copy link
Contributor

vangent commented May 7, 2023

Thanks for the report. I will see what I can do for this, I like the general idea and think it's feasible.

If I send a PR, will you able to verify the performance gain? I can verify correctness fairly well, but not the underlying memory churn you're seeing under load.

pgavlin added a commit to pgavlin/go-cloud that referenced this issue May 8, 2023
Add Upload and Download methods to blob.Bucket and optional
Uploader and Downloader interfaces to driver. If a driver.Bucket
implements those interfaces, Upload and Download will call through those
interfaces. This allows backends that can implement pull-style uploads
and push-style downloads more efficiently than push-style uploads and
pull-style downloads to do so.

These changes include an implementation of driver.Uploader for
s3blob.Bucket. That implementation allows s3blob.Bucket to avoid
allocating temporary buffers if the input is an io.ReaderAt and an
io.Seeker. This can dramatically reduce allocation volume for services
that upload large amounts of data to S3.

Fixes google#3245
Fixes google#2807
@pgavlin
Copy link
Author

pgavlin commented May 8, 2023

If I send a PR, will you able to verify the performance gain? I can verify correctness fairly well, but not the underlying memory churn you're seeing under load.

Definitely. FWIW, I just pushed #3247 with some ideas--happy to take that further or to review and validate your changes!

@vangent
Copy link
Contributor

vangent commented May 9, 2023

@pgavlin can you give #3248 a try? I haven't added tests or looked at most of the drivers, but I think it should work. Happy for comments on the Pull Request as well.

@vangent
Copy link
Contributor

vangent commented May 24, 2023

Ping @pgavlin

@vangent
Copy link
Contributor

vangent commented Jun 1, 2023

@pgavlin ....? Bueller?

@pgavlin
Copy link
Author

pgavlin commented Jun 2, 2023

Hey, sorry for not responding sooner--I've bene pretty slammed with other work. I'll give #3248 a go tomorrow morning.

@pgavlin
Copy link
Author

pgavlin commented Jun 7, 2023

Okay, finally got to this. Yes, I think that #3248 would work for us. We have a benchmark that I've used to measure the relative allocation overhead of our current implementation that calls s3manager.Upload, an implementation that uses the baseline gocloud with ReadFrom, and an implementation that uses #3248 and Upload. The results show that #3248 eliminates the 5MB allocation overhead incurred by baseline gocloud. Full stats are below.

goos: darwin
goarch: arm64
                                                 │   base.txt   │            gocloud.txt             │        gocloud-upload.txt         │
                                                 │    sec/op    │   sec/op     vs base               │   sec/op     vs base              │
S3StorageClient/PutObject/Uncompressed/256_B-10    293.0µ ± 10%   528.9µ ± 6%  +80.52% (p=0.002 n=6)   307.1µ ± 1%       ~ (p=0.065 n=6)
S3StorageClient/PutObject/Uncompressed/512_B-10    310.4µ ± 12%   541.3µ ± 1%  +74.37% (p=0.002 n=6)   322.8µ ± 1%  +3.99% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/1.0_kB-10   317.0µ ±  7%   558.5µ ± 7%  +76.17% (p=0.002 n=6)   331.3µ ± 1%  +4.52% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/2.0_kB-10   328.1µ ±  6%   579.4µ ± 4%  +76.58% (p=0.002 n=6)   344.0µ ± 1%  +4.83% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/4.1_kB-10   352.8µ ±  4%   583.2µ ± 1%  +65.28% (p=0.002 n=6)   373.7µ ± 1%  +5.91% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/8.2_kB-10   408.4µ ±  1%   625.4µ ± 1%  +53.13% (p=0.002 n=6)   413.0µ ± 1%  +1.12% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/16_kB-10    502.0µ ±  0%   710.6µ ± 1%  +41.53% (p=0.002 n=6)   505.5µ ± 1%  +0.68% (p=0.015 n=6)
S3StorageClient/PutObject/Uncompressed/33_kB-10    753.7µ ±  1%   953.1µ ± 1%  +26.46% (p=0.002 n=6)   772.4µ ± 0%  +2.49% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/66_kB-10    1.284m ±  0%   1.460m ± 9%  +13.74% (p=0.002 n=6)   1.282m ± 1%       ~ (p=0.699 n=6)
S3StorageClient/PutObject/Uncompressed/131_kB-10   2.350m ±  0%   2.559m ± 1%   +8.89% (p=0.002 n=6)   2.347m ± 0%  -0.17% (p=0.041 n=6)
S3StorageClient/PutObject/Uncompressed/262_kB-10   4.696m ±  0%   4.945m ± 0%   +5.31% (p=0.002 n=6)   4.680m ± 1%       ~ (p=0.065 n=6)
S3StorageClient/PutObject/Uncompressed/524_kB-10   8.835m ±  0%   9.265m ± 1%   +4.88% (p=0.002 n=6)   8.830m ± 6%       ~ (p=0.818 n=6)
S3StorageClient/PutObject/Uncompressed/1.0_MB-10   17.05m ±  0%   18.03m ± 0%   +5.75% (p=0.002 n=6)   17.02m ± 1%       ~ (p=0.180 n=6)
S3StorageClient/PutObject/Uncompressed/2.1_MB-10   33.19m ±  0%   34.57m ± 0%   +4.16% (p=0.002 n=6)   33.17m ± 1%       ~ (p=0.699 n=6)
S3StorageClient/PutObject/Uncompressed/4.2_MB-10   66.37m ±  0%   69.29m ± 0%   +4.39% (p=0.002 n=6)   66.90m ± 1%  +0.79% (p=0.004 n=6)
S3StorageClient/PutObject/Uncompressed/8.4_MB-10   124.9m ±  2%   131.7m ± 2%   +5.43% (p=0.002 n=6)   125.3m ± 1%       ~ (p=0.485 n=6)
S3StorageClient/PutObject/Compressed/256_B-10      298.8µ ±  2%   535.3µ ± 2%  +79.16% (p=0.002 n=6)   310.4µ ± 1%  +3.89% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/512_B-10      315.7µ ±  3%   553.9µ ± 1%  +75.45% (p=0.002 n=6)   324.2µ ± 1%  +2.69% (p=0.026 n=6)
S3StorageClient/PutObject/Compressed/1.0_kB-10     326.0µ ±  5%   562.1µ ± 1%  +72.42% (p=0.002 n=6)   334.4µ ± 0%       ~ (p=0.132 n=6)
S3StorageClient/PutObject/Compressed/2.0_kB-10     342.8µ ±  2%   577.1µ ± 1%  +68.35% (p=0.002 n=6)   349.0µ ± 2%  +1.80% (p=0.026 n=6)
S3StorageClient/PutObject/Compressed/4.1_kB-10     376.1µ ±  1%   600.1µ ± 1%  +59.55% (p=0.002 n=6)   374.6µ ± 1%       ~ (p=0.394 n=6)
S3StorageClient/PutObject/Compressed/8.2_kB-10     415.7µ ±  1%   641.5µ ± 3%  +54.32% (p=0.002 n=6)   412.6µ ± 0%  -0.74% (p=0.009 n=6)
S3StorageClient/PutObject/Compressed/16_kB-10      510.3µ ±  0%   724.4µ ± 0%  +41.97% (p=0.002 n=6)   504.2µ ± 1%  -1.19% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/33_kB-10      758.2µ ±  1%   960.3µ ± 7%  +26.66% (p=0.002 n=6)   773.8µ ± 1%  +2.06% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/66_kB-10      1.274m ±  0%   1.463m ± 1%  +14.83% (p=0.002 n=6)   1.284m ± 0%  +0.76% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/131_kB-10     2.352m ±  0%   2.565m ± 5%   +9.07% (p=0.002 n=6)   2.345m ± 1%  -0.31% (p=0.041 n=6)
S3StorageClient/PutObject/Compressed/262_kB-10     4.688m ±  0%   4.974m ± 0%   +6.11% (p=0.002 n=6)   4.694m ± 5%       ~ (p=0.485 n=6)
S3StorageClient/PutObject/Compressed/524_kB-10     8.845m ±  0%   9.284m ± 1%   +4.96% (p=0.002 n=6)   8.805m ± 0%  -0.46% (p=0.004 n=6)
S3StorageClient/PutObject/Compressed/1.0_MB-10     17.05m ±  0%   18.04m ± 1%   +5.78% (p=0.002 n=6)   17.09m ± 0%       ~ (p=0.818 n=6)
S3StorageClient/PutObject/Compressed/2.1_MB-10     33.22m ±  0%   34.71m ± 2%   +4.47% (p=0.002 n=6)   33.36m ± 0%  +0.42% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/4.2_MB-10     66.00m ±  4%   67.96m ± 0%        ~ (p=0.065 n=6)   65.84m ± 0%  -0.24% (p=0.041 n=6)
S3StorageClient/PutObject/Compressed/8.4_MB-10     125.9m ±  1%   128.3m ± 0%   +1.91% (p=0.002 n=6)   125.9m ± 0%       ~ (p=0.699 n=6)
geomean                                            2.215m         2.887m       +30.33%                 2.242m       +1.22%

                                                 │   base.txt   │              gocloud.txt              │         gocloud-upload.txt         │
                                                 │     B/op     │     B/op       vs base                │     B/op      vs base              │
S3StorageClient/PutObject/Uncompressed/256_B-10    861.8Ki ± 0%   5965.4Ki ± 0%  +592.23% (p=0.002 n=6)   866.2Ki ± 0%  +0.51% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/512_B-10    861.9Ki ± 0%   5965.7Ki ± 0%  +592.11% (p=0.002 n=6)   866.3Ki ± 0%  +0.51% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/1.0_kB-10   863.2Ki ± 0%   5967.3Ki ± 0%  +591.32% (p=0.002 n=6)   867.7Ki ± 0%  +0.53% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/2.0_kB-10   866.5Ki ± 0%   5970.7Ki ± 0%  +589.06% (p=0.002 n=6)   871.0Ki ± 0%  +0.52% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/4.1_kB-10   874.2Ki ± 0%   5975.3Ki ± 0%  +583.54% (p=0.002 n=6)   876.5Ki ± 0%  +0.27% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/8.2_kB-10   882.2Ki ± 0%   5984.4Ki ± 0%  +578.37% (p=0.002 n=6)   884.9Ki ± 0%  +0.31% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/16_kB-10    898.7Ki ± 0%   6002.0Ki ± 0%  +567.88% (p=0.002 n=6)   901.3Ki ± 0%  +0.30% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/33_kB-10    953.2Ki ± 0%   6055.7Ki ± 0%  +535.29% (p=0.002 n=6)   955.0Ki ± 0%  +0.18% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/66_kB-10    1.009Mi ± 0%    5.989Mi ± 0%  +493.51% (p=0.002 n=6)   1.012Mi ± 0%  +0.24% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/131_kB-10   1.165Mi ± 0%    6.143Mi ± 0%  +427.24% (p=0.002 n=6)   1.167Mi ± 0%  +0.18% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/262_kB-10   1.495Mi ± 0%    6.474Mi ± 0%  +333.09% (p=0.002 n=6)   1.496Mi ± 0%  +0.08% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/524_kB-10   2.118Mi ± 0%    7.093Mi ± 0%  +234.85% (p=0.002 n=6)   2.119Mi ± 0%  +0.06% (p=0.009 n=6)
S3StorageClient/PutObject/Uncompressed/1.0_MB-10   3.370Mi ± 0%    8.345Mi ± 0%  +147.64% (p=0.002 n=6)   3.370Mi ± 0%       ~ (p=0.699 n=6)
S3StorageClient/PutObject/Uncompressed/2.1_MB-10   5.839Mi ± 0%   10.814Mi ± 0%   +85.21% (p=0.002 n=6)   5.840Mi ± 0%       ~ (p=0.132 n=6)
S3StorageClient/PutObject/Uncompressed/4.2_MB-10   5.827Mi ± 0%    5.832Mi ± 0%    +0.10% (p=0.002 n=6)   5.829Mi ± 0%  +0.04% (p=0.009 n=6)
S3StorageClient/PutObject/Uncompressed/8.4_MB-10   10.98Mi ± 0%    10.99Mi ± 0%    +0.04% (p=0.004 n=6)   10.99Mi ± 0%       ~ (p=0.310 n=6)
S3StorageClient/PutObject/Compressed/256_B-10      862.8Ki ± 0%   5965.1Ki ± 0%  +591.38% (p=0.002 n=6)   866.0Ki ± 0%  +0.38% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/512_B-10      863.1Ki ± 0%   5965.9Ki ± 0%  +591.22% (p=0.002 n=6)   866.4Ki ± 0%  +0.39% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/1.0_kB-10     863.3Ki ± 0%   5967.3Ki ± 0%  +591.20% (p=0.002 n=6)   868.6Ki ± 0%  +0.61% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/2.0_kB-10     867.0Ki ± 0%   5971.0Ki ± 0%  +588.67% (p=0.002 n=6)   870.9Ki ± 0%  +0.45% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/4.1_kB-10     871.8Ki ± 0%   5975.8Ki ± 0%  +585.44% (p=0.002 n=6)   876.7Ki ± 0%  +0.56% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/8.2_kB-10     881.3Ki ± 0%   5984.8Ki ± 0%  +579.05% (p=0.002 n=6)   885.0Ki ± 0%  +0.41% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/16_kB-10      898.5Ki ± 0%   6002.1Ki ± 0%  +568.02% (p=0.002 n=6)   901.6Ki ± 0%  +0.34% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/33_kB-10      953.4Ki ± 0%   6055.8Ki ± 0%  +535.21% (p=0.002 n=6)   955.0Ki ± 0%  +0.17% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/66_kB-10      1.009Mi ± 0%    5.989Mi ± 0%  +493.42% (p=0.002 n=6)   1.012Mi ± 0%  +0.24% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/131_kB-10     1.166Mi ± 0%    6.142Mi ± 0%  +426.82% (p=0.002 n=6)   1.167Mi ± 0%  +0.11% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/262_kB-10     1.495Mi ± 0%    6.474Mi ± 0%  +332.94% (p=0.002 n=6)   1.496Mi ± 0%       ~ (p=0.093 n=6)
S3StorageClient/PutObject/Compressed/524_kB-10     2.118Mi ± 0%    7.093Mi ± 0%  +234.85% (p=0.002 n=6)   2.119Mi ± 0%  +0.04% (p=0.015 n=6)
S3StorageClient/PutObject/Compressed/1.0_MB-10     3.370Mi ± 0%    8.345Mi ± 0%  +147.66% (p=0.002 n=6)   3.370Mi ± 0%       ~ (p=0.818 n=6)
S3StorageClient/PutObject/Compressed/2.1_MB-10     5.839Mi ± 0%   10.814Mi ± 0%   +85.20% (p=0.002 n=6)   5.838Mi ± 0%  -0.03% (p=0.004 n=6)
S3StorageClient/PutObject/Compressed/4.2_MB-10     10.84Mi ± 0%    15.82Mi ± 0%   +45.93% (p=0.002 n=6)   10.84Mi ± 0%       ~ (p=0.180 n=6)
S3StorageClient/PutObject/Compressed/8.4_MB-10     21.03Mi ± 0%    30.97Mi ± 0%   +47.29% (p=0.002 n=6)   21.03Mi ± 0%       ~ (p=0.699 n=6)
geomean                                            1.638Mi         7.045Mi       +330.09%                 1.642Mi       +0.23%

                                                 │  base.txt   │            gocloud.txt             │        gocloud-upload.txt         │
                                                 │  allocs/op  │  allocs/op   vs base               │  allocs/op   vs base              │
S3StorageClient/PutObject/Uncompressed/256_B-10     644.0 ± 0%    715.5 ± 0%  +11.10% (p=0.002 n=6)    695.5 ± 0%  +8.00% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/512_B-10     643.0 ± 1%    715.0 ± 0%  +11.20% (p=0.002 n=6)    695.0 ± 0%  +8.09% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/1.0_kB-10    643.0 ± 0%    715.5 ± 0%  +11.28% (p=0.002 n=6)    695.0 ± 0%  +8.09% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/2.0_kB-10    643.0 ± 1%    716.0 ± 0%  +11.35% (p=0.002 n=6)    695.0 ± 0%  +8.09% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/4.1_kB-10    646.5 ± 0%    716.0 ± 0%  +10.75% (p=0.002 n=6)    696.0 ± 0%  +7.66% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/8.2_kB-10    645.5 ± 0%    716.0 ± 0%  +10.92% (p=0.002 n=6)    696.0 ± 0%  +7.82% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/16_kB-10     644.5 ± 0%    716.0 ± 0%  +11.09% (p=0.002 n=6)    695.0 ± 0%  +7.84% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/33_kB-10     646.0 ± 0%    717.0 ± 0%  +10.99% (p=0.002 n=6)    695.0 ± 0%  +7.59% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/66_kB-10     647.0 ± 0%    716.5 ± 0%  +10.74% (p=0.002 n=6)    697.0 ± 0%  +7.73% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/131_kB-10    648.0 ± 0%    716.0 ± 0%  +10.49% (p=0.002 n=6)    698.0 ± 0%  +7.72% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/262_kB-10    712.0 ± 0%    781.5 ± 0%   +9.76% (p=0.002 n=6)    763.0 ± 0%  +7.16% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/524_kB-10    718.0 ± 0%    783.0 ± 0%   +9.05% (p=0.002 n=6)    767.0 ± 0%  +6.82% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/1.0_MB-10    722.0 ± 0%    787.5 ± 0%   +9.07% (p=0.002 n=6)    770.0 ± 0%  +6.65% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/2.1_MB-10    725.0 ± 0%    790.0 ± 0%   +8.97% (p=0.002 n=6)    774.0 ± 0%  +6.76% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/4.2_MB-10    723.0 ± 0%    783.0 ± 0%   +8.30% (p=0.002 n=6)    774.0 ± 0%  +7.05% (p=0.002 n=6)
S3StorageClient/PutObject/Uncompressed/8.4_MB-10   2.602k ± 0%   2.664k ± 0%   +2.40% (p=0.002 n=6)   2.655k ± 0%  +2.08% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/256_B-10       644.5 ± 0%    715.5 ± 0%  +11.02% (p=0.002 n=6)    695.0 ± 0%  +7.84% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/512_B-10       644.0 ± 0%    715.0 ± 0%  +11.02% (p=0.002 n=6)    695.5 ± 0%  +8.00% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/1.0_kB-10      643.5 ± 0%    715.5 ± 0%  +11.19% (p=0.002 n=6)    696.0 ± 0%  +8.16% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/2.0_kB-10      644.0 ± 0%    716.0 ± 0%  +11.18% (p=0.002 n=6)    695.0 ± 0%  +7.92% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/4.1_kB-10      644.0 ± 0%    716.5 ± 0%  +11.26% (p=0.002 n=6)    696.0 ± 0%  +8.07% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/8.2_kB-10      645.0 ± 0%    716.0 ± 0%  +11.01% (p=0.002 n=6)    696.0 ± 0%  +7.91% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/16_kB-10       644.0 ± 0%    716.0 ± 0%  +11.18% (p=0.002 n=6)    695.0 ± 0%  +7.92% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/33_kB-10       646.0 ± 0%    717.0 ± 0%  +10.99% (p=0.002 n=6)    695.0 ± 0%  +7.59% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/66_kB-10       647.0 ± 0%    716.0 ± 0%  +10.66% (p=0.002 n=6)    697.0 ± 0%  +7.73% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/131_kB-10      649.0 ± 0%    715.5 ± 0%  +10.25% (p=0.002 n=6)    698.0 ± 0%  +7.55% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/262_kB-10      712.0 ± 0%    782.0 ± 0%   +9.83% (p=0.002 n=6)    762.5 ± 0%  +7.09% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/524_kB-10      717.0 ± 0%    782.5 ± 0%   +9.14% (p=0.002 n=6)    767.5 ± 0%  +7.04% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/1.0_MB-10      721.0 ± 0%    787.5 ± 0%   +9.22% (p=0.002 n=6)    771.0 ± 0%  +6.93% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/2.1_MB-10      725.0 ± 0%    790.0 ± 0%   +8.97% (p=0.002 n=6)    772.5 ± 0%  +6.55% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/4.2_MB-10      728.5 ± 0%    796.0 ± 0%   +9.27% (p=0.002 n=6)    776.5 ± 0%  +6.59% (p=0.002 n=6)
S3StorageClient/PutObject/Compressed/8.4_MB-10     2.610k ± 0%   2.680k ± 0%   +2.72% (p=0.002 n=6)   2.655k ± 0%  +1.72% (p=0.002 n=6)
geomean                                             728.6         800.5        +9.87%                  780.8       +7.17%

If you're interested, I can isolate the benchmark from our internal wrappers and toss it in a gist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants