Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce WriteAll allocs #10810

Merged
merged 2 commits into from
Nov 3, 2020
Merged

Reduce WriteAll allocs #10810

merged 2 commits into from
Nov 3, 2020

Commits on Nov 2, 2020

  1. Reduce WriteAll allocs 

    WriteAll saw 127GB allocs in a 5 minute timeframe for 4MiB buffers used by `io.CopyBuffer` even if they are pooled.

Since all writers appear to write byte buffers, just send those instead and write directly.

The files are opened through the `os` package so they have no special properties anyway.

This removes the alloc and copy for each operation.

REST sends content length so a precise alloc can be made.

This should probably be released alongside an API upgrade since we now send and rely on content-length.
    @klauspost
    klauspost committed Nov 2, 2020
    Configuration menu
    Copy the full SHA
    402db8e View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into reduce-writeall-allocs

    @harshavardhana
    harshavardhana committed Nov 2, 2020
    Configuration menu
    Copy the full SHA
    da5cb01 View commit details
    Browse the repository at this point in the history