Skip to content

compress/flate: add Reset method to let callers avoid garbage #6138

Closed
Closed
compress/flate: add Reset method to let callers avoid garbage#6138
Labels
FrozenDueToAgeGarbageCollector
Milestone
Go1.2

Description

@bradfitz
bradfitz
opened on Aug 13, 2013
Related to issue #6136, flate.NewWriter generates a lot of garbage which can't be
controlled by the calling application.

Options:

-- internal memory reuse in the flate package
-- Reset method (like bufio.Reader and bufio.Writer)
-- let buffers be passed in

Attachments:

  1. k0EYnXJ_LT.0.svg (15027 bytes)

Activity

  1. bradfitz commented on Aug 13, 2013

    @bradfitz
    bradfitz
    on Aug 13, 2013
    ContributorAuthor

    Comment 1:

    Each call to flate.NewWriter allocates 1.4 MB of memory, and Close doesn't reuse any of
it.

  2. rsc commented on Aug 13, 2013

    @rsc
    rsc
    on Aug 13, 2013
    Contributor

    Comment 2:

    Reset method please.
(Rationale: Having the reset method means that the buffer is attached to the Writer, not
magically moving around, so it gives the caller more control and also keeps the reuse
local to a specific package.)

  3. rsc commented on Aug 13, 2013

    @rsc
    rsc
    on Aug 13, 2013
    Contributor

    Comment 3:

    I wish it did generate a "log of garbage". That would make things easier.

  4. bradfitz commented on Aug 13, 2013

    @bradfitz
    bradfitz
    on Aug 13, 2013
    ContributorAuthor

    Comment 4:

    This issue was updated by revision c7d352c.

    TestOver65kFiles spends all its time garbage collecting.
Removing the 1.4 MB of allocations per each of the 65k
files brings this from 34 seconds to 0.23 seconds.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12894043

  5. robpike commented on Aug 16, 2013

    @robpike
    robpike
    on Aug 16, 2013
    Contributor

    Comment 5:

    Looks fixed to me. Reopen if you disagree.

    Status changed to Fixed.

  6. bradfitz commented on Aug 16, 2013

    @bradfitz
    bradfitz
    on Aug 16, 2013
    ContributorAuthor

    Comment 6:

    Per comment #2, this bug focused to be about adding a Reset method on flate.Writer so
callers can avoid 1.4 MB of garbage per writer if it affects them.

    Owner changed to @bradfitz.

    Status changed to Accepted.

  7. remyoudompheng commented on Aug 24, 2013

    @remyoudompheng
    remyoudompheng
    on Aug 24, 2013
    Contributor

    Comment 7:

    Should we create separate issues for compress/zlib and compress/gzip ? Most people never
use compress/flate directly, and the most common demand is probably Resetting a
gzip.Writer.

  8. remyoudompheng commented on Aug 29, 2013

    @remyoudompheng
    remyoudompheng
    on Aug 29, 2013
    Contributor

    Comment 8:

    This issue was closed by revision f5f0e40.

    Status changed to Fixed.

  9. bradfitz commented on Aug 30, 2013

    @bradfitz
    bradfitz
    on Aug 30, 2013
    ContributorAuthor

    Comment 9:

    This issue was updated by revision db12f9d.

    R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/13435043
  10. bradfitz
    added
    fixed
    GarbageCollector
    go1.2maybe
    on Aug 30, 2013
  11. bradfitz
    self-assigned this
    on Aug 30, 2013
  12. rsc
    added this to the Go1.2 milestone on Apr 14, 2015
  13. rsc
    removed
    go1.2maybe
    on Apr 14, 2015
  14. golang
    locked and limited conversation to collaborators on Jun 24, 2016
  15. gopherbot
    added
    FrozenDueToAge
    on Jun 24, 2016
  16. rsc
    unassigned
    bradfitz
    on Jun 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeGarbageCollector

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions