Skip to content

archive/zip: reading many small files allocates a lot of memory #14289

Description

@dziemba

archive/zip creates a new compress/flate.Reader for each file read from the archive. If there are many small files in an archive, this allocates a lot of memory.

compress/flate allows re-using of Readers, see https://codereview.appspot.com/97140043/

Using this in archive/zip reduces the amount of memory allocated/GCed. In a real-world scenario when reading ~ 500 small files from a ZIP archive this gives a speedup of 1.5x-2x.

Version: go1.6rc2

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions