cabal upload: use ByteString instead of String #1614

Merged
merged 1 commit into from Dec 14, 2013

Projects

None yet

4 participants

@trofi
trofi commented Dec 13, 2013

The difference is seen nicely on Raincat package (9MB tarball):
$ /usr/bin/time cabal upload --check Raincat-1.1.1.3.tar.gz
39.92user
0.38system
4:02.50elapsed
16%CPU (0avgtext+0avgdata 1855712maxresident)k

Insane amounts of used RAM (1.8GB) seem to stem from 'String'
inefficiency especially on 64-bit platforms where overhead is
about 16 times:

  • 97% of CPU time is taken by GC scans (according to +RTS -sstderr)
  • 9MB tarball would take ~150MBs of RAM, 100MB tarball would...

Attempt to pack them into HTTP headers leads to further growth.

The patch only changes underlying structure to ByteString.Lazy.Char8:
$ /usr/bin/time patched-cabal upload --check Raincat-1.1.1.3.tar.gz

0.25user
0.16system
4:28.61elapsed
0%CPU (0avgtext+0avgdata 66864maxresident)k

In short: 1.8GB -> 66MB RAM reduction + 16% -> 0% CPU usage.

Reported-by: "Mikhail S. Pobolovets" styx.mp@gmail.com
Signed-off-by: Sergei Trofimovich slyfox@gentoo.org

@trofi trofi cabal upload: use ByteString instead of String
The difference is seen nicely on Raincat package (9MB tarball):
    $ /usr/bin/time cabal upload --check Raincat-1.1.1.3.tar.gz
    39.92user
    0.38system
    4:02.50elapsed
    16%CPU (0avgtext+0avgdata 1855712maxresident)k

Insane amounts of used RAM (1.8GB) seem to stem from 'String'
inefficiency especially on 64-bit platforms where overhead is
about 16 times:

 - 97% of CPU time is taken by GC scans (according to +RTS -sstderr)
 - 9MB tarball would take ~150MBs of RAM, 100MB tarball would...

Attempt to pack them into HTTP headers leads to further growth.

The patch only changes underlying structure to ByteString.Lazy.Char8:
    $ /usr/bin/time patched-cabal upload --check Raincat-1.1.1.3.tar.gz

    0.25user
    0.16system
    4:28.61elapsed
    0%CPU (0avgtext+0avgdata 66864maxresident)k

In short: 1.8GB -> 66MB RAM reduction + 16% -> 0% CPU usage.

Reported-by: "Mikhail S. Pobolovets" <styx.mp@gmail.com>
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
dbb5088
@23Skidoo
Member

LGTM.

@tibbe
Member
tibbe commented Dec 13, 2013

LGTM

@styx
styx commented Dec 14, 2013

👍

@tibbe tibbe merged commit dbb5088 into haskell:master Dec 14, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment