New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed optimizations #18

Merged
merged 2 commits into from Mar 1, 2017

Conversation

Projects
None yet
2 participants
@maoe
Member

maoe commented Feb 12, 2017

This PR does two speed optimizations:

  • Use bytestring builder rather than String to pretty print events
  • Avoid repetitive reconstruction of event decoder

The improvement is rather drastic:

% ls -lh ghc-events.eventlog
-rw-r--r--  1 maoe  staff    54M Feb 10 18:30 ghc-events.eventlog
% cat ghc-events.eventlog > /dev/null
% ./ghc-events.master inc ghc-events.eventlog > master.out +RTS -sstderr
 207,094,188,704 bytes allocated in the heap
   1,387,436,192 bytes copied during GC
          92,904 bytes maximum residency (2 sample(s))
          44,920 bytes maximum slop
               2 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0     397464 colls,     0 par    9.747s   9.987s     0.0000s    0.0012s
  Gen  1         2 colls,     0 par    0.001s   0.001s     0.0004s    0.0007s

  INIT    time    0.000s  (  0.002s elapsed)
  MUT     time   73.893s  ( 75.288s elapsed)
  GC      time    9.748s  (  9.988s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time   83.642s  ( 85.278s elapsed)

  %GC     time      11.7%  (11.7% elapsed)

  Alloc rate    2,802,625,175 bytes per MUT second

  Productivity  88.3% of total user, 88.3% of total elapsed
% ./ghc-events.speed-opt inc ghc-events.eventlog > speed-opt.out +RTS -sstderr
  13,985,347,472 bytes allocated in the heap
      11,922,160 bytes copied during GC
          85,936 bytes maximum residency (2 sample(s))
          43,616 bytes maximum slop
               2 MB total memory in use (0 MB lost due to fragmentation)

                                     Tot time (elapsed)  Avg pause  Max pause
  Gen  0     25140 colls,     0 par    0.502s   0.514s     0.0000s    0.0007s
  Gen  1         2 colls,     0 par    0.001s   0.001s     0.0004s    0.0007s

  INIT    time    0.000s  (  0.002s elapsed)
  MUT     time    4.009s  (  4.384s elapsed)
  GC      time    0.502s  (  0.515s elapsed)
  EXIT    time    0.000s  (  0.000s elapsed)
  Total   time    4.513s  (  4.901s elapsed)

  %GC     time      11.1%  (10.5% elapsed)

  Alloc rate    3,488,077,971 bytes per MUT second

  Productivity  88.9% of total user, 89.5% of total elapsed
% diff -w master.out speed-opt.out; echo $?
0

The behavior is almost the same except for the leading space in timestamps. master uses printf "%9d" but this branch replaced it with BB.word64Dec so no leading space. Please let me know if this is a problem.

Also this branch is based off of the travis branch. I can rebase onto master if that's better.

@maoe

This comment has been minimized.

Show comment
Hide comment
@maoe

maoe Feb 12, 2017

Member

Rebased onto the latest maoe/travis.

Member

maoe commented Feb 12, 2017

Rebased onto the latest maoe/travis.

@kvelicka

This comment has been minimized.

Show comment
Hide comment
@kvelicka

kvelicka Feb 15, 2017

Collaborator

Looks promising! I'll have a proper look at it later this week.

Collaborator

kvelicka commented Feb 15, 2017

Looks promising! I'll have a proper look at it later this week.

@kvelicka

This comment has been minimized.

Show comment
Hide comment
@kvelicka

kvelicka Feb 16, 2017

Collaborator

Yup, looks really good indeed! I'm happy with this going into master, unless @Mikolaj / @simonmar object.

Collaborator

kvelicka commented Feb 16, 2017

Yup, looks really good indeed! I'm happy with this going into master, unless @Mikolaj / @simonmar object.

@maoe

This comment has been minimized.

Show comment
Hide comment
@maoe

maoe Feb 21, 2017

Member

I forgot a whitespace in inc output. Pushed a fix for it. Let me know if it's better to squash the commit into 23116c3.

Member

maoe commented Feb 21, 2017

I forgot a whitespace in inc output. Pushed a fix for it. Let me know if it's better to squash the commit into 23116c3.

@kvelicka

This comment has been minimized.

Show comment
Hide comment
@kvelicka

kvelicka Feb 21, 2017

Collaborator

Yep, I think you might as well squash it.

Collaborator

kvelicka commented Feb 21, 2017

Yep, I think you might as well squash it.

@maoe

This comment has been minimized.

Show comment
Hide comment
@maoe

maoe Feb 21, 2017

Member

Done.

Member

maoe commented Feb 21, 2017

Done.

@maoe

This comment has been minimized.

Show comment
Hide comment
@maoe

maoe Feb 24, 2017

Member

Now that I got commit access and @kvelicka and @Mikolaj seem to be happy with this patch. I'd like to merge it if @simonmar doesn't object.

Member

maoe commented Feb 24, 2017

Now that I got commit access and @kvelicka and @Mikolaj seem to be happy with this patch. I'd like to merge it if @simonmar doesn't object.

maoe added some commits Feb 9, 2017

Switch to ByteString Builder to speed up ghc-events inc
This change speeds up ghc-events inc by 30%:

./ghc-events.master inc ghc-events.eventlog > /dev/null  69.63s user 0.60s system 99% cpu 1:10.33 total
./ghc-events.speed-opt inc ghc-events.eventlog > /dev/null  46.12s user 0.36s system 99% cpu 46.517 total
Avoid expensive decoder reconstruction by caching
This patch drastically improves the performance of ghc-events inc:

% cat ghc-events.eventlog > /dev/null
% time ./ghc-events.builder inc ghc-events.eventlog > builder.out
./ghc-events.builder inc ghc-events.eventlog > builder.out  46.54s user 0.70s system 99% cpu 47.293 total
% time ./ghc-events.cache inc ghc-events.eventlog > cache.out
./ghc-events.cache inc ghc-events.eventlog > cache.out  4.33s user 0.35s system 99% cpu 4.706 total
% diff builder.out cache.out; echo $?
0
@maoe

This comment has been minimized.

Show comment
Hide comment
@maoe

maoe Mar 1, 2017

Member

I'm going to rebase this onto current master. There has been no objections so far. I think I'll merge this once CI passed after rebase.

Member

maoe commented Mar 1, 2017

I'm going to rebase this onto current master. There has been no objections so far. I think I'll merge this once CI passed after rebase.

@maoe maoe merged commit d4232b3 into haskell:master Mar 1, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@maoe maoe deleted the maoe:speed-opt branch Mar 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment