Introduce basic benchmarks of the library. #64

qnikst · 2018-10-14T22:10:13Z

Introduce basic benchmarks of the log mechanism that allows
testing the overhead of the each feature added.

Example output:

Dump 10k messages (in a forked process):
baseline                                           0.001114674s
stdout<string                                      0.005194026s
stdout<text                                        0.004967748s
stdout<bytestring                                  0.002918046s
stderr<bytestring                                  0.01753503s
{stdout<>stderr}<bytestring                        0.017461043s
stdout<format<message                              0.009314225s
stdout<format<message{with callstack}              0.009744045s
stdout<format<message{with callstack:5}            0.018102189s
stdout<format<message{with callstack:50}           0.018013676s
stdout<bytestring<format<message                   0.009570143s
stdout<bytestring<rich-message<upgrade<message     0.134770134s

Currently benchmarks are implemented outside of the main
application as I had trouble to make cabal new-configure find
the build plan without --allow-newer. Also these benchmarks
may introduce dependencies on the other logging libraries in the
future.

At this point of time only the simplest benchmarks were introduced.
This benchmarks are testing sequencial dump of the logs with different
features added atop. Theoretically current benchmarks may be moved
to the application and criterion can be used for measuring time.
However I planned to make more long running benchmarks where criterion
may not shine, and such benchmarks may make CI time much larger for
not purpose.

Introduce basic benchmarks of the log mechanism that allows testing the overhead of the each feature added. Example output: Dump 10k messages (in a forked process): baseline 0.001114674s stdout<string 0.005194026s stdout<text 0.004967748s stdout<bytestring 0.002918046s stderr<bytestring 0.01753503s {stdout<>stderr}<bytestring 0.017461043s stdout<format<message 0.009314225s stdout<format<message{with callstack} 0.009744045s stdout<format<message{with callstack:5} 0.018102189s stdout<format<message{with callstack:50} 0.018013676s stdout<bytestring<format<message 0.009570143s stdout<bytestring<rich-message<upgrade<message 0.134770134s Currently benchmarks are implemented outside of the main application as I had trouble to make `cabal new-configure` find the build plan without `--allow-newer`. Also these benchmarks may introduce dependencies on the other logging libraries in the future. At this point of time only the simplest benchmarks were introduced. This benchmarks are testing sequencial dump of the logs with different features added atop. Theoretically current benchmarks may be moved to the application and criterion can be used for measuring time. However I planned to make more long running benchmarks where criterion may not shine, and such benchmarks may make CI time much larger for not purpose.

chshersh · 2018-10-16T12:45:46Z

@qnikst Thank you for these benchmarks! It's kinda strange that outputting ByteString to stderr is almost 10x times slower than outputting to stdout 🤔

I think it's okay to keep these particular benchmarks in separate package. They might be huge, so no need to move into existing package. I'm okay to not use criterion or any other benchmark library here.

chshersh · 2018-10-16T12:46:31Z

co-log-benchmark/co-log-benchmark.cabal

@@ -0,0 +1,32 @@
+name:                co-log-benchmark


Can you please update cabal.project file and add name of this package there?
And also update .travis.yml file for stack section to build this package on CI as well

Yep, will do

chshersh · 2018-10-16T12:46:59Z

co-log-benchmark/Setup.hs

@@ -0,0 +1,2 @@
+import Distribution.Simple
+main = defaultMain


I think that Setup.hs is not needed for cabal, it works without this file pretty fine

that require you to have cabal-install. But with Setup.hs, you may install library without cabal-install that what most of the distributions do.

@qnikst Could you, please, elaborate on this? How is it possible to install Haskell library without cabal-install and why one might want to do this?

you can always run:

ghc Setup.hs ./Setup configure ./Setup build ./Setup install

It will not install any dependency, but you don't need any additional tool to do that. Source distribution like Gentoo, NixOS do that, and other when you prepare a binary package. Because dependencies and package management is solved by the special tools.

chshersh · 2018-10-16T12:47:55Z

co-log-benchmark/Main.hs

+     Nothing -> pure ()
+     Just f  -> f
+
+-- | Measure the running time of the process.


Thanks again for the documentation!

chshersh · 2018-10-16T12:49:02Z

co-log-benchmark/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2018 Alexander Vershilov


I wonder, how should I update the LICENSE if I want to do something with the benchmarks (like, refactor or add new ones)? 🤔

oh.. that was autogenerated.

qnikst · 2018-10-16T12:56:57Z

Seems putBSLn from relude is hitting some lucky pass.

qnikst · 2018-10-16T20:25:55Z

I hope I've fixed everything that was needed.

chshersh

@qnikst Thank you very much for contributing these benchmarks!

chshersh

Overall looks good. But then I've noticed couple more things that should be fixed

chshersh · 2018-10-17T13:44:39Z

stack.yaml

@@ -3,6 +3,7 @@ resolver: lts-12.9
 packages:
  - co-log
  - co-log-core
+  - co-log-benchmark


You also need co-log-benchmark to .travis.yml here:

https://github.com/kowainik/co-log/blob/76964369c38152b2ad10bd3c452e0e7a0a90dfb9/.travis.yml#L79-L80

co-log-benchmark/co-log-benchmark.cabal

co-log-benchmark/Main.hs

chshersh · 2018-10-18T14:37:57Z

co-log-benchmark/co-log-benchmark.cabal

@@ -23,10 +22,11 @@ executable co-log-features-bench
  main-is:             Main.hs
  -- other-modules:
  -- other-extensions:
-  build-depends:       base >=4.10 && <4.13,
+  build-depends:       base-noprelude >=4.10 && <4.13,


Heh. Prelude in the co-log package is now in other-modules so base-noprelude trick won't work. Anyway, you didn't used relude before in co-log-benchmarks so I don't expect that addition of relude is required... Need to see CI failure to check what's wrong

I need to either use Prelude trick:

import Data.Semigroup import Prelude

so semigroup will be imported in all GHC versions, or use relude.

chshersh · 2018-10-18T14:39:13Z

co-log-benchmark/Main.hs

+  ( main
+  ) where
+
+import Data.Time.Clock


Suggested change

import Data.Time.Clock

import Data.Semigroup ((<>))

import Data.Time.Clock

I hope that should fix CI error. But the last commit with base-noprelude should be reverted. Alternative, you can add Prelude module to the co-log-benchmarks packages

This reverts commit b39e6e3.

chshersh

Ok, CI passes and it looks good 👍

Thank you again for the contribution!

qnikst mentioned this pull request Oct 14, 2018

Performance benchmarks #41

Closed

chshersh assigned qnikst Oct 16, 2018

chshersh requested review from chshersh and vrom911 October 16, 2018 12:27

chshersh added the benchmarks label Oct 16, 2018

chshersh added this to In progress in #2: Hacktoberfest (October, 2018) via automation Oct 16, 2018

chshersh added the Hacktoberfest https://hacktoberfest.digitalocean.com/ label Oct 16, 2018

chshersh reviewed Oct 16, 2018

View reviewed changes

Update LICENSE

73541ab

Include co-log-benchmark in the tests.

9ad7876

chshersh approved these changes Oct 17, 2018

View reviewed changes

#2: Hacktoberfest (October, 2018) automation moved this from In progress to Reviewer approved Oct 17, 2018

#2: Hacktoberfest (October, 2018) automation moved this from Reviewer approved to Needs review Oct 17, 2018

chshersh suggested changes Oct 17, 2018

View reviewed changes

chshersh and others added 3 commits October 17, 2018 16:54

Update co-log-benchmark/Main.hs

7006d24

Update co-log-benchmark/co-log-benchmark.cabal

620fc2a

Use base-noprelude in benchmarks.

b39e6e3

chshersh reviewed Oct 18, 2018

View reviewed changes

qnikst added 2 commits October 18, 2018 17:50

Revert "Use base-noprelude in benchmarks."

d024ca5

This reverts commit b39e6e3.

Add semigroup import.

ffba254

#2: Hacktoberfest (October, 2018) automation moved this from Needs review to Reviewer approved Oct 18, 2018

chshersh approved these changes Oct 18, 2018

View reviewed changes

chshersh merged commit 13ec457 into co-log:master Oct 18, 2018

#2: Hacktoberfest (October, 2018) automation moved this from Reviewer approved to Done: PR's Oct 18, 2018

chshersh removed this from Done: PR's in #2: Hacktoberfest (October, 2018) Sep 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce basic benchmarks of the library. #64

Introduce basic benchmarks of the library. #64

qnikst commented Oct 14, 2018 •

edited

Loading

chshersh commented Oct 16, 2018

chshersh Oct 16, 2018

qnikst Oct 16, 2018

chshersh Oct 16, 2018

qnikst Oct 16, 2018

chshersh Oct 16, 2018

qnikst Oct 16, 2018

chshersh Oct 16, 2018

chshersh Oct 16, 2018

qnikst Oct 16, 2018

qnikst commented Oct 16, 2018

qnikst commented Oct 16, 2018

chshersh left a comment

chshersh left a comment

chshersh Oct 17, 2018

chshersh Oct 18, 2018

qnikst Oct 18, 2018

chshersh Oct 18, 2018

chshersh Oct 18, 2018

chshersh left a comment

	import Data.Time.Clock
	import Data.Semigroup ((<>))
	import Data.Time.Clock

Introduce basic benchmarks of the library. #64

Introduce basic benchmarks of the library. #64

Conversation

qnikst commented Oct 14, 2018 • edited Loading

chshersh commented Oct 16, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qnikst commented Oct 16, 2018

qnikst commented Oct 16, 2018

chshersh left a comment

Choose a reason for hiding this comment

chshersh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chshersh left a comment

Choose a reason for hiding this comment

qnikst commented Oct 14, 2018 •

edited

Loading