Investigate benchmark failures #7

RyanGlScott · 2016-05-15T23:28:08Z

I ran the benchmarking script, and unsurprisingly, lots of stuff broke (as also observed here). Here's what broke, why it broke, and if it's easy to fix. I've attempted to truncate each of the build logs to only contain the failing bits.

acid-state-0.14.0

Fixed in Fix compilation of benchmarks acid-state/acid-state#71
attoparsec-0.13.0.2

Fixed in Make cabal sdist package up the benchmarks correctly haskell/attoparsec#120
buffer-builder-0.2.4.1

Fixed in Include test.json in extra-source-files chadaustin/buffer-builder#10
cacophony-0.4.0

Seems to be fixed in HEAD?
case-insensitive-1.2.0.6

Fixed in Fix compilation of benchmarks basvandijk/case-insensitive#21

cases-0.1.3

+ stack bench --stack-yaml hacky-stack.yaml --only-dependencies
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package criterion-plus in known packages

--  Failure when adding dependencies:
      criterion-plus: needed (==0.1.*), stack configuration has no specified version (latest applicable is 0.1.3)
    needed for package cases-0.1.3

Recommended action: try adding the following to your extra-deps in /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cases-0.1.3/hacky-stack.yaml
- criterion-plus-0.1.3

You may also want to try the 'stack solver' command

cassava-0.4.5.0

Fixed in Include benchmarks/*.csv with cabal sdist haskell-hvr/cassava#117

cipher-aes-0.2.11

+ stack bench --stack-yaml hacky-stack.yaml --only-dependencies
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package crypto-cipher-benchmarks in known packages

--  Failure when adding dependencies:
      crypto-cipher-benchmarks: needed (>=0.0.4), stack configuration has no specified version (latest applicable is 0.0.5)
    needed for package cipher-aes-0.2.11

Recommended action: try adding the following to your extra-deps in /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cipher-aes-0.2.11/hacky-stack.yaml
- crypto-cipher-benchmarks-0.0.5

You may also want to try the 'stack solver' command

cipher-aes128-0.7.0.1

+ stack bench --stack-yaml hacky-stack.yaml --benchmark-arguments=--output /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/cipher-aes128-0.7.0.1.html --csv /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/cipher-aes128-0.7.0.1.csv --raw /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/cipher-aes128-0.7.0.1.crit
cipher-aes128-0.7.0.1: build (lib + bench)
Linking /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cipher-aes128-0.7.0.1/.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/setup/setup ...
Result of NI Intrinsics Test: True
Preprocessing library cipher-aes128-0.7.0.1...
In-place registering cipher-aes128-0.7.0.1...
Preprocessing benchmark 'bench' for cipher-aes128-0.7.0.1...
[3 of 3] Compiling Main             ( Benchmark/bench.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.5.0/build/bench/bench-tmp/Main.o )

/nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cipher-aes128-0.7.0.1/Benchmark/bench.hs:22:53:
    Ambiguous occurrence ‘encryptGCM’
    It could refer to either ‘Crypto.Cipher.AES128.encryptGCM’,
                             imported from ‘Crypto.Cipher.AES128’ at Benchmark/bench.hs:1:1-27
                          or ‘Crypto.Cipher.AES.encryptGCM’,
                             imported from ‘Crypto.Cipher.AES’ at Benchmark/bench.hs:2:1-24

cipher-blowfish-0.0.3

+ stack bench --stack-yaml hacky-stack.yaml --only-dependencies
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package crypto-cipher-benchmarks in known packages

--  Failure when adding dependencies:
      crypto-cipher-benchmarks: needed (-any), stack configuration has no specified version (latest applicable is 0.0.5)
    needed for package cipher-blowfish-0.0.3

Recommended action: try adding the following to your extra-deps in /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cipher-blowfish-0.0.3/hacky-stack.yaml
- crypto-cipher-benchmarks-0.0.5

You may also want to try the 'stack solver' command

cipher-camellia-0.0.2

+ stack bench --stack-yaml hacky-stack.yaml --only-dependencies
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package crypto-cipher-benchmarks in known packages

--  Failure when adding dependencies:
      crypto-cipher-benchmarks: needed (-any), stack configuration has no specified version (latest applicable is 0.0.5)
    needed for package cipher-camellia-0.0.2

Recommended action: try adding the following to your extra-deps in /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cipher-camellia-0.0.2/hacky-stack.yaml
- crypto-cipher-benchmarks-0.0.5

You may also want to try the 'stack solver' command

cipher-des-0.0.6

+ stack bench --stack-yaml hacky-stack.yaml --only-dependencies
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package crypto-cipher-benchmarks in known packages

--  Failure when adding dependencies:
      crypto-cipher-benchmarks: needed (-any), stack configuration has no specified version (latest applicable is 0.0.5)
    needed for package cipher-des-0.0.6

Recommended action: try adding the following to your extra-deps in /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cipher-des-0.0.6/hacky-stack.yaml
- crypto-cipher-benchmarks-0.0.5

You may also want to try the 'stack solver' command

cipher-rc4-0.1.4

+ stack bench --stack-yaml hacky-stack.yaml --only-dependencies
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package crypto-cipher-benchmarks in known packages

--  Failure when adding dependencies:
      crypto-cipher-benchmarks: needed (>=0.0.3), stack configuration has no specified version (latest applicable is 0.0.5)
    needed for package cipher-rc4-0.1.4

Recommended action: try adding the following to your extra-deps in /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/cipher-rc4-0.1.4/hacky-stack.yaml
- crypto-cipher-benchmarks-0.0.5

You may also want to try the 'stack solver' command

cmark-0.5.2

Opened Restore cmark benchmarks commercialhaskell/stackage#1458
compdata-0.10

Fixed in pa-ba/compdata@b155e5d
cryptohash-0.11.6

Fix pending Make benchmarks work with modern criterion vincenthz/hs-cryptohash#43
cryptol-2.3.0

Fixed in Put bench/data/*.cry in extra-source-files GaloisInc/cryptol#333

dbus-0.10.12

+ stack bench --stack-yaml hacky-stack.yaml --benchmark-arguments=--output /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/dbus-0.10.12.html --csv /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/dbus-0.10.12.csv --raw /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/dbus-0.10.12.crit
dbus-0.10.12: build (lib + bench)
Preprocessing library dbus-0.10.12...

/nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/dbus-0.10.12/lib/DBus/Client.hs:4:14: Warning:
    -XOverlappingInstances is deprecated: instead use per-instance pragmas OVERLAPPING/OVERLAPPABLE/OVERLAPS
In-place registering dbus-0.10.12...
Preprocessing benchmark 'dbus_benchmarks' for dbus-0.10.12...

/nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/dbus-0.10.12/benchmarks/DBusBenchmarks.hs:22:18:
    Could not find module ‘Criterion.Config’
    Perhaps you meant
      Criterion.Monad (from criterion-1.1.1.0@crite_4me1DTmD68lB0z7OFxfUdJ)
    Use -v to see a list of the files searched for.

fast-builder-0.0.0.4

Fixed in Benchmarks fail to run on GHC 7.10.3 takano-akio/fast-builder#3

Frames-0.1.2.1

+ stack bench --stack-yaml hacky-stack.yaml --benchmark-arguments=--output /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/Frames-0.1.2.1.html --csv /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/Frames-0.1.2.1.csv --raw /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-res/lts-5.16/jenkinsbuild-8/node-CREST_cutter/2016-05-15-15:02:30/Frames-0.1.2.1.crit
Frames-0.1.2.1: build (lib + bench)
Preprocessing library Frames-0.1.2.1...
In-place registering Frames-0.1.2.1...
Preprocessing benchmark 'insurance' for Frames-0.1.2.1...
[1 of 1] Compiling Main             ( benchmarks/InsuranceBench.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.5.0/build/insurance/insurance-tmp/Main.o )

/nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/Frames-0.1.2.1/benchmarks/InsuranceBench.hs:1:1:
    Exception when trying to run compile-time code:
      data/FL2.csv: openFile: does not exist (No such file or directory)
    Code: tableTypes "Ins" "data/FL2.csv"

Opened Benchmarks fail to compile on GHC 7.10.3 acowley/Frames#47

gitson-0.5.1

Seems to be fixed in HEAD
hashable-1.2.4.0

Fixed in Fix compilation of benchmarks haskell-unordered-containers/hashable#115
hindent-4.6.3

Fixed in https://github.com/chrisdone/hindent/pull/188
hledger-0.27

Fixed in Benchmarks fail to run in latest hledger from hackage simonmichael/hledger#342
http2-1.4.5

Seems to be fixed on HEAD
http-link-header-1.0.1

Fixed: Fix compilation of benchmarks sjshuck/http-link-header#4
jose-jwt-0.7

Seems to be fixed in HEAD
kdt-0.2.4

Opened Benchmarks fail to run on GHC 7.10.3 giogadi/kdt#4
lens-4.13

Will be fixed in lens-4.14.
lifted-base-0.2.3.6

Fixed in Make benchmarks compile with modern criterion basvandijk/lifted-base#30.
lucid-2.9.5

Fixed in Fix compilation of benchmarks chrisdone/lucid#52
machines-0.5.1

Will be fixed in machines-0.6.1
morte-1.4.2

Seems to be fixed in HEAD
picoparsec-0.1.2.3

Fixed in https://bitbucket.org/blamario/picoparsec/pull-requests/1/fix-benchmarks-on-ghc-7103/diff
stm-containers-0.2.10

Bump benchmark dependency upper bounds nikita-volkov/stm-containers#10
tar-0.5.0.3

Opened Benchmarks fail to run on GHC 7.10.3 haskell/tar#18
thyme-0.3.5.5

Fixed in liyang/thyme@0c83be2 ?

ttrie-0.1.2.1

+ stack bench --stack-yaml hacky-stack.yaml --only-dependencies
While constructing the BuildPlan the following exceptions were encountered:

--  While attempting to add dependency,
    Could not find package criterion-plus in known packages

--  Failure when adding dependencies:
      criterion-plus: needed (-any), stack configuration has no specified version (latest applicable is 0.1.3)
    needed for package ttrie-0.1.2.1

Recommended action: try adding the following to your extra-deps in /nfs/home/crest-team/jenkins_workspaces/cutter/workspace/benchmark_stackage/label/CREST_cutter/.bench-build/lts-5.16/ttrie-0.1.2.1/hacky-stack.yaml
- criterion-plus-0.1.3

You may also want to try the 'stack solver' command

unordered-containers-0.2.5.1

Fixed in Fix compilation of benchmarks haskell-unordered-containers/unordered-containers#134 and Fix benchmark compilation on stack haskell-unordered-containers/unordered-containers#135
vector-binary-instances-0.2.3.2

Due to a stack bug. See stack "errors" when building package with benchmark that depends on itself commercialhaskell/stack#2153
warp-3.2.2

Fixed in Make warp benchmarks build with modern criterion yesodweb/wai#551
web-routing-0.6.2

Fixed in philopon/web-routing@4edd987
xmlgen-0.6.2.1

Fix pending Make benchmarks compile with modern criterion skogsbaer/xmlgen#4
yi-rope-0.7.0.1

Fixed in Fix compilation of benchmarks yi-editor/yi-rope#6

Possibly spurious failures

Of the failures above, some of those are possibly our fault. We should take a closer look at the following:

vector-binary-instances-0.2.3.2 is failing due to a stack bug.

Two packages depend on criterion-plus, which is currently blacklisted from Stackage (I opened nikita-volkov/criterion-plus#2). These packages' benchmarks depend on it:

cases-0.1.3
ttrie-0.1.2.1

A lot of packages depend on crypto-cipher-benchmarks, which is currently blacklisted from Stackage. These packages' benchmarks depend on it:

cipher-aes-0.2.11
cipher-blowfish-0.0.3
cipher-camellia-0.0.2
cipher-des-0.0.6
cipher-rc4-0.1.4

A good amount of the database-related package give errors mentioning something to the effect of Connection refused. I should investigate to see if this is surmountable.

avers-0.0.15
hasql-0.19.6
hedis-0.6.10
mongoDB-2.0.10
redis-io-0.5.2
rethinkdb-2.2.0.4
rethinkdb-client-driver-0.0.22

These ones don't use criterion. We should probe for this at the beginning:

bzlib-conduit-0.2.1.3 (opened Fix compilation of benchmarks snoyberg/bzlib-conduit#2)
carray-0.1.6.3
distributed-process-0.5.5.1
sync-mht-0.3.8.4

The text was updated successfully, but these errors were encountered:

RyanGlScott changed the title ~~Investigate benchmark build failures~~ Investigate benchmark failures May 17, 2016

This was referenced Jun 8, 2016

Fix compilation of benchmarks haskell-unordered-containers/unordered-containers#134

Merged

Fix compilation of benchmarks snoyberg/bzlib-conduit#2

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Investigate benchmark failures #7

Investigate benchmark failures #7

RyanGlScott commented May 15, 2016 •

edited

Loading

Investigate benchmark failures #7

Investigate benchmark failures #7

Comments

RyanGlScott commented May 15, 2016 • edited Loading

Possibly spurious failures

RyanGlScott commented May 15, 2016 •

edited

Loading