stack bench does not work

[codygman]

While trying to make progress looking into #286 I ran into an issue running ghcide benchmarks.

Stack bench doesn't work as of 765967d:

[cody@nixos:~/code/ghcide]$ PAGER=cat git log --oneline -n1 && stack bench
765967d (HEAD -> master, origin/master, origin/HEAD) Add session-loader to hie.yaml (#714)
ghcide> benchmarks
Running 1 benchmarks...
Benchmark benchHist: RUNNING...
# git (for bench-hist/upstream/commitid)
fatal: ambiguous argument 'upstream/master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Error when running Shake build system:
  at want, called at bench/hist/Main.hs:92:3 in main:Main
* Depends on: all
  at need, called at bench/hist/Main.hs:113:7 in main:Main
* Depends on: bench-hist/results.csv
  at need, called at bench/hist/Main.hs:219:5 in main:Main
* Depends on: bench-hist/upstream/ghcide
  at &%>, called at bench/hist/Main.hs:149:3 in main:Main
* Depends on: bench-hist/upstream/ghcide bench-hist/upstream/ghc.path
  at readFile', called at bench/hist/Main.hs:155:19 in main:Main
  at need, called at src/Development/Shake/Internal/Derived.hs:118:15 in shake-0.18.5-9Mcvx0L4F69H98msVE3acd:Development.Shake.Internal.Derived
* Depends on: bench-hist/upstream/commitid
  at command, called at bench/hist/Main.hs:133:26 in main:Main
* Raised the exception:
Development.Shake.command, system command failed
Command line: git rev-list -n 1 upstream/master
Exit code: 128
Stderr:
fatal: ambiguous argument 'upstream/master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'


Benchmark benchHist: ERROR

--  While building package ghcide-0.2.0 using:
      /home/cody/.stack/setup-exe-cache/x86_64-linux-nix/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.5 --builddir=.stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1 bench benchHist
    Process exited with code: ExitFailure 1

Neither does cabal, it fails with the same error:

[nix-shell:/tmp/newfolder/haskell-language-server/ghcide]$ cabal bench 
Build profile: -w ghc-8.8.4 -O1
In order, the following will be built (use -v for more details):
 - ghcide-0.2.0 (bench:benchHist) (first run)
Preprocessing benchmark 'benchHist' for ghcide-0.2.0..
Building benchmark 'benchHist' for ghcide-0.2.0..
Running 1 benchmarks...
Benchmark benchHist: RUNNING...
# git (for bench-hist/upstream/commitid)
fatal: ambiguous argument 'upstream/master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Error when running Shake build system:
  at want, called at bench/hist/Main.hs:92:3 in main:Main
* Depends on: all
  at need, called at bench/hist/Main.hs:113:7 in main:Main
* Depends on: bench-hist/results.csv
  at need, called at bench/hist/Main.hs:219:5 in main:Main
* Depends on: bench-hist/upstream/ghcide
  at &%>, called at bench/hist/Main.hs:149:3 in main:Main
* Depends on: bench-hist/upstream/ghcide bench-hist/upstream/ghc.path
  at readFile', called at bench/hist/Main.hs:155:19 in main:Main
  at need, called at src/Development/Shake/Internal/Derived.hs:118:15 in shake-0.19.1-86XfWjXkHUSERkTkeZ7DVQ:Development.Shake.Internal.Derived
* Depends on: bench-hist/upstream/commitid
  at command, called at bench/hist/Main.hs:133:26 in main:Main
* Raised the exception:
Development.Shake.command, system command failed
Command line: git rev-list -n 1 upstream/master
Exit code: 128
Stderr:
fatal: ambiguous argument 'upstream/master': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'



Benchmark benchHist: ERROR
cabal: Benchmarks failed for bench:benchHist from ghcide-0.2.0.

[codygman]

I'm trying to workaround it by redownloading everything with stack and running stack bench now after trying cabal.

Edit: It didn't work either, updated description accordingly.

[codygman]

Ah, it pulls "upstream/master" and I don't have that in my git repo. Should probably put a git remote add upstream https://github.com/digital-asset/ghcide.git && git fetch --all before stack bench so things just work by default.

[codygman]

Hm, that's still not enough it seems. It looks like the benchmarks are expecting a branch called ghc.path?

[cody@nixos:~/code/ghcide]$ stack bench 
ghcide-0.2.0: unregistering (local file changes: bench/hist/Main.hs)
ghcide> configure (lib + exe + bench)
Configuring ghcide-0.2.0...
ghcide> build (lib + exe + bench)
Preprocessing library for ghcide-0.2.0..
Building library for ghcide-0.2.0..
Preprocessing executable 'ghcide-test-preprocessor' for ghcide-0.2.0..
Building executable 'ghcide-test-preprocessor' for ghcide-0.2.0..
Preprocessing executable 'ghcide' for ghcide-0.2.0..
Building executable 'ghcide' for ghcide-0.2.0..
Preprocessing executable 'ghcide-bench' for ghcide-0.2.0..
Building executable 'ghcide-bench' for ghcide-0.2.0..
Preprocessing benchmark 'benchHist' for ghcide-0.2.0..
Building benchmark 'benchHist' for ghcide-0.2.0..
[1 of 1] Compiling Main
Linking .stack-work/dist/x86_64-linux-nix/Cabal-2.4.0.1/build/benchHist/benchHist ...
ghcide> copy/register
Installing library in /home/cody/code/ghcide/.stack-work/install/x86_64-linux-nix/1ab077909885cd27895e5c1ead5f46712849129d46e5ebc602e96382ea9b3779/8.6.5/lib/x86_64-linux-ghc-8.6.5/ghcide-0.2.0-HMXIvSD1KinKqA0E6bpsEd
Installing executable ghcide-test-preprocessor in /home/cody/code/ghcide/.stack-work/install/x86_64-linux-nix/1ab077909885cd27895e5c1ead5f46712849129d46e5ebc602e96382ea9b3779/8.6.5/bin
Installing executable ghcide in /home/cody/code/ghcide/.stack-work/install/x86_64-linux-nix/1ab077909885cd27895e5c1ead5f46712849129d46e5ebc602e96382ea9b3779/8.6.5/bin
Installing executable ghcide-bench in /home/cody/code/ghcide/.stack-work/install/x86_64-linux-nix/1ab077909885cd27895e5c1ead5f46712849129d46e5ebc602e96382ea9b3779/8.6.5/bin
Registering library for ghcide-0.2.0..
ghcide> benchmarks
Running 1 benchmarks...
Benchmark benchHist: RUNNING...
xxxxxxxxxxxxxxxxxxxxxxxxxxxx  1: 
gitTHing is: upstream/master
# git (for bench-hist/upstream/commitid)
# git (for bench-hist/upstream/ghcide bench-hist/upstream/ghc.path)
Preparing worktree (detached HEAD 765967d)
HEAD is now at 765967d Add session-loader to hie.yaml (#714)
# stack (for bench-hist/upstream/ghcide bench-hist/upstream/ghc.path)
Error when running Shake build system:
  at want, called at bench/hist/Main.hs:92:3 in main:Main
* Depends on: all
  at need, called at bench/hist/Main.hs:113:7 in main:Main
* Depends on: bench-hist/results.csv
  at need, called at bench/hist/Main.hs:225:5 in main:Main
* Depends on: bench-hist/upstream/ghcide
  at &%>, called at bench/hist/Main.hs:155:3 in main:Main
* Depends on: bench-hist/upstream/ghcide bench-hist/upstream/ghc.path
* Raised the exception:
stack: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)


Benchmark benchHist: ERROR
Completed 2 action(s).

I can't say I understand the reasoning for these.

[pepeiborra]

Thanks for reporting!
I'll take a look soon

[pepeiborra]

The benchmarks do not assume a branch called ghc.path.

But they do assume stack is in the PATH, and that's the assumption that is unfulfilled here. Can you check and confirm?