New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stack exec is unreasonably slow #1883

Closed
cocreature opened this Issue Mar 8, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@cocreature
Contributor

cocreature commented Mar 8, 2016

I primarily use stack exec to execute the executables I have just built. In the vast majory of cases (probably also for other people) all stack exec needs to do is set the PATH correctly since my executables don’t depend on any other settings. However the overhead stack exec adds has slowly grown to the point where it becomes annoying:

stack exec shake-reve -- --help  0.91s user 0.19s system 99% cpu 1.101 total

vs

./.stack-work/dist/x86_64-linux/Cabal-1.22.5.0/build/shake-reve/shake-reve   0.01s user 0.00s system 72% cpu 0.009 total

I realize that to completely set up the environment stack exec has to do more than just set the path correctly but if that causes such a drastic slowdown, could we get a --fast (probably needs a better name) option for the common case where it just needs to set the path?

@luigy

This comment has been minimized.

Contributor

luigy commented Mar 8, 2016

Yes, it would be great if the performance could be improved.
In the meantime, I've been doing stack exec $SHELL to take the hit only once since I only need my PATH to be set

mgsloan added a commit that referenced this issue Mar 9, 2016

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Mar 9, 2016

I've annotated the log with some notes:

2016-03-08 20:42:06.393949: [debug] Checking for project config at: /home/mgsloan/fpco/stack/stack.yaml @(stack_H2uuVvppPbeAJ1uiFxrFbt:Stack.Config src/Stack/Config.hs:761:9)
2016-03-08 20:42:06.394302: [debug] Loading project config file stack.yaml @(stack_H2uuVvppPbeAJ1uiFxrFbt:Stack.Config src/Stack/Config.hs:779:13)

Determining if stack is linked with gmp4
2016-03-08 20:42:06.396023: [debug] Run process: ldd /home/mgsloan/.local/bin/stack @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Decoding the resolver's set of packages (~0.02s)
2016-03-08 20:42:06.407513: [debug] Trying to decode /home/mgsloan/.stack/build-plan-cache/x86_64-linux/lts-5.3.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-03-08 20:42:06.424316: [debug] Success decoding /home/mgsloan/.stack/build-plan-cache/x86_64-linux/lts-5.3.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)

Decoding the hackage index (~0.3s)
2016-03-08 20:42:06.425239: [debug] Trying to decode /home/mgsloan/.stack/indices/Hackage/00-index.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:55:5)
2016-03-08 20:42:06.722030: [debug] Success decoding /home/mgsloan/.stack/indices/Hackage/00-index.cache @(stack_H2uuVvppPbeAJ1uiFxrFbt:Data.Binary.VersionTagged src/Data/Binary/VersionTagged.hs:64:13)

Checking if the system compiler can be used (~0.06s)
2016-03-08 20:42:06.742570: [debug] Run process: ghc --info @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Getting the exact ghc version (~0.04s)
2016-03-08 20:42:06.806545: [debug] Run process: ghc --numeric-version @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Getting the cabal version (~0.03s)
2016-03-08 20:42:06.845667: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal version @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)

Getting the location of the global package DB (~0.03s)
2016-03-08 20:42:06.869424: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_H2uuVvppPbeAJ1uiFxrFbt:System.Process.Read src/System/Process/Read.hs:269:3)
2016-03-08 20:42:06.891551: [debug] Run process: echo hi @(stack_H2uuVvppPbeAJ1uiFxrFbt:Stack.Exec src/Stack/Exec.hs:61:5)

So, about 3/5 of the time is being spent decoding the hackage index, which isn't being used.

It's tricky to do anything about the remaining 2/5, because in order to populate the PATH env var, we need to know the ghc and cabal version being used. For system installations managed by stack, we could trust that the version is accurate - we're still asking ghc for its version in that case.

You'll save something like 0.06s by doing --no-system-ghc, skipping the system ghc check.

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Mar 9, 2016

I've pushed a commit which only loads up the hackage index when necessary. Now it takes 50% less time or so! There's still some work to do here, but not relevant to stack exec: #1892

@cocreature

This comment has been minimized.

Contributor

cocreature commented Mar 9, 2016

@mgsloan This is a huge improvement! Thanks a lot. It’s now below the point where it starts to be painful for me, so feel free to close this if there is no way to improve it further (with reasonable effort).

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Mar 10, 2016

Welcome, glad it's below the point of being painful!

Yeah, I think optimizing it further is more effort than it's worth at this point. I've written down a misc TODO note to maybe revisit the ghc version checking stuff for stack installed ghcs.

@mgsloan mgsloan closed this Mar 10, 2016

@mgsloan mgsloan added this to the P2: Should milestone Mar 10, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment