New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setupEnv: Run some processes concurrently #2346

Merged
merged 2 commits into from Aug 6, 2016

Conversation

Projects
None yet
3 participants
@sjakobi
Contributor

sjakobi commented Jul 6, 2016

This should make most commands slightly faster.
On my machine, stack path takes ~20% less time than before, a no-op
stack build in a single-package project ~15%.

@borsboom borsboom added the in progress label Jul 6, 2016

@sjakobi

This comment has been minimized.

Contributor

sjakobi commented Jul 6, 2016

So, what I actually don't know, is whether this safe, especially querying ghc-pkg asynchronously.

Can someone with more experience with asynchronous programming take a look?

BTW here's some debug output from a stack version on top of my better-process-logging branch for #2327:

~ $ stack-bpl-async -v path --dist-dir --global-pkg-db
Version 1.1.3, Git revision 7934d6ce5eb9b841a6f248b427e2e8812fd69845 (dirty) (3753 commits) x86_64 hpack-0.14.0
2016-07-06 19:40:57.493231: [debug] Checking for project config at: /home/simon/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493629: [debug] Checking for project config at: /home/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493763: [debug] Checking for project config at: /stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:823:9)
2016-07-06 19:40:57.493909: [debug] No project config file found, using defaults. @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:845:13)
2016-07-06 19:40:57.496410: [debug] Checking whether stack was built with libgmp4 @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:327:5)
2016-07-06 19:40:57.496909: [debug] Run process: /usr/bin/ldd /home/simon/.local/bin/stack-bpl-async @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.512142: [debug] Process finished in 15 ms: /usr/bin/ldd /home/simon/.local/bin/stack-bpl-async @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.512561: [debug] Stack was not built with libgmp4 @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:331:14)
2016-07-06 19:40:57.512688: [info] Run from outside a project, using implicit global project config @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:462:13)
2016-07-06 19:40:57.514316: [info] Using resolver: nightly-2016-06-21 from implicit global project's config file: /home/simon/.stack/global-project/stack.yaml @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Config src/Stack/Config.hs:476:32)
2016-07-06 19:40:57.514602: [debug] Trying to decode /home/simon/.stack/build-plan-cache/x86_64-linux/nightly-2016-06-21.cache @(Kkm9qBx54bb6fPiRXOOC4T:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:51:5)
2016-07-06 19:40:57.527888: [debug] Success decoding /home/simon/.stack/build-plan-cache/x86_64-linux/nightly-2016-06-21.cache @(Kkm9qBx54bb6fPiRXOOC4T:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:60:13)
2016-07-06 19:40:57.534286: [debug] Getting system compiler version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup src/Stack/Setup.hs:345:17)
2016-07-06 19:40:57.534606: [debug] Run process: /usr/bin/ghc --info @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.560165: [debug] Process finished in 25 ms: /usr/bin/ghc --info @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.563992: [debug] Asking GHC for its version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup.Installed src/Stack/Setup/Installed.hs:97:13)
2016-07-06 19:40:57.564233: [debug] Getting Cabal package version @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:165:5)
2016-07-06 19:40:57.564356: [debug] Getting global package database location @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-07-06 19:40:57.564437: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc --numeric-version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.565233: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.565852: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.602662: [debug] Process finished in 37 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db field --simple-output Cabal version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.606331: [debug] Process finished in 34 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.607750: [debug] Process finished in 43 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc --numeric-version @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.607943: [debug] Resolving package entries @(Kkm9qBx54bb6fPiRXOOC4T:Stack.Setup src/Stack/Setup.hs:226:5)
2016-07-06 19:40:57.608657: [debug] Getting global package database location @(Kkm9qBx54bb6fPiRXOOC4T:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-07-06 19:40:57.608962: [debug] Run process: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
2016-07-06 19:40:57.631108: [debug] Process finished in 21 ms: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/bin/ghc-pkg --no-user-package-db list --global @(Kkm9qBx54bb6fPiRXOOC4T:System.Process.Read src/System/Process/Read.hs:292:3)
global-pkg-db: /home/simon/.stack/programs/x86_64-linux/ghc-8.0.1/lib/ghc-8.0.1/package.conf.d
dist-dir: .stack-work/dist/x86_64-linux/Cabal-1.24.0.0
setupEnv: Run some processes concurrently
This should make most commands slightly faster.
On my machine, `stack path` takes ~20% less time than before, a no-op
`stack build` in a single-package project ~15%.

@sjakobi sjakobi force-pushed the sjakobi:async-setupenv branch from 7423161 to 875954c Jul 21, 2016

@sjakobi

This comment has been minimized.

Contributor

sjakobi commented Jul 21, 2016

So, what I actually don't know, is whether this safe, especially querying ghc-pkg asynchronously.

Now, I figure there's no reason why ghc-pkg wouldn't be able to perform two reads concurrently.

@sjakobi sjakobi removed the in progress label Jul 21, 2016

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Aug 6, 2016

Makes sense!

@mgsloan mgsloan merged commit 099ca78 into commercialhaskell:master Aug 6, 2016

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment