New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

large stack build uses a lot of space in temp directory #3018

Closed
joeyh opened this Issue Feb 21, 2017 · 2 comments

Comments

Projects
None yet
3 participants
@joeyh

joeyh commented Feb 21, 2017

General summary/comments (optional)

When stack build needs to build a lot of dependencies, it uses a lot of disk space in the system temp directory. In particular, build directories for all packages are left there after the libraries have been installed into .stack-work, and are only cleaned up after the build finishes.

Steps to reproduce

stack build in git-annex source tree on a system with eg a 1.3 gb tmpfs for /tmp. (The default on a modern debian system with 4 gb of ram.)

Expected

successful build

Actual

Build will eventually use over a GB of space in /tmp/stacknnnn and fail. The temp directory is then freed, and re-running the build picks up where it left off, clearly not needing any of the stuff that was being kept in the temp directory.

Workaround

Set TMP to point to a directory with more space than /tmp

Suggested fix

Delete package build directories of dependencies after the copy/register step.

Stack version

1.1.2

I'm fairly sure current stack also behaves this way too. Looking at https://github.com/commercialhaskell/stack/blob/4ee027d2d93f42e790ff170c139094cc8dbb2373/src/Stack/Build/Execute.hs , withExecuteEnv is used to run the build plan and uses a single withSystemTempDir

Method of installation

apt-get

@nponeccop

This comment has been minimized.

nponeccop commented Mar 5, 2017

Stack 1.3.2 on ArchLinux behaves the same way.

du -sBM /tmp/stack30907/ and ls -1 /tmp/stack30907/ | wc -l constantly grow. In my case tmpfs eats up all swap space so last few pakage builds are very very slow (seem to run GC too much but I didn't really check that) and finally I get an OOM error.

mgsloan added a commit that referenced this issue Mar 10, 2017

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented Mar 10, 2017

I've pushed a fix for this using the suggested fix of

Delete package build directories of dependencies after the copy/register step.

Good idea @joeyh !

@mgsloan mgsloan closed this Mar 10, 2017

mgsloan added a commit that referenced this issue Mar 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment