New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stack setup on appveyor fails #1418

Closed
ndmitchell opened this Issue Nov 24, 2015 · 16 comments

Comments

Projects
None yet
4 participants
@ndmitchell
Contributor

ndmitchell commented Nov 24, 2015

See https://ci.appveyor.com/project/ndmitchell/hlint/build/1.0.44, which just gets stack.exe and runs stack setup --verbose. It fails at the end with:

Running C:\Program Files\Git\cmd\git.exe fetch --tags --depth=1 exited with ExitFailure 128
fatal: Not a git repository (or any of the parent directories): .git
Command exited with code 1

Is it possible on a never-used-before machine stack setup fails?

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 25, 2015

I see this in the log:

Restoring build cache
Restoring "c:\sr"...OK

I wonder if something might have gone funny in that build cache. Stack seems to think it already has a package index and just needs to update it, but git doesn't think it's a valid repo.

Could add a check to Stack to ensure the package index is a valid repo before trying to update it (if it's not, just blow it away and start over like it wasn't there), I suppose.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Nov 25, 2015

Good thinking, I turned off the cache, and it seems a lot happier. Seems like Stack wiping invalid repos would be very useful.

@borsboom

This comment has been minimized.

Contributor

borsboom commented Nov 25, 2015

A pull request would be most welcome!

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Dec 18, 2015

For info, I've now seen this on a different project on Appveyor, so not a one-off, but still not common.

@quyse

This comment has been minimized.

Contributor

quyse commented Dec 18, 2015

Note Appveyor docs says that cache size after compression must be no bigger than 100 Mb. What I see is that on my machine the hackage index alone is > 300 Mb.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Dec 18, 2015

@quyse - my guess is that is a wild guideline, and that they don't enforce it, partly because they don't give any hint how big a projects cache is or let you see what it contains.

Looking at the cache sizes for the compiled stuff on my machine I see things in the range 40Mb-160Mb, where the 160Mb compresses down to 20Mb. I'm not sure if I cache anything else or not - it's really pretty hard to figure out in Appveyor.

@quyse

This comment has been minimized.

Contributor

quyse commented Dec 18, 2015

@ndmitchell Well, you had setting cache: "c:\\sr", i.e. stack root. The biggest thing I've seen there is Hackage index (indices\Hackage\00-index.tar), and I guess it appears when you use non-stackage dependencies. On my machine they took 434 Mb.

I just came up with a script using additional git repository on Gitlab (because they don't have 1Gb per repo/100 Mb per file limits as Github has) in order to cache stack's stuff manually. Script gist: https://gist.github.com/quyse/7178ec5737235bdf60bf, build on appveyor using it: https://ci.appveyor.com/project/quyse/flaw/build/1.0.15. I think thanks to this script I won additional ~20 minutes so build doesn't timeout. It seems like a bit of abuse of Gitlab though (which delivers ~500 Mb for each build), so not really elegant solution. Appveyor really needs bigger cache limits.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Dec 19, 2015

My builds take near the 8 min mark, see https://ci.appveyor.com/project/ndmitchell/hlint. I guess I only have Stackage dependencies. I also run stack setup each time and don't cache that bit - caching that adds 20 mins or so.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Dec 30, 2015

I'm now seeing this almost every other build for one of my projects, which I does not seem special in any way. There may be some restoring of old states going on that recorrupts things.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Dec 30, 2015

PR now in #1580. In the absence of other evidence, I'm going to assume someone at Stack HQ sabotaged my Appveyor instance to make it recur enough to be worth fixing 😛

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented Jan 3, 2016

I think it's reasonable to assume #1580 fixed it, and I'll reopen if that turns out to not be the case.

@ndmitchell ndmitchell closed this Jan 3, 2016

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented May 7, 2016

In the last week I've suddenly got this on 5 different Appveyor repos. They are all using:

Version 1.1.0, Git revision 0e9430aad55841b5ff2c6c2851f0548c16bce7cf (3540 commits) i386 hpack-0.13.0

Seems like a regression in the latest release?

@ndmitchell ndmitchell reopened this May 7, 2016

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented May 7, 2016

Output is:

Updating package index Hackage (mirrored at https://github.com/commercialhaskell/all-cabal-hashes.git) ...
Fetching package index ...
fatal: Not a git repository (or any of the parent directories): .git
Process exited with ExitFailure 128: C:\Program Files\Git\cmd\git.exe fetch --tags --depth=1
Command exited with code 1

The related code is https://github.com/commercialhaskell/stack/blob/master/src/Stack/PackageIndex.hs#L260. I think the immediate cause is that the caller has switched from using readProcessNull which throws ReadProcessException to callProcessInheritStderrStdout which throws ExitFailure.

I note that at #1580 (comment) I said:

the process functions ... are all a bit weirdly inconsistent

This bug seems to be a symptom of that, with seemingly harmless refactorings changing subtle things like which exceptions are raised.

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented May 11, 2016

I've now hit this about 6 times in the last few days. Getting increasingly annoying. Can someone say if they agree/disagree with my diagnosis. I suspect this was broken by 967c3f0 as part of #1223.

@mgsloan

This comment has been minimized.

Collaborator

mgsloan commented May 12, 2016

@ndmitchell Sounds about right. Does e52b2c5 fix it?

Feel free to re-open if not

@mgsloan mgsloan closed this May 12, 2016

@ndmitchell

This comment has been minimized.

Contributor

ndmitchell commented May 23, 2016

Thanks @mgsloan! It seems to have stopped recurring in the last couple of days, most appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment