Build failures should probably be more fatal

[KristofferC]

Right now, a build failure prints an error message but then nothing actually fails. The project and manifest files are still updated. The package can still be imported. It doesn't really make sense, if a package fails to build then it is unusable and we shouldn't pretend that everything is fine.

There are two ways to handle this:

• Just remove the package from packages if it fails to build. On next add it will get downloaded again and another build attempt will be tried.
• Somehow keep the package around but indicate that it hasn't been built yet. On next add there is no need to download the package but we know that it hasn't been built so let's try that again. The problem here is that maybe the package got in a bad state from failing to build the first time.

[pfitzseb]

The clean-slate approach seems much more robust and easier to implement, so I'd be in favour of that, fwiw.

[StefanKarpinski]

The "clean slate" approach is how Pkg2 used to work, but people complained, so we changed it. They wanted the build directory to stay around so that they could go in and fix the build manually after a failure occurred. That always struck me as a bad idea for anyone who’s not a fairly expert user, which was apparently correct (from some reports we've gotten from various places) but we never hear about it here because non-expert users don’t post on GitHub. So I don't think the "clean slate" approach is quite right. The approach of marking the package as not successfully built is what we'd have to go with, to give expert users a chance to go in and fix the build and non-expert users a clear, persistent indication that something is broken. Of course hopefully all of this is mooted by Binary Builder and Artifacts.

[KristofferC]

The approach of marking the package as not successfully built is what we'd have to go with, to give expert users a chance to go in and fix the build

What I am mostly focusing on here is Pkg.add where you cannot go in and fix the build.

[pfitzseb]

Yeah, imho it doesn't really make any sense to try and fix build problems for added packages, especially since we're at least trying to make sure those are immutable.

[StefanKarpinski]

What I am mostly focusing on here is Pkg.add where you cannot go in and fix the build.

Why can't you? The deps/ directory is writeable and is where the build step puts its results.

[KristofferC]

You can't change the build.jl file and the directory is in a really weird place. No one is going to go in there and fix something, they'll dev it in that case.

[StefanKarpinski]

Ok, so maybe suggest that they dev and try to fix the build if it fails? Or prompt the user?

[fingolfin]

So there are indeed two mostly disjoint target groups here:

1. "regular users" (i.e. somebody who "just wants it to work): for those I agree that a clean slate most likely is best
2. "power users": they might want to debug the issue; for this, it is IMHO vital to be able to retain (a copy of) the failed build.

People in group 1 "just want it to work"; they lack the resources(time, interest, ability, ...) to dig deeper; so keeping failed builds around is a net-negative for them (waste storage, too ;-), and a clean slate probably about the best solution for them.

People in group 2 might include the authors of a package who are trying to debug why it failed to build; but also anybody else who is interested in getting to the bottom of things, for whatever reason. This can even include "regular users" who received instructions by a package author to collect some info about a failing build, to help the package author triage the issue.

For a package manager I worked a long time ago (Fink, if anybody remembers that), we solved this tension of requirements by adding a --keep-build-dir option, which default to false. Perhaps such an option could be added to Pkg.build, similar to verbose?

BTW, a major nuisance for me in the past with the clean slate approach would have been that I would have to download a couple hundred MB of assets with each attempt. But now all these assets are artifacts, and those would survive here (wouldn't they? unless the user immediately did a gc or so, but users in group 1 are unlikely to do that). With that issue removed, I personally would be happy to go for the clean slate approach, as long as a keep_build_dir (or whatever) option was there to avoid it.

[KristofferC]

"power users": they might want to debug the issue; for this, it is IMHO vital to be able to retain (a copy of) the failed build.

You would just dev then which would (of course) keep the files around.

[fingolfin]

OK, fair point!

That said, I've not yet tried to walk a "regular user" through the steps to dev a package. I can't see a major obstacle with that, but then I am often surprised what all can go wrong with that (they are all supposed to be scientists working on computational algebra, yet somehow some manage to be almost completely computer illiterate... it baffles me). I'll make sure to try it the next time breaks somewhere.

But assuming it works well (I see no reason why not), I'd be fully in favor of "clean slate" approach.

[simonbyrne]

Similarly, Pkg.build() should throw an error if the build fails. This would be useful for CI scripts where build failures only manifest themselves as problems when attempting to load the package.