Can't compose Shake rules because FileQ/FileA types are not exported #388

Open
ezyang opened this Issue Dec 31, 2015 · 11 comments

Projects

None yet

2 participants

@ezyang
Contributor
ezyang commented Dec 31, 2015

I wanted to make a new rule on Haskell modules, the idea being that you could need a module and that would automatically imply both the hi and o files exist. The reason I need to reimplement it is to work around #382; my workaround idea was to reimplement the "Files" rule, but to simply dynamically change how much of the response type we looked at when determining up-to-dateness. (So, with -dynamic-too, we look at all four of hi/o/dyn_hi/dyn_o, but without it we only consider the plain hi/o. (I was hoping that if no rebuild occurs without -dynamic-too, then recompilation is avoided when I invoke with -dynamic-too the next time.)

But I can't do this easily! Why not? Because the most straightforward way to implement a new composite rule on top of file is to reuse the existing FileA and FileQ types... but they're not exported.

I understand that there's a good reason these are not exported: encapsulation. So I'm OK with any other solution which solves my proximal problem. Perhaps #185/#191/#192 are relevant.

@ezyang
Contributor
ezyang commented Dec 31, 2015

Subsumed by #392.

@ezyang ezyang closed this Dec 31, 2015
@ezyang
Contributor
ezyang commented Dec 31, 2015

I'm going to reopen this, because the thing that ended up easiest was exporting a few more modules from Shake and then reimplementing this.

@ezyang ezyang reopened this Dec 31, 2015
@ndmitchell
Owner

Hmm, I'd like to hope there is a way to do the hi/dyn_hi problem without opening up file rules. Let's tackle that ticket before deciding on this one.

@ndmitchell
Owner

Random thought I'm jotting down to evaluate properly later: If the Rule only had a single type parameter, and returned MaybeStored Value always, then you would have to export FileQ, but not FileA, which seems much more abstract. You could also have a monoid that joined MaybeStored Value to "compose" Shake rules. It also fits with other transitions happening, although does lose some type safety and makes it hard to switch on the return type, which might be fatal.

@ezyang
Contributor
ezyang commented Dec 31, 2015

Looking at my code, it seems like this would be sufficient. Loss of type safety is not so nice though. (You kind of want an existential type!)

@ezyang
Contributor
ezyang commented Feb 13, 2016

Still on hot path.

@ndmitchell ndmitchell referenced this issue in snowleopard/hadrian Mar 6, 2016
Closed

Object file not rebuild when hi file is dirty #216

@ndmitchell
Owner

A related issue from @snowleopard: I think it may be interesting to consider the following relaxed (or lazy) AND rule which could solve the C/Haskell object files problem we have without relying on the "cbits" folder trick.

[X, Y, Z] `relaxedANDrule` $ \outs -> ...

The difference between the usual AND rule (&%>) and relaxedANDrule is that the latter promises to build all of [X, Y, Z] but only if they will ever be needed by other rule(s). The usual AND rule promises to always build them. The relaxed rule promises to build only what's actually necessary (it's almost as if it was lazy, but I'm not sure it can be called that). With this rule we can write

["//*.o", "//*.hi"] `relaxedANDrule` $ \[o, hi] -> if isHaskellObject o then runGhc o hi else runGcc o

And it will just work, because no rule will ever need interface files corresponding to C sources. Note, this also works fine in case of concurrent needs on *.o and *.hi files for a Haskell source. I think the difference in the implementation between usual and relaxed AND rules is only in some post-rule error checking? What do you think of this?

@ndmitchell
Owner

I think that relaxedANDrule is really a rule that produces a pair of files, the first one strict, the second one is a Maybe - it might be there or might not. Hopefully once you can have composable file rules you'll be able to have any combination of pairs, lists, eithers, maybe - and then you can write a custom relaxedANDrule that does exactly what you want (and there might be a default Maybe [File] rule supplied corresponding to relaxedAND).

@ezyang
Contributor
ezyang commented May 4, 2016

I guess this might solve the proximal problem? Hard to say without actually attempting to implement it this way. I don't see how it solves the problem described in #382, esp. if it only affects rule checking.

@ndmitchell
Owner

Agreed, I'm moving away from this idea - I was actually trying to tackle it as you commented it. Still a fair bit more thought required, but I feel I'm getting closer...

@ndmitchell
Owner

I think this will be done as part of #453.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment