Support haddock response files #1681

Closed
srayuws opened this Issue Feb 12, 2014 · 17 comments

Projects

None yet

6 participants

@srayuws
srayuws commented Feb 12, 2014

Enviornment:
windows 7 64bit ,
haskell-platform 2013.2,
cabal.exe version 1.18.0.2 using version 1.18.1.2 of the Cabal library
haddock.exe version 2.13.2

HOW I MEET THIS:
I have a fresh haskell-platform 2013.2, and run the following:

# edit the config file to enable documentation
cabal update
cabal install cabal-install
cabal update
cabal install yesod  # this will install a lot hackages and run for minutes

At the end of the output, there are a few lines:

Updating documentation index
C:\Users\srayuws\AppData\Roaming\cabal\doc\index.html
cabal: C:\Program Files (x86)\Haskell Platform\2013.2.0.0\bin\haddock.exe:
does not exist

At this time, if I install or reinstall any hackage, it will say haddock.exe does not exist.
By the way, it seems only the last installed hackage could not find haddock.exe, the document for its dependencies are gererated well.

But if i run "C:\Program Files (x86)\Haskell Platform\2013.2.0.0\bin\haddock.exe" --version, it works fine.

I have also noticed that after delete the doc directory (with about 100+ hackages in it) in %APPDATA%\cabal\, then reinstall some hackages which could not find haddock.exe before, everything goes fine again...

@srayuws
srayuws commented Mar 7, 2014

Hello, I have made some more investigation on this problem.

By checking the log with -v3, I guess it may caused by the very very long argument list.

After install about 170+ hackages, I try to reinstall hxt, which have about 20 dependencies. The command to generate document for hxt works fine, which have 160+ arguments and about 13k characters long.

But it fails to update the overall document index. This time the command have about 200+ arguments and 50k characters long.

I have also tried another hackage yesod-auth ( about 140 dependencies) . At this time, it fails to generate the document for itself, and skip updating the overall index. The command it runs has about 240+ items and 37k characters long.

Is there some solution or workaround to solve this problem?

@srayuws
srayuws commented Mar 7, 2014

Hello,

I find a workaround here. Haddock provides an API haddock which can directly work on the argument list and does not rely on the System.Process nor the MAX_PATH limitation.

Using haddock with the long argument list copied from cabal log, I successfully updated my local document index.

However there is some disadvantages for using haddock in cabal-install.
First, the APIs in Haddock is marked as unstable. And using this would also add haddock as a dependency to cabal-install.

I have no idea if there is some better solution to get rid of this problem.

@23Skidoo
Member
23Skidoo commented Mar 7, 2014

The standard solution for these kinds of problems is response files. I don't know whether haddock supports them; Cabal itself currently doesn't.

@Fuuzetsu
Member
Fuuzetsu commented Mar 7, 2014

We could probably implement a flag in Haddock to support this (i.e. we treat @somefile differently when the flag is present) if cabal guys want to use it.

@23Skidoo
Member
23Skidoo commented Mar 7, 2014

Sure, if Haddock supported response files, we'd add support for invoking it with @file to Cabal.

@Fuuzetsu
Member
Fuuzetsu commented Mar 7, 2014

Cool. I'll implement it soon, perhaps it'll be ready by tomorrow. Watch this Haddock ticket.

@randen
Collaborator
randen commented Mar 28, 2015

In building the Haskell Platform 2015.2.0.0 (for GHC 7.10.1) on the Windows platform, we are now unable to build the haddocks for OpenGLRaw, due to exactly the problem noted in haskell/haddock#285 so adding another vote and some urgency for this feature in Cabal.

One question I have is how cabal-install will expose this at the command-line level since cabal-install's options are a mix of its own and haddock's, so would the user specify only some of the haddock options in the response file? Or would Cabal re-arrange the user-provided response file contents to include some parts that it adds to the actual haddock invocation? Or would cabal-install allow its own response file for all of its options (including the ones it passes over to any other executable), then, perhaps optionally (e.g., looking at the version number of, in this case, haddock) supply the invoked executable with a response file of Cabal's own creation? Could get complicated?

The simple approach I took in my experiments to fix this was to not do a full-on "response file" approach (i.e., all command line arguments in one file) but only to allow specifying the file list to haddock via a file, which eventually would still hit the limitation if using many other non-file options (e.g., --read-interface lists can be very long as well), but it did avoid some of the things to consider I mention above. Just wanted to mention in case it helps you.

In any event, the solution in Cabal and cabal-install will need to match what haddock does, and then the HP build code will need to invoke cabal-install to match.

@ttuegel ttuegel changed the title from haddock.exe does not exist to Support haddock response files Apr 23, 2015
@ttuegel ttuegel added this to the _|_ milestone Apr 23, 2015
@randen
Collaborator
randen commented May 11, 2015

What does a milestone of "bottom" indicate? The problem we have building the docs for large libraries on the Windows platform needs some kind of solution. If using response files is not a good solution for some reason (understandably, it is not needed for other environments), but we do need something or Windows builds will have missing docs for large libraries (and may even flat out fail to complete).

The Haskell Platform team humbly requests that a fix for this be included in the haddock and cabal which are shipped whatever the next official ghc release, likely 7.10.2.

@Fuuzetsu
Member

There is a problem with the Haddock's implementation of response files which probably needs to be fixed first.

@23Skidoo
Member

There is a problem with the Haddock's implementation of response files which probably needs to be fixed first.

That is, haskell/haddock#379, without which response files are pretty useless.

@Fuuzetsu
Member

Right. I actually realised that we could have just one generic implementation in Cabal and Haddock could import that and feed args through it before doing Haddock-y stuff. It'd ensure consistent implementation which I believe is what we're after.

@23Skidoo
Member

Can Haddock depend on Cabal?

@Fuuzetsu
Member

It already does.

https://github.com/haskell/haddock/blob/master/haddock.cabal#L59 and a similar line in haddock-api if we're not in GHC tree. This is OK because Cabal doesn't depend on Haddock and comes with GHC.

@23Skidoo
Member

process or base are probably more appropriate places for this - I bet that getArgsWithResponseFiles could be generally useful.

@NightRa
NightRa commented Sep 28, 2015

Any progress on this?

@23Skidoo
Member

Fixed by #2746.

@NightRa Thanks for the heads-up.

@23Skidoo 23Skidoo closed this Sep 28, 2015
@23Skidoo 23Skidoo modified the milestone: Cabal-1.24, _|_ Sep 28, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment