hp2ps depends on "ghcautoconf.h" and "ghcplatform.h", but does not yet need it.

[angerman]

Reading shake-build/cfg/system.config...
Reading package dependencies...
| Successfully generated 'utils/hp2ps/stage0/package-data.mk'.
Reading utils/hp2ps/stage0/package-data.mk...
Reading dependencies from utils/hp2ps/stage0/build/.dependencies...
/----------
| Running GccM Stage0 with arguments:
|   ... skipping 13 arguments ...
|   utils/hp2ps/Utilities.c
\----------

In file included from utils/hp2ps/Utilities.c:1:
In file included from utils/hp2ps/Main.h:4:
includes/ghcconfig.h:4:10: fatal error: 'ghcautoconf.h' file not found
#include "ghcautoconf.h"
         ^
1 error generated.
shakeArgsWith   0.000s    0%                           
Function shake  0.019s    5%  =                        
Database read   0.000s    0%                           
With database   0.000s    0%                           
Running rules   0.314s   94%  ======================== 
Total           0.334s  100%                           
Error when running Shake build system:
* inplace/bin/hp2ps
* utils/hp2ps/stage0/build/Utilities.o
* OracleQ (DependenciesKey ("utils/hp2ps/stage0/build/.dependencies","utils/hp2ps/stage0/build/Utilities.o"))
* utils/hp2ps/stage0/build/.dependencies
* utils/hp2ps/stage0/build/Utilities.c.deps
user error (Development.Shake.cmd, system command failed
Command: /usr/bin/clang -E -MM -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iutils/hp2ps/stage0/build/autogen -MF utils/hp2ps/stage0/build/Utilities.c.deps -MT utils/hp2ps/stage0/build/Utilities.o -x c utils/hp2ps/Utilities.c
Exit code: 1
Stderr:
In file included from utils/hp2ps/Utilities.c:1:
In file included from utils/hp2ps/Main.h:4:
includes/ghcconfig.h:4:10: fatal error: 'ghcautoconf.h' file not found
#include "ghcautoconf.h"
         ^
1 error generated.
)

and after ./shake-build/build.sh includes/ghcautoconf.h

Reading shake-build/cfg/system.config...
Reading package dependencies...
Reading utils/hp2ps/stage0/package-data.mk...
Reading dependencies from utils/hp2ps/stage0/build/.dependencies...
/----------
| Running GccM Stage0 with arguments:
|   ... skipping 13 arguments ...
|   utils/hp2ps/Utilities.c
\----------

In file included from utils/hp2ps/Utilities.c:1:
In file included from utils/hp2ps/Main.h:4:
includes/ghcconfig.h:5:10: fatal error: 'ghcplatform.h' file not found
#include "ghcplatform.h"
         ^
1 error generated.
shakeArgsWith   0.000s    0%                           
Function shake  0.018s   12%  ===                      
Database read   0.006s    3%  =                        
With database   0.000s    0%                           
Running rules   0.119s   83%  =========================
Total           0.143s  100%                           
Error when running Shake build system:
* inplace/bin/hp2ps
* utils/hp2ps/stage0/build/Utilities.o
* OracleQ (DependenciesKey ("utils/hp2ps/stage0/build/.dependencies","utils/hp2ps/stage0/build/Utilities.o"))
* utils/hp2ps/stage0/build/.dependencies
* utils/hp2ps/stage0/build/Utilities.c.deps
user error (Development.Shake.cmd, system command failed
Command: /usr/bin/clang -E -MM -Iincludes -Iincludes/dist -Iincludes/dist-derivedconstants/header -Iincludes/dist-ghcconstants/header -Iutils/hp2ps/stage0/build/autogen -MF utils/hp2ps/stage0/build/Utilities.c.deps -MT utils/hp2ps/stage0/build/Utilities.o -x c utils/hp2ps/Utilities.c
Exit code: 1
Stderr:
In file included from utils/hp2ps/Utilities.c:1:
In file included from utils/hp2ps/Main.h:4:
includes/ghcconfig.h:5:10: fatal error: 'ghcplatform.h' file not found
#include "ghcplatform.h"
         ^
1 error generated.
)

[snowleopard]

I have a bad feeling about this.

What happens is:

• We run clang -MM to compute dependencies of file hp2ps/Utilities.c
• It includes hp2ps/Main.h
• The latter includes generated file includes/ghcconfig.h
• But the file has not yet been generated, so clang -MM doesn't know what to do: what if includes/ghcconfig.h includes something else? We need to have it in order to determine accurate dependencies. So, clang -MM gives up! I wonder if gcc -MM gives up as well (I haven't seen this failure yet).

Not sure what's the best way to proceed with this. I can add a manual dependency: need includes files before running other rules, but this is fragile.

[snowleopard]

@ndmitchell Have you got any idea on how to handle this nicely? I don't have anything except for needing generated files explicitly.

[snowleopard]

I've added explicit dependencies. We may come back to this issue if a better approach is discovered.

[Ericson2314]

Hi, I've been lurkingfollowing along until I can get my computer replaced and actually contribute, but I think I have a solution for this.

Unfortunately, the non-lexical way CPP works makes things more complicated in that macros defined in headers included earlier influence the dependencies of headers included later.

I propse adding at the back of the INCLUDE path a directory with empty files named for each generated file (hopefully we can figure out that list up front). Then, when clang outputs a dependency on one of those stubs, we know we need to rebuilt it. Because of the aforemtioned non-lexicallity, after building all generated header files, we repeated the process. After a finite number of rounds, all relevant header files will be discovered and built (we'll know because clang will no longer fall back on the stubs).

[snowleopard]

@Ericson2314 Hi there! Adding a directory with fake files is a nice trick. I don't see how it helps though: we still need to explicitly list these files somewhere: we list them as dependencies in src/Rules/Dependencies.hs and everything seems to work well. You propose to list them in a special directory, and furthermore also run clang -MM a number of rounds, which would be hard to implement.

[ndmitchell]

One quite easy solution is to orderOnly depend on those files before running -MM. That ensures the files exists, but doesn't depend on them, and then -MM can spot them properly. The advantage of orderOnly is you can be much more free with it as it will reduce parallelism by a very tiny amount, but won't cause unnecessary rebuilds.

[snowleopard]

@ndmitchell Thanks! I didn't think of order-only dependencies. Looking at the Shake documentation this is exactly the intended use case for orderOnly.

(In an ideal world we would have a dependency discovery mechanism that would call orderOnly [file] whenever it encounters an include file statement while traversing given source files, so we wouldn't have to list such dependencies manually.)

[Ericson2314]

In an ideal world...

Well, in general CPP variables can be used in the #include path, so that won't work.

However, intercepting file access, as proposed in that linked issue, is the best of all worlds. Dirt simple, no up-front lists, and builds only the headers that are needed.