On some MingW versions Tup do not detect compiler output #68

Open
foresterLV opened this Issue Jun 1, 2012 · 5 comments

Comments

Projects
None yet
2 participants
Contributor

foresterLV commented Jun 1, 2012

On MingW tools I am using Tup does not detect compiler output (it works with latest MigW however).
I have put test project + mingw copy to https://www.dropbox.com/s/giie7l71daqt16d/tup_mingw_gcc_nooutput.zip.
When running "git init"->"git upd" I am getting:

* 1) .\mingw\bin\g++.exe test.cpp -o test.exe
 *** tup errors ***
tup error: Expected to write to file 'test.exe' from cmd 848 but didn't
 *** Command ID=848 ran successfully, but tup failed to save the dependencies.
 [.] 100%
 *** tup: 1 job failed.

But apparently test.exe is created by the tool.
tup v0.6-161-g62fc306 on Windows 7 x64.

Owner

gittup commented Jun 1, 2012

On Fri, Jun 1, 2012 at 12:01 PM, foresterLV
reply@reply.github.com
wrote:

On MingW tools I am using Tup does not detect compiler output (it works with latest MigW however).
I have put test project + mingw copy to https://www.dropbox.com/s/giie7l71daqt16d/tup_mingw_gcc_nooutput.zip.
When running "git init"->"git upd" I am getting:

   * 1) .\mingw\bin\g++.exe test.cpp -o test.exe
    *** tup errors ***
   tup error: Expected to write to file 'test.exe' from cmd 848 but didn't
    *** Command ID=848 ran successfully, but tup failed to save the dependencies.
    [.] 100%
    *** tup: 1 job failed.

But apparently test.exe is created by the tool.
tup v0.6-161-g62fc306 on Windows 7 x64.

Strange, when I try to run your example I get a different error:

  • 100% 1) .\mingw\bin\g++.exe test.cpp -o test.exe

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
*** tup errors ***
*** Command ID=848 failed with return value 3
*** tup: 1 job failed.

Maybe this is a separate issue? Or a problem with the tools in the zip file?

Do you have procmon installed? If so, can you try running the g++
command with procmon watching it and see what API is used to create
test.exe?

Thanks,
-Mike

Contributor

foresterLV commented Jun 1, 2012

I'l take a look. BTW regarding this issue and Cygwin crash, why not to move away from dll injection complication to something simple as FindFirstChangeNotification API to monitor directory changes?

Owner

gittup commented Jun 1, 2012

On Fri, Jun 1, 2012 at 2:42 PM, foresterLV
reply@reply.github.com
wrote:

I'l take a look. BTW regarding this issue and Cygwin crash, why not to move away from dll injection complication to something simple as FindFirstChangeNotification API to monitor directory changes?

I would be fine with moving away from dll injection if a better
approach can be found. However from looking at
FindFirstChangeNotification, it looks like this would only detect file
writes? Tup uses the DLL injection to determine all file accesses by
the program - files opened for reading are inputs and files opened for
writing are outputs. I also don't see an easy way using that API to
know which process wrote to a file if multiple processes were
executed in parallel. For example, if two .c files are compiled at the
same time, tup needs to get a list of file accesses as follows:

gcc -c bar.c -o bar.o:

  • read bar.c
  • read bar.h
  • read shared.h
  • write bar.o

gcc -c foo.c -o foo.o:

  • read foo.c
  • read foo.h
  • read shared.h
  • write foo.o

This is how tup knows to re-compile both files if shared.h is updated,
but only foo.c if foo.h is updated, etc. The DLL injection is supposed
to allow this to work not just for gcc, but for any program. If we get
a notification that bar.o was written, would tup be able to know that
it was the "gcc -c bar.c -o bar.o" command that wrote it? Do we get
notifications that it also opened bar.c, bar.h, and shared.h?

Obviously the issues you've reported with it should be resolved, and
if there is a better way to go about getting a list of files like this
with parallel sub-processes, that would certainly be desirable.

Thanks,
-Mike

Contributor

foresterLV commented Jun 1, 2012

Yes you are right, I was not thinking about read-only access and dependency info retrieved this way. FindFirstChangeNotification indeed only catches write access and there is no way to get info on source process.
It is good that you replied so quickly because I was in process of hacking tup into using FindFirstChangeNotification :).
Saved some time. I will try to look into hook then.

Owner

gittup commented Jun 1, 2012

On Fri, Jun 1, 2012 at 4:19 PM, foresterLV
reply@reply.github.com
wrote:

Yes you are right, I was not thinking about read-only access and dependency info retrieved this way. FindFirstChangeNotification indeed only catches write access and there is no way to get info on source process.
It is good that you replied so quickly because I was in process of hacking tup into using FindFirstChangeNotification :).
Saved some time. I will try to look into hook then.

That API may still be useful in a separate area. In Linux, tup uses
inotify as a monitoring process that runs in the background to see
when the user changes foo.c or the Tupfiles or whatever. Then when it
starts up it doesn't have to scan through the whole directory tree to
see what files have changed. Currently in Windows and OSX tup doesn't
have a monitor implementation, so it always has to do a directory
scan. This only really helps on very large projects, so it isn't a
high priority thing to do (a hook that works better would definitely
take precedence :). To get an idea of how much time a monitor saves,
you can look at the first two lines printed by tup:

[ tup ] [0.000s] Scanning filesystem...
[ tup ] [0.008s] Reading in new environment variables...

In this case, 8ms are spent scanning the fs. With a monitor, it
wouldn't have to do that. This is for tup itself, which is why the
time is so small. For a larger project it does help, though.

Also just FYI if you are looking to contribute code to tup, I do ask
contributors to sign a licensing agreement so that tup can be dual
licensed: http://gittup.org/tup/icla.txt

Thanks,
-Mike

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment