Distcc causing "access violation failures" on Windows XP

[GoogleCodeExporter]

Distcc version: 3.2rc1 built for Windows/Cygwin
Running on Windows XP
Running 'make -j ...' (make version: 3.82, mingw, using cmd.exe as its shell)

In my situation, 'make' stops with errors like this:
'../../os/inos/src/cinosmcfastload.cpp'
'../../os/inos/src/cinosmcguardian.cpp'
The process cannot access the file because it is being used by another process.
os/inos/src/subdir.mk:660: recipe for target 'os/inos/src/cinosmcjob.o' failed
make: *** [os/inos/src/cinosmcjob.o] Error 1
make: *** Waiting for unfinished jobs....

The problem only occur if 'make' invokes distcc (instead of gcc directly). I 
compiled that project on Windows XP, Vista and 7. The mentioned failure only 
occurred on Windows XP.


Analysis:
---------
After some investigations, I found out that the mentioned error message is 
coming from cmd.exe, trying to execute a batchfile (.bat) temporarily created 
by make.exe within the Windows temp folder. Here's the (slightly obfuscated) 
ProcMon entry:
"06:58:18.8787020","cmd.exe","5196","CreateFile","C:\Documents and 
Settings\...\Local Settings\Temp","SHARING VIOLATION","Desired Access: Read 
Data/List Directory, Synchronize, Disposition: Open, Options: Directory, 
Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, 
AllocationSize: n/a"

Funnily, the distcc.log contains tons of
(dcc_cleanup_tempfiles_inner) cleanup C:/DOCUME~1/ZG_DRA~1/LOCALS~1/Temp 
failed: Operation not permitted

According to ProcMon, distcc indeed tries to delete the "Windows temp folder", 
right before make.exe tries to access it in the mentioned error situation:
"06:58:18.8638327","distcc.exe","364","CreateFile","C:\Documents and 
Settings\...\Local Settings\Temp","SUCCESS","Desired Access: Read EA, Read 
Attributes, Read Control, Disposition: Open, Options: Open For Backup, Open 
Reparse Point, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: 
n/a, OpenResult: Opened"
"06:58:18.8648711","distcc.exe","364","CreateFile","C:\Documents and 
Settings\...\Local Settings\Temp","SHARING VIOLATION","Desired Access: Delete, 
Disposition: Open, Options: Open For Backup, Attributes: n/a, ShareMode: 
Delete, AllocationSize: n/a"
"06:58:18.8786835","distcc.exe","364","CreateFile","C:\Documents and 
Settings\...\Local Settings\Temp","SUCCESS","Desired Access: Read Data/List 
Directory, Delete, Synchronize, Disposition: Open, Options: Synchronous IO 
Non-Alert, Open For Backup, Attributes: n/a, ShareMode: Read, Write, Delete, 
AllocationSize: n/a, OpenResult: Opened"
"06:58:18.8787020","cmd.exe","5196","CreateFile","C:\Documents and 
Settings\...\Local Settings\Temp","SHARING VIOLATION","Desired Access: Read 
Data/List Directory, Synchronize, Disposition: Open, Options: Directory, 
Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, 
AllocationSize: n/a"

Another, somewhat unrelated, thing is that distcc (on Cygwin) does not respect 
the TMPDIR env var, which I've explicitly set to something different than 
"C:\Documents and Settings\...\Local Settings\Temp"

Also note that when I set DISTCC_SAVE_TEMPS=1 the problem was not reproducible 
anymore. Moreover, the "dcc_cleanup_tempfiles_inner" distcc.log entries 
disappeared.


Solution:
---------
The root cause of the problem seems to be in dcc_get_tmp_top:
 1. The Windows/Cygwin version does not respect the TMPDIR env var. Not checking this env var must be a bug unless the distcc man page is wrong
 2. The Windows/Cygwin version registers the directory gotten from the from Windows functions (GetTempPath) at 'dcc_add_cleanup'. This is the reason why distcc later tries to delete this folder. This looks very strange to me, as distcc should better not delete the Windows temp folder, probably for the same reasons as it doesn't remove '/tmp' when run on *nix systems. A second reason for not deleting it is that distcc never creates the temporary directory if it doesn't exist (empirically tested)

Making a long story short: According to my understanding, dcc_get_tmp_top 
should look something like the version in the attached patch. With that patch 
applied, the mentioned problems are not reproducible anymore.

Raphael

Original issue reported on code.google.com by zull...@gmail.com on 20 Feb 2013 at 12:42

Attachments:

• distcc.patch