I have a Wix project being built from within visual studio. I have several calls to the task within . When I do a build, it completes successfully, but devenv still has handles open to all the DLLs and EXEs that heat ran over. Since those are the DLLs and EXEs in the project ouput folder, I can't do any more builds because the output files cannot be written to. I have to close visual studio and start it again after every build of the wix project :(
If I comment out the calls, those handles don't get left open.
What's left open: 32-bit managed DLLs and EXEs. The non-managed DLLs that are covered by the same calls to heat are not getting left open, nor are the 64-bit managed DLLs.
The workaround did not work for me. I ran procmcon to look for heat.exe running, or for anything touching the output files from heat. Even with the workaround in place, heat.exe never ran, and all access to the output files was from devenv.exe, so either I didn't manage to get that property into the project correctly (I put it at the very start of the project file), or the HeatDirectory task isn't obeying it.
An additional note: looking at devenv in process explorer when the files are stuck open, it's not just that the files are open, but the assemblies show up in the list of loaded DLLs. Which suggests why the 64 bit managed dlls aren't getting stuck, since the 32 bit devenv process can't load 64 bit DLLs.
Also, FWIW, the wix build I'm using is 3.5.2325.0, I've seen this general behavior with prior 3.5 builds as well (never got around to investigating it in depth before).
OK, I at least figured out why the workaround didn't work ... that property only applies when using items, not when explicitly calling the task. Setting the corresponding RunAsSeparateProcess attribute on each HeatDirectory task made heat run out of process, and avoiding the handle leak in devenv.