bazel clean --expunge often fails on Windows

[rokstrnisa]

$ bazel clean --expunge
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
ERROR: C:/tools/msys64/var/tmp/Bazel/$KNt4ZiP/command.log (Permission denied).

command.log contains no information about the bazel clean --expunge failure, only output of the previous build commands.

[laszlocsomor]

The command.log is what contains the output above, so the clean command is trying to log the clean event and delete the log file at the same time.

[laszlocsomor]

I did some debugging. On Linux we can delete the command.log at any time using rm, but on Windows we always get "Device or resource busy" as long as the bazel server is running (i.e. until you run bazel shutdown).

[jakemac53]

I get a similar issue when using new_local_repository, it seems the tools\msys64 directory is read only, and it keeps reverting back to read only on its own if I change it....

[laszlocsomor]

@jakemac53 : Thanks for reporting that. It doesn't sound like the same issue to me. Is it reproducible? If so, could you please report a separate issue for it?

As for this bug I made sure the command.log is closed before attempting to actually clean the output tree and now the command can proceed, but then finds another open file jvm.out. We may need to find a principled way to close all file handles before cleaning. The alternative would be to open files with delete permission (I recall CreateFile has something similar on Windows) but I haven't found any way to do that with java.io/java.nio.

[petemounce]

Is 7f3ffbc in 0.4.0-rc3?

λ bazel clean --expunge
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
ERROR: C:/tools/msys64/var/tmp/Bazel/pkGed45b/command.log (Permission denied).

[laszlocsomor]

The release candidate was cut prior to this commit. :(
@damienmg , can we cut the new RC at a later commit? This is not the first change I'd like to get into 0.4

[damienmg]

+Kristina Chodorow kchodorow@google.com is the 0.4 release manager. I
guess we can cherry-pick it since a RC today still means we can release on
wednesday. How important it is?

On Mon, Oct 31, 2016 at 4:57 PM László Csomor notifications@github.com
wrote:

The release candidate was cut prior to this commit. :(
@damienmg https://github.com/damienmg , can we cut the new RC at a
later commit? This is not the first change I'd like to get into 0.4

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#1586 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADjHfyhd_e6pkSeTEEOd4JhWmStC6q7rks5q5g_mgaJpZM4JYSwo
.

[laszlocsomor]

It's a P1, so important. When's the next release expected, if this doesn't make into 0.4?

[damienmg]

Every month so somewhere in November.

On Mon, Oct 31, 2016 at 5:24 PM László Csomor notifications@github.com
wrote:

It's a P1, so important. When's the next release expected, if this doesn't
make into 0.4?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#1586 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADjHf7SXUSL5F9MoLcF_O7Izlxa9swPnks5q5hY4gaJpZM4JYSwo
.

[ulfjack]

It's certainly in 0.4.2 now.

[laszlocsomor]

We still see "bazel clean" failures sometimes, and it may be due to the persistent java workers holding on to open files, which is also the suspected culprit of #2538.

[dslomov]

@laszlocsomor @philwo what's the right target milestone? Is it still a P1?

[philwo]

(Sorry, wrong bug. Moving my comment to the correct one.)

[laszlocsomor]

Sorry, dcaee09 is an irrelevant commit.

[damienmg]

Steren probably have the wrong bug number in he change

[Helcaraxan]

This is still an issue.
From what I have seen @laszlocsomor's comment #1586 (comment) is spot-on as I have to kill all Bazel processes manually before the bazel clean works again.

[philwo]

I ran into this just now:

> bazel clean --expunge                                                                                                   
INFO: Reading 'startup' options from C:\Users\philwo/.bazelrc: --output_user_root=c:/src/_bazel_philwo                    
INFO: Starting clean (this may take a while). Consider using --expunge_async if the clean takes more than several minutes.
ERROR: C:/src/_bazel_philwo/o7dye3qe/java.log (Permission denied)                                                         

@laszlocsomor explained that it happens because the Java server tries to delete its log file, which it holds an open handle to (because its stdout is redirected to that file) and thus fails to delete it. We could create the file with delete sharing permissions, which would allow the server to delete it despite the open handle, but then writes to the deleted file will be lost.

@philwo pointed out that we only try to delete the log file with clean --expunge, but when doing so, we also exit the server process after the clean command finished - so the situation where we try to write to the deleted log file should actually never happen (and even if it happens, it's probably rather irrelevant).

[laszlocsomor]

@laszlocsomor explained that it happens because the Java server tries to delete its log file

I realized this is wrong. The file in question is "java.log", which is created and held open by the java process, see startup_options.cc:416. The file I talked about earlier was "jvm.log", which is indeed created in blaze::CreateJvmOutputFile so we could open that with deletion sharing.

[laszlocsomor]

Related bugs: #3043, #2480, #1906