bazel fails if workspace is not on C: #1463

Closed
tomzx opened this Issue Jul 1, 2016 · 47 comments

Projects

None yet
@tomzx
tomzx commented Jul 1, 2016 edited

Hi,

I'm trying to build master (a18add1) following the instruction provided and I'm getting the error below. The path to my java jdk is correct and the folder does indeed exist. Furthermore, I can see that the first part of the compile script calls javac from that directory without any issue.

I'm willing to try and debug this out if someone is willing to provide me with some guidance.

Thanks!

$ ./compile.sh
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    ./compile.sh compile /path/to/bazel
🍃  Building Bazel from scratch......
🍃  Building Bazel with Bazel.
.WARNING: C:/tools/msys64/tmp/bazel.jR1zd19q/out/external/bazel_tools/WORKSPACE:1: Workspace name in C:/tools/msys64/tmp/bazel.jR1zd19q/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
ERROR: no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_73 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
ERROR: no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_73 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
INFO: Elapsed time: 1.048s

Building output/bazel
@aehlig
Member
aehlig commented Jul 4, 2016

Assigning to @dslomov who has more knowledge about Windows.

CC @kchodorow

@dslomov dslomov was assigned by aehlig Jul 4, 2016
@Lunarsong

I'm seeing the same error here when trying to build 89484da .

@petemounce
Contributor
petemounce commented Aug 17, 2016 edited

I am able to compile bazel tags 0.3.0 and 0.3.1, and current master (60af9db) on Windows 10, but I get the same error when trying to compile (with either version) my in-house project.

Sample output:

$ bazel build --verbose_failures --cpu='x64_windows_msvc' --host_cpu='x64_windows_msvc' cpp/protocgenfabric/...                .
ERROR: no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
ERROR: no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
INFO: Elapsed time: 1.948s

(There's also a reference to running blaze clean which I guess is a typo)

I should mention - that JDK directory definitely exists.

To try to eliminate the space in the Program Files being an issue, I've used an admin prompt to do mklink /j prfi "c:\Program Files". That hasn't helped:

Pete@petemounce-win-imp MSYS /s/e
$ echo $JAVA_HOME
C:/Program Files/Java/jdk1.8.0_102

Pete@petemounce-win-imp MSYS /s/e
$ export JAVA_HOME=c:/prfi/Java/jdk1.8.0_102

Pete@petemounce-win-imp MSYS /s/e
$ $JAVA_HOME/bin/java -version
java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

Pete@petemounce-win-imp MSYS /s/e
$ $JAVA_HOME/bin/javac -version
javac 1.8.0_102

Pete@petemounce-win-imp MSYS /s/e
$ bazel build --verbose_failures --cpu='x64_windows_msvc' --host_cpu='x64_windows_msvc' cpp/protocgenfabric/...
ERROR: in target '//external:jdk-default': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
INFO: Elapsed time: 0.280s

Where is it that bazel is deciding to use that JDK if not the JAVA_HOME variable?

@divydal
divydal commented Aug 25, 2016

Hi, I have the same problem when trying to build 7b5e1af

@sunsided
sunsided commented Sep 6, 2016

Same here, trying to build Bazel master (180d1b5) or 0.3.1 on a German Windows 10 Version 1607, using msys2 x64 and jdk1.8.0_74:

$ ./compile.sh
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    ./compile.sh compile /path/to/bazel
🍃  Building Bazel from scratch.......
D:\dev\FremdeSources\bazel>call "C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/VCVARSALL.BAT" amd64
Microsoft (R) C/C++-Optimierungscompiler Version 19.00.24210 für x64
Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten.

windows_error_handling.cc
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocale(341): warning C4530: C++-Handler verwendet, aber Entladesemantik ist nicht aktiviert. Geben Sie /EHsc an.
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\exception(359): warning C4577: "noexcept" wird ohne angegebenen Ausnahmebehandlungsmodus verwendet. Die Beendigung bei einer Ausnahme ist nicht sichergestellt. Geben Sie "/EHsc" an.
windows_file_operations.cc
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocale(341): warning C4530: C++-Handler verwendet, aber Entladesemantik ist nicht aktiviert. Geben Sie /EHsc an.
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\exception(359): warning C4577: "noexcept" wird ohne angegebenen Ausnahmebehandlungsmodus verwendet. Die Beendigung bei einer Ausnahme ist nicht sichergestellt. Geben Sie "/EHsc" an.
windows_processes.cc
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocale(341): warning C4530: C++-Handler verwendet, aber Entladesemantik ist nicht aktiviert. Geben Sie /EHsc an.
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\iosfwd(343): warning C4577: "noexcept" wird ohne angegebenen Ausnahmebehandlungsmodus verwendet. Die Beendigung bei einer Ausnahme ist nicht sichergestellt. Geben Sie "/EHsc" an.
Code wird generiert...
Microsoft (R) Incremental Linker Version 14.00.24210.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/dll
/implib:D:\dev\msys64\tmp\bazel_N9GyDiSO\jni\windows_jni.lib
/out:D:\dev\msys64\tmp\bazel_N9GyDiSO\jni\windows_jni.dll
windows_error_handling.obj
windows_file_operations.obj
windows_processes.obj
   Bibliothek "D:\dev\msys64\tmp\bazel_N9GyDiSO\jni\windows_jni.lib" und Objekt "D:\dev\msys64\tmp\bazel_N9GyDiSO\jni\windows_jni.exp" werden erstellt.
.
🍃  Building Bazel with Bazel.
.WARNING: D:/dev/msys64/tmp/bazel_vSsqtL7b/out/external/bazel_tools/WORKSPACE:1: Workspace name in D:/dev/msys64/tmp/bazel_vSsqtL7b/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
ERROR: in target '//external:jdk-default': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_74 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
INFO: Elapsed time: 0,832s

and on subsequent calls to compile.sh:

$ ./compile.sh
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO:    ./compile.sh compile /path/to/bazel
🍃  Building Bazel from scratch......
🍃  Building Bazel with Bazel.
.WARNING: D:/dev/msys64/tmp/bazel_cJf7e10a/out/external/bazel_tools/WORKSPACE:1: Workspace name in D:/dev/msys64/tmp/bazel_cJf7e10a/out/external/bazel_tools/WORKSPACE (@io_bazel) does not match the name given in the repository's definition (@bazel_tools); this will cause a build error in future versions.
ERROR: no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_74 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
ERROR: no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_74 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
INFO: Elapsed time: 0,798s
@petemounce
Contributor
petemounce commented Sep 17, 2016 edited

This is still a problem for me in my own source tree (but not bazel's) at current HEAD, c484f19.

I have Windows 10 Anniversary Update, Bash-on-Windows is installed. I have jdk1.8.0_102

I get the same error with powershell (since #1453 allows its use) and msys2 x64. Edit: I get the same error regardless of whether b-o-w bash.exe appears first or later than msys2's in PATH I think.

@sunsided @divydal do you have b-o-w too...?

@sunsided

@petemounce Yes, installed. Didn't try it though because I need Windows binaries.

@jared-improbable

This is also an issue for me on Windows 10, using the nightly built here

$ bazel-nightly.exe info
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:473:12: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:474:13: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:475:14: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:480:12: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:481:13: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:482:11: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:493:11: Variables HOST_CFG and DATA_CFG are deprecated in favor of strings "host" and "data" correspondingly.
WARNING: C:/Users/jared/AppData/Local/Temp/Bazel/9ncInwMl/external/io_bazel_rules_scala/scala/scala.bzl:545:21: Argument `cfg = "host"` or `cfg = "data"` is required if `executable = True` is provided for a label.
ERROR: in target '//external:jdk-default': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).```

`C:/Program Files/Java/jdk1.8.0_102` does exist.
@dslomov
Contributor
dslomov commented Oct 4, 2016

We were able to repro this when a workspace is not on C: drive. Fix for that will hopefully be coming soon, but is it the case for everyone reporting this?

@petemounce
Contributor

Yes, my source is on my s: drive. Good thought!

@divydal
divydal commented Oct 4, 2016

Yes , my workspace is on d:.

@sunsided
sunsided commented Oct 4, 2016

Same, sources on D:

@tomzx
tomzx commented Oct 4, 2016 edited

@dslomov Yes mine isn't stored on the C: drive. However I did try to create a symlink on the C:, assuming it might related to that, but that didn't work either. I'm assuming symlinks are resolved?

@petemounce
Contributor

@tomzx how did you create the symlink (ie, exact command)? There are a couple of different types on Windows, with differing characteristics.

@tomzx
tomzx commented Oct 4, 2016 edited

@petemounce IIRC, mklink /D C:\bazel E:\bazel.

@petemounce
Contributor

Ah ok. That's what I would have tried too. Do any of the methods listed in http://superuser.com/questions/752538/mklink-vs-junction-exe allow workaround?

@dslomov dslomov assigned laszlocsomor and unassigned dslomov Oct 5, 2016
@dslomov dslomov added this to the 0.4 milestone Oct 5, 2016
@laszlocsomor
Contributor

FTR i just did subst d: c:\d_drive and that works too (bonus: good old #1744 still around):

$ bazel info
ERROR: CreateFile(C:\tools\msys64\var\tmp\Bazel\WLSiXh7n\install): The system cannot find the file specified.
 (2)
...........
ERROR: in target '//external:jdk-default': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build)

I'll dive into that part of the code and see what I find.

@petemounce
Contributor
petemounce commented Oct 6, 2016 edited

c484f19: When I run bazel commands on my c: drive, they work; when I run them from my d: drive then I get this.

I tested this by running bazel build and bazel info in the root of this repo while on c: and d:, same system as I reported from. I get this error when on d:.

I get it inside msys2 shell and powershell. Unfortunately I can't easily test this against my in-house project because that would currently mean moving it to an unencrypted drive, which I won't do. Still, pretty indicative.

(Also, same when I run the bazel.exe that lives on c: from my d: drive, fwiw, by fully qualifying the path to the exe)

@laszlocsomor
Contributor

@petemounce : Thanks for reporting it. I'm seeing the same. Looking for the root cause and fixing soon.

@laszlocsomor
Contributor
laszlocsomor commented Oct 6, 2016 edited

The problem is rooted in a design limitation of Bazel, namely that a file system is assumed to have a single root. That's not true on Windows, and so the hack we've been using is treating "C:" as a top-level directory, not a root. WindowsFileSystem.getFilesystemRoot() reports / as the root.

This is breaking down if the workspace is on another drive, and we have to fix this design flaw properly.

This hack also means that all RootedPath objects are incorrect in the sense that their root is / and not the drive letter, so the RootedPath for C:\ itself is [/]/[C:].

No PathFragment objects are absolute on Windows because C: is just a directory, / is the root.

These factors lead to more problems down the road, for example in FileFunction.resolveFromAncestors where:

rootedPath = "/C:"  // root="/", relative="C:"
relativePath = "C:"
baseName = "C:"
parentRealRootedPath = "/"  // root="/", relative=""
parentRealRootedPath.getRelativePath() = ""
parentRealRootedPath.getRelativePath().getRelative(baseName) = ""
---> thus realRootedPath = "/"  // root="/", relative=""

Which means that we attempt to evaluate C:-relative paths relative to the current drive (D:), and this manifests as incorrect-looking missing file errors.

@laszlocsomor
Contributor

One more factor: when a PathFragment is just C:, then its segments are empty, though isAbsolute = false, and driveLetter = 'C'. When it's a path like C:/Program Files, then the driveLetter = 'C' (fine), isAbsolute = false (still not fine I guess) but segments = ["C:", "Program Files"]. This discrepancy is bad, this is what's causing parentRealRootedPath.getRelativePath().getRelative(baseName) in my previous comment to be "/", because parentRealRootedPath.getRelativePath() returns the empty PathFragment, and baseName is PathFragment("C:"), and the .getRelative call joins the segments which neither of these PathFragments have any of.

@laszlocsomor
Contributor

@dslomov : Your workaround idea works!

Workaround: create a junction somewhere on "C:" pointing to the workspace. Then bazel works.

To fix this bug properly we have to introduce the concept of multiple filesystem roots.

@laszlocsomor
Contributor

Dropping priority because there's a workaround.

@laszlocsomor laszlocsomor added P2 and removed P1 labels Oct 6, 2016
@petemounce
Contributor

@laszlocsomor So, the workaround is to (from an admin cmd shell) mklink /d c:\<somewhere> <actual workspace>, right...? I wasn't able to get it to work:

# I have my bazel clone at d:\bazel for testing this
mklink /d c:\bazel d:\bazel
c:
dir

 Volume in drive C is foobar
 Volume Serial Number is blahblah

 Directory of c:\

....redacted...

06/10/2016  13:50    <SYMLINKD>     bazel [d:\bazel]

....redacted...

cd c:\bazel
powershell #which has a profile that adds msys DLL to my PATH
bazel info
ERROR: in target '//external:jdk-default': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
@laszlocsomor
Contributor

@petemounce : I did this from a non-Admin command prompt:

mklink /j c:\work\bazel_ws d:\path\to\actual\bazel_ws

Then in msys, I cd'd into /c/work/bazel_ws and build there.

@laszlocsomor
Contributor
laszlocsomor commented Oct 6, 2016 edited

FYI, mklink /d creates a directory symlink, mklink /j creates a junction. Here's the difference explained: http://superuser.com/a/343079

@petemounce
Contributor
petemounce commented Oct 6, 2016 edited

That didn't work for me (Windows 10 anniversary update)

In non-admin cmd shell:

rmdir c:\bazel # remove previous experiment
mklink /j c:\bazel d:\bazel
Junction created for c:\bazel <<===>> d:\bazel
set PATH=c:\tools\msys64\usr\bin;%PATH%
dir

 Volume in drive C is foobar
 Volume Serial Number is blahblah

 Directory of C:\
.....
06/10/2016  14:12    <JUNCTION>     bazel [d:\bazel]
......


 cd c:\bazel

bazel info
ERROR: in target '//external:jdk-default': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).

With msys2:

Pete@foobar MSYS ~
$ cd /c/bazel

Pete@foobar MSYS /c/bazel
$ /c/Users/Pete/bazel/output/bazel.exe info # bazel is not on my PATH in msys2
ERROR: in target '//external:jdk-default': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
@tomzx
tomzx commented Oct 6, 2016 edited

Same result as @petemounce for me on Windows 7 SP1. However in my case, I'm still stuck at the ./compile.sh step.

@laszlocsomor
Contributor

@petemounce , @tomzx : It works if the junction is not the last path element. I don't know why and I haven't investigated (and probably won't unless I have to) but here it is.

My D: drive is a mapped folder from C: (using subst, see #1463 (comment)), and my E: is a USB mass storage.

I have the following bazel workspaces:

d:\bazel_ws
d:\bazel\ws
e:\bazel_ws

In cmd.exe:

C:\>mklink /j bazel_d1 d:
Local volumes are required to complete the operation.

C:\>mklink /j bazel_d1 d:\
Local volumes are required to complete the operation.

C:\>mklink /j bazel_d1 d:\bazel_ws
Junction created for bazel_d1 <<===>> d:\bazel_ws

C:\>mklink /j bazel_d2 d:\bazel\
Junction created for bazel_d2 <<===>> d:\bazel\

C:\>mklink /j bazel_e1 e:\bazel_ws
Junction created for bazel_e1 <<===>> e:\bazel_ws

C:\>mklink /j bazel_e2 e:
Junction created for bazel_e2 <<===>> e:

C:\>dir bazel_d1
...
2016-10-05  11:11 AM    <DIR>          foo
2016-10-05  11:11 AM                 0 WORKSPACE
...

C:\>dir bazel_d2
...
2016-10-06  03:37 PM    <DIR>          ws
...

C:\>dir bazel_e1
...
2016-10-06  01:29 PM                13 hello.txt
2016-10-06  01:30 PM                 0 WORKSPACE
...

C:\>dir bazel_e2
...
2016-10-06  01:29 PM    <DIR>          bazel_ws
...

(Note that I couldn't create a junction just to D:, only to a subdirectory in it. I don't know why this is, probably a limitation of NTFS.)

In msys:

$ cd /c/bazel_d1 ; bazel --batch info >&/dev/null ; echo $?
2

$ cd /c/bazel_d2/ws ; bazel --batch info >&/dev/null ; echo $?
0

$ cd /c/bazel_e1 ; bazel --batch info >&/dev/null ; echo $?
2

$ cd /c/bazel_e2/bazel_ws ; bazel --batch info >&/dev/null ; echo $?
0
@dslomov dslomov changed the title from "Inconsistent filesystem operations" on Windows to bazel fails if workspace is not on C: Oct 7, 2016
@laszlocsomor laszlocsomor added P1 and removed P2 labels Oct 7, 2016
@laszlocsomor
Contributor

Bumping priority again because this is a serious blow on usability.

@laszlocsomor
Contributor

I have a fix for this, stay tuned.

@laszlocsomor
Contributor

Out for review. If all goes well, we can push the fix to GH today or tomorrow.

@petemounce
Contributor

Hi @laszlocsomor - I didn't spot this on [gerrit|https://bazel.googlesource.com/bazel/]. Is it somewhere else? Happy to try it out, since it's the next thing to a blocker for us.

@laszlocsomor
Contributor

Hey @petemounce , it's not yet submitted, sorry about that. It should be in today or Monday, if not I'll ping this thread.

@youkpan
youkpan commented Oct 15, 2016

seems java.exe path problem

set JAVA_HOME=C:/Program\ Files/Java/jdk1.8.0_101&c:\bazel.exe
Couldn't find java at 'C:/Program\ Files/Java/jdk1.8.0_101/bin/java'.

@dslomov
Contributor
dslomov commented Oct 15, 2016

@youkpan this is not really related to this bug, is it? In the future, please file a separate issue if the problem is unrelated to the problem discussed in the thread.
Anyhow, in your case the content of JAVA_HOME variable after your command is literally C:/Program\ Files/Java/jdk1.8.0_101/bin/java, including a \ before the space. The following should work:

C:> set JAVA_HOME=C:/Program Files/Java/jdk1.8.0_101
C:> bazel.exe
@dslomov dslomov referenced this issue in tensorflow/tensorflow Oct 15, 2016
Closed

Windows Support and Documentation #17

@petemounce
Contributor

For later reference, I'm able to work around this by

  • having my source code inside a sub directory on my separate drive (s:\whatever\here)
  • create a junction to that from inside a directory on c:
    • mklink /j c:\src\s-drive s:\whatever\here
@laszlocsomor
Contributor

FYI I'm still working on this change, still not submitted.

@laszlocsomor
Contributor

Submitted internally, will push it to GitHub in the next 24 hours.

@laszlocsomor
Contributor

Looks like I made a mistake in WindowsFileSystem.java retrieving UNIX_ROOT (if you don't see that then the change is not on GitHub yet), but I sent out a fix for it. If I'm lucky we'll push both changes to GH in the same run.

@laszlocsomor
Contributor

If not, then in the meantime the workaround will be to pass --host_jvm_args=-Dbazel.windows_unix_root="C:/tools/msys64/" to bazel.

@bazel-io bazel-io pushed a commit that referenced this issue Oct 18, 2016
@laszlocsomor @philwo laszlocsomor + philwo VFS: implement a Windows-specific Path subclass
The new subclass WindowsFileSystem.WindowsPath is
aware of Windows drives.

This change:
- introduces a new factory for Path objects so
  FileSystems can return a custom implementation
  that instantiates filesystem-specific Paths
- implements the WindowsPath subclass of Path that
  is aware of Windows drives
- introduces the bazel.windows_unix_root JVM
  argument that defines the MSYS root, which
  defines the absolute Windows path that is the
  root of all Unix paths that Bazel creates (e.g.
  "/usr/lib" -> "C:/tools/msys64/usr/lib") except
  if the path is of the form "/c/foo" which is
  treated as "C:/foo"
- removes all Windows-specific logic from Path

PathFragment is still aware of drive letters and
it has to remain so because it is unaware of file
systems.

WindowsPath restricts the allowed path strings to
absolute Unix paths where the first segment, if
any, is a volume specifier. From now on if Bazel
attempts to create a WindowsPath from an absolute
Unix path, Bazel will make it relative to
WindowsPath.UNIX_ROOT, unless the first component
is a single-letter name (e.g. "/c/foo" which is
"C:/foo").

Subclassing Path is necessary because a Unix-style
absolute path doesn't sufficiently define a full
Windows path, as it may be relative to any drive.

Fixes #1463

--
MOS_MIGRATED_REVID=136350304
e0d7a54
@bazel-io bazel-io pushed a commit that referenced this issue Oct 18, 2016
@laszlocsomor @philwo laszlocsomor + philwo VFS, WindowsFileSystem: fix UNIX_ROOT retrieval
Executing bash.exe directly instead of through
cmd.exe doesn't seem to work. This change fixes
that problem.

Fixes #1463 (again)

--
MOS_MIGRATED_REVID=136364606
8bb4299
@bazel-io bazel-io pushed a commit that referenced this issue Oct 19, 2016
@philwo philwo Rollback of commit 8bb4299.
*** Reason for rollback ***

Suspected root cause for Windows bootstrap on Bazel CI breakage:

java.lang.NullPointerException
	at com.google.devtools.build.lib.vfs.Path$1.run(Path.java:123)

http://ci.bazel.io/view/Bazel%20bootstrap%20and%20maintenance/job/Bazel/922/JAVA_VERSION=1.8,PLATFORM_NAME=windows-x86_64/console

*** Original change description ***

VFS, WindowsFileSystem: fix UNIX_ROOT retrieval

Executing bash.exe directly instead of through
cmd.exe doesn't seem to work. This change fixes
that problem.

Fixes #1463 (again)

--
MOS_MIGRATED_REVID=136581532
4b4b9d3
@laszlocsomor laszlocsomor reopened this Oct 20, 2016
@laszlocsomor
Contributor

I figured out why the bootstrapping job was broken and why we had to roll this change back. The WindowsPath.translateParent call returned a different parent for top-level directories (UNIX_ROOT instead of /), but we registered the child with the old parent, not the new one.

@laszlocsomor
Contributor

Fix is out for review, stay tuned.

@bazel-io bazel-io pushed a commit that closed this issue Oct 25, 2016
@laszlocsomor @katre laszlocsomor + katre VFS: implement a Windows-specific Path subclass
This change rolls forward commit e0d7a54 and
commit 8bb4299, and adds a bugfix:
- FileSystem.PathFactory got a new translatePath
  method that WindowsFileSystem.PathFactory
  overrides to translate absolute Unix paths to
  MSYS-relative paths
- Path.getCachedChildPath calls this translatePath
  method so the child path is registered with the
  correct (translated) parent and under the
  correct name (e.g. "C:" instead of say "c")

Below is the rest of the original change
description:

The new subclass WindowsFileSystem.WindowsPath is
aware of Windows drives.

This change:
- introduces a new factory for Path objects so
  FileSystems can return a custom implementation
  that instantiates filesystem-specific Paths
- implements the WindowsPath subclass of Path that
  is aware of Windows drives
- introduces the bazel.windows_unix_root JVM
  argument that defines the MSYS root, which
  defines the absolute Windows path that is the
  root of all Unix paths that Bazel creates (e.g.
  "/usr/lib" -> "C:/tools/msys64/usr/lib") except
  if the path is of the form "/c/foo" which is
  treated as "C:/foo"
- removes all Windows-specific logic from Path

PathFragment is still aware of drive letters and
it has to remain so because it is unaware of file
systems.

WindowsPath restricts the allowed path strings to
absolute Unix paths where the first segment, if
any, is a volume specifier. From now on if Bazel
attempts to create a WindowsPath from an absolute
Unix path, Bazel will make it relative to
WindowsPath.UNIX_ROOT, unless the first component
is a single-letter name (e.g. "/c/foo" which is
"C:/foo").

Subclassing Path is necessary because a Unix-style
absolute path doesn't sufficiently define a full
Windows path, as it may be relative to any drive.

Fixes #1463

--
MOS_MIGRATED_REVID=137149483
ca99bb7
@bazel-io bazel-io closed this in ca99bb7 Oct 25, 2016
@petemounce
Contributor

@laszlocsomor Thanks! Just tried this out with 0.4.0-rc3; works great.

@petemounce
Contributor

@laszlocsomor Actually, weirdly, this just failed from within powershell and msys2. I'm not sure how I tested, above, coming back to this.

λ bazel build //Cpp/WorkerSdk:WorkerSdk
.....
ERROR: in target '//external:jdk': no such package '@local_jdk//': com.google.devtools.build.lib.skyframe.InconsistentFilesystemException: Inconsistent filesystem operations. C:/Program Files/Java/jdk1.8.0_102 is no longer an existing directory. Did you delete it during the build? The results of the build are not guaranteed to be correct. You should probably run 'blaze clean' and investigate the filesystem inconsistency (likely due to filesytem updates concurrent with the build).
INFO: Elapsed time: 0.234s
s:\everything [feature/WIT-761-bazel-for-cpp-on-windows ≡ +2 ~2 -0 !]
λ bazel version
Build label: 0.4.0rc3
Build target: bazel-out/local-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Oct 26 13:40:38 2016 (1477489238)
Build timestamp: 1477489238
Build timestamp as int: 1477489238
@laszlocsomor
Contributor

@petemounce : Indeed, the 0.4.0-rc3 doesn't contain this change. I assume it was rolled back at the time we cut the canary; we had to roll it back and forward a couple of times. It works for me at 9b61b0f, both under msys and powershell.

For powershell I have to set some envvars and bazel prints some errors -- i don't know if it's normal, i never used bazel under PS:

PS C:\work\bazel2> $global:PATH = $env:Path

PS C:\work\bazel2> $global:BAZEL_SH = "c:\tools\msys64\usr\bin\bash.exe"

PS C:\work\bazel2> cd d:\bazel_ws

PS D:\bazel_ws> C:/tools/msys64/var/tmp/Bazel/MGAgSgPx/execroot/bazel2/bazel-out/local-fastbuild/bin/src/bazel.exe --batch info --color=no --curses=no
ERROR: 'null' is not recognized as an internal or external command,
operable program or batch file.
 (exit code: 1)
WARNING: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/external/bazel_tools/tools/cpp/cc_configure.bzl:57:3:
Auto-Configuration Warning: 'BAZEL_PYTHON' is not set, start looking for python in PATH.
WARNING: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/external/bazel_tools/tools/cpp/cc_configure.bzl:57:3:
Auto-Configuration Warning: Python found at C:/Python27/python.exe
WARNING: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/external/bazel_tools/tools/cpp/cc_configure.bzl:57:3:
Auto-Configuration Warning: 'BAZEL_VS' is not set, start looking for the latest Visual Studio installed.
WARNING: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/external/bazel_tools/tools/cpp/cc_configure.bzl:57:3:
Auto-Configuration Warning: 'BAZEL_SH' is not set, start looking for bash in PATH.
WARNING: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/external/bazel_tools/tools/cpp/cc_configure.bzl:57:3:
Auto-Configuration Warning: Bash binary found at C:/tools/msys64/usr/bin/bash.exe
WARNING: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/external/bazel_tools/tools/cpp/cc_configure.bzl:57:3:
Auto-Configuration Warning: Visual Studio found at C:\Program Files (x86)/Microsoft Visual Studio 14.0
bazel-bin: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/execroot/bazel_ws/bazel-out/local-fastbuild/bin
bazel-genfiles: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/execroot/bazel_ws/bazel-out/local-fastbuild/genfiles
bazel-testlogs: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/execroot/bazel_ws/bazel-out/local-fastbuild/testlogs
command_log: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/command.log
committed-heap-size: 301MB
execution_root: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/execroot/bazel_ws
gc-count: 4
gc-time: 59ms
install_base: C:/tools/msys64/var/tmp/_bazel_laszlocsomor/install/e456d4f6cf0d1a673f1ea38bb8525ecf
max-heap-size: 7621MB
message_log: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/message.log
output_base: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n
output_path: C:/tools/msys64/var/tmp/Bazel/WLSiXh7n/execroot/bazel_ws/bazel-out
package_path: %workspace%
release: development version
server_pid: 1644
used-heap-size: 73MB
workspace: D:/bazel_ws
@petemounce
Contributor
petemounce commented Oct 31, 2016 edited

@laszlocsomor in the docs, BAZEL_SH wants path separators to be / not \. That's what my own var is set with. Also, I set those via $env:BAZEL_SH if I do them manually within a powershell, rather than $global:BAZEL_SH.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment