Windows: Menhir dependency fails to build

[bryphe]

esy version: master + Windows fixes
Operating System: Windows

Issue:
menhir fails to install with this error when running the ocamlbuild -where command:

     'C:\Users\bryph\.esy\3_\b\opam__slash__menhir-20171013.0.0-05503f0d\src\_stage1'
         The system cannot find the path specified.
         Warning: Won't be able to compile a native plugin
         Failure: Cannot find "ocamlbuild.cmo" in ocamlbuild -where
     directory.
         make[2]: *** [Makefile:38: stage1] Error 2

Some notes on my environment:

• ocamlbuild seems to be installed successfully - it is available here: C:/Users/bryph/.esy/3_/i/opam__slash__ocamlbuild-0.12.0-5e55ac68/bin

Running __ocamlbuild -where from the build environment gives some strange results - it returns C:/Users/bryph/.esy/3_/s/opam__slash__ocamlbuild-0.12.0-5e55ac68/lib/ocamlbuild, which is the staging folder. I validated that the library ocamlbuild.cmo is actually available in the correct folder, the install folder: C:/Users/bryph/.esy/3_/i/opam__slash__ocamlbuild-0.12.0-5e55ac68/lib/ocamlbuild

So the mystery that needs to be solved is - why is ocamlbuild -where pointing to the staging folder instead of the install folder? It's possible that something isn't wired up quite right in the environment.

One other thing I noticed, that may or may not be related, is that the esy-installer step failed for ocamlbuild:

# esy-build-package: running: "sh" "-c" "(esy-installer || true)"

[DEBUG]: Running command: "node" "E:/esy-bash/bin/esy-bash.js" "--env" "E:/test-file9_1531265326..dat" "sh" "-c" "(esy-installer || true)"

esy-bash: executing bash command: sh "-c" "(esy-installer || true)" | nonce: 1
 -- using environment file: E:/test-file9_1531265326..dat
 -- script file: C:/Users/bryph/AppData/Local/Temp/__esy-bash__1531265326858__1__.sh
ocamlbuild.byte                  => C:/Users/bryph/.esy/3_/s/opam__slash__ocamlbuild-0.12.0-5e55ac68/bin/ocamlbuild.byte
[ERROR] Failure during install
# opam-version         1.2.2
# os                   cygwin
# command              install -m 0755 /cygdrive/c/Users/bryph/.esy/3_/b/opam__slash__ocamlbuild-0.12.0-5e55ac68/ocamlbuild.byte C:/Users/bryph/.esy/3_/s/opam__slash__ocamlbuild-0.12.0-5e55ac68/bin/ocamlbuild.byte
# path                 /cygdrive/c/Users/bryph/.esy/3_/b/opam__slash__ocamlbuild-0.12.0-5e55ac68
# exit-code            1
# env-file             /tmp/opam-5568/log/log-5568-5a61c5.env
# stdout-file          /tmp/opam-5568/log/log-5568-5a61c5.out
# stderr-file          /tmp/opam-5568/log/log-5568-5a61c5.err
### stderr ###
# install: cannot create regular file 'C:/Users/bryph/.esy/3_/s/opam__slash__ocamlbuild-0.12.0-5e55ac68/bin/ocamlbuild.byte': File exists
esy-bash: process exited with code 0

Minimal package.json repro:

{
  "name": "test-menhir",
  "version": "0.0.0",
  "esy": {
    "build": [
      "true"
    ]
  },
  "dependencies": {
    "@opam/menhir": "*"
  },
  "devDependencies": {
    "ocaml": "esy-ocaml/ocaml#6aacc05"
  }
}

[bryphe]

One other thing worth looking at is that on the forked-opam repository for Windows, there were Windows specific patches introduced: https://github.com/fdopen/opam-repository-mingw/tree/master/packages/menhir/menhir.20180703, ie: fdopen/opam-repository-mingw@54c3e01

[andreypopp]

@bryphe ocamlbuild -where returning paths with /s/ means fastreplacestring does't work — it should rewrite all /s/ path with /i/.

[andreypopp]

It looks like the old opam to esy conversion cannot deal with ocamlbuild from opam-repository-mingw at all.

[bryphe]

@bryphe ocamlbuild -where returning paths with /s/ means fastreplacestring does't work — it should rewrite all /s/ path with /i/.

Do you have any tips for debugging this, @andreypopp ? Do we cache these paths? I started adding some logging here https://github.com/esy/esy/tree/bryphe/investigation/debug-logging - but figured you might know some good places to look, too.

It looks like the old opam to esy conversion cannot deal with ocamlbuild from opam-repository-mingw at all.

Hmm, what did you mean by this? Is this something we will need to address with esy-install?

[bryphe]

I logged out the fastreplacestring calls and see it run for each of the installed build outputs for ocamlbuild, like:

rewritePrefixInFile - origPrefix: C:\Users\bryph\.esy\3_\s\opam__slash__ocamlbuild-0.12.0-5757e6fe destPrefix: C:\Users\bryph\.esy\3_\i\opam__slash__ocamlbuild-0.12.0-5757e6fe path: C:\Users\bryph\.esy\3_\s\opam__slash__ocamlbuild-0.12.0-5757e6fe\_esy\storePrefix
...
rewritePrefixInFile - origPrefix: C:\Users\bryph\.esy\3_\s\opam__slash__ocamlbuild-0.12.0-5757e6fe destPrefix: C:\Users\bryph\.esy\3_\i\opam__slash__ocamlbuild-0.12.0-5757e6fe path: C:\Users\bryph\.esy\3_\s\opam__slash__ocamlbuild-0.12.0-5757e6fe\bin\ocamlbuild.native.exe

But, when I search the files, I see stuff like:

                    �'      C:/Users/bryph/.esy/3_/s/opam__slash__ocamlbuild-0.12.0-5757e6fe/bin   ♥�'      C:/Users
/bryph/.esy/3_/s/opam__slash__ocamlbuild-0.12.0-5757e6fe/lib   ♥�▼      C:/Users/bryph/.esy/3_/s/ocaml-4.6.4-d402855c/ lib      ♠�'      C:/Users/bryph/.esy/3_/s/opam__slash__ocamlbuild-0.12.0-5757e6fe/lib   ♥�      a      ♠�      o

I believe there's another case of slash mismatches - we're replacing with the windows style \ slash, but the paths are actually forward slashes.

I'm not sure we have any guarantee on the form of the slashes here, so our best bet might be to run fastreplacestring twice - once with forward slashes (replacing with forward slashes), and once with back slashes (replacing with backslashes) - that should atch either case, as long as there aren't mixed slashes in the binaries..