[Windows] Add wslMode to settings

[jchavarri]

It seems the WSL mode in Windows is very powerful, as it allows to keep all tools in their original OSX/Linux versions and plug them in into the native Windows IDEs.

Adding a setting to enable wslMode would allow Windows users to quickly configure the extension without the need to install bridges or any other hack, and it would allow us to tweak the data that goes from the server to the VSCode client, to make sure everything works properly.

A couple of use cases for this setting:

• Prepending bash -ic to paths: so instead of having to install ocaml-on-windows, just adding this setting on VSCode would work
• Adapting file paths: because bsb or merlin would be running on Linux mode, we would need to adapt from file:///mnt/c/project/myFile.re to C:\project\myFile.re so the diagnostics work properly (underline text on file, double click to open, etc)

There are probably more cases.

@freebroccolo @chenglou @superherointj

[superherointj]

Having 'wslMode' set should change the default value for undefined path variables only and not affect variables holding custom values.

"reason.path.bsb": "bash -ic bsb",
"reason.path.ocamlfind": "bash -ic ocamlfind",
"reason.path.ocamlmerlin": "bash -ic ocamlmerlin",
"reason.path.opam": "bash -ic opam",
"reason.path.rebuild": "bash -ic rebuild",
"reason.path.refmt": "bash -ic refmt",
"reason.path.refmterr": "bash -ic refmterr",
"reason.path.rtop": "bash -ic rtop"

[superherointj]

I made a function to convert unix path to windows format, like:
From: "file:///mnt/d/dev/meta/xxx/webapp/server/phoenix/assets/src/Cities.re";
To: "D:\dev\meta\xxx\webapp\server\phoenix\assets\src\Cities.re";

let unixPathToWin = sPath => {
    let idxDrive = sPath.search("/mnt/");
    let r = "";
    if (idxDrive > 1) {
        let driveLetter = sPath[idxDrive + 5];
        let folders = sPath.substring(idxDrive + 5 + 2)
        let foldersArr = folders.split('/');
        let foldersInWin = foldersArr.join('\\');
        r = driveLetter.toUpperCase() + ":\\" + foldersInWin;
    }
    return r;
}

In case it is invalid it will return empty.
Feel free to improve on it if necessary. Or even discard it. I'm putting it here as a convenience.

[ghost]

Getting the extension to work with tooling installed via WSL is something I've wanted to have for awhile now so I'm open to doing something like this but I'd like to understand the details a little better first. Mainly I just want to be careful about how much complexity and maintenance burden we incur from platform specific support. But I think we can probably make this work.

Adding a setting to enable wslMode would allow Windows users to quickly configure the extension without the need to install bridges or any other hack, and it would allow us to tweak the data that goes from the server to the VSCode client, to make sure everything works properly.

What kind of modifications would need to be made when this setting is enabled? I briefly tried to get OCaml tooling through WSL to work with this extension several months ago but ran into several issues which I don't remember anymore. Is it really as simple as just running the commands through the windows bash now?

Prepending bash -ic to paths: so instead of having to install ocaml-on-windows, just adding this setting on VSCode would work

Some care may need to be taken when running merlin this way. If I recall correctly, there can be issues with IO buffering in some cases that will cause communication between merlin and ocaml-language-server to fail, but not immediately.

Adapting file paths: because bsb or merlin would be running on Linux mode, we would need to adapt from file:///mnt/c/project/myFile.re to C:\project\myFile.re so the diagnostics work properly (underline text on file, double click to open, etc)

The vscode-uri package provides functionality to translate between URIs and OS paths so that shouldn't be a problem. We're already using it in some places in the server.

[superherointj]

There was a breakthrough in thinking by realizing BuckleScript couldn't be used on native Windows because it was generating invalid Merlin files and that was causing problems with Merlin.

• Automating adding 'bash -ci' prefix is a convenience change.

• Right now, the most important thing to happen to get extension working w/ Windows & WSL is to fix Unix/Windows path translation issue .

#TheOneWindowsThing

[isutton]

Take a look in the vscode-node-debug extension that makes use of it.

[ghost]

Thanks. Looks like most of the relevant code from vscode-node-debug for WSL support is here.

[jchavarri]

What kind of modifications would need to be made when this setting is enabled?

Speaking only for the bsb case: we would need to translate the URIs from one system to the other.

Prepending bash -ic is a nice to have that avoid users having to somehow create runners that include this.

Besides these 2, maybe there are other requirements, but I'm not aware of anything else atm.

Is it really as simple as just running the commands through the windows bash now?

In the bsb case, it seems so.

Some care may need to be taken when running merlin this way. If I recall correctly, there can be issues with IO buffering in some cases that will cause communication between merlin and ocaml-language-server to fail, but not immediately.

Good to know. We could start with bsb, which usage is simpler technically (no process remains live and is queried over time), and seems to be working already, and then move to merlin.

Looks like most of the relevant code from vscode-node-debug for WSL support is here.

woah this is exactly what we'd need. It even has a config.useWSL param. And the function that translates between systems (i saw vscode-uri has some utils but nothing that removes the /mnt part) (Edit: windowsPathToWSLPath goes the other way around, we'd need something that translates uris from WSL to Windows)

---

@freebroccolo if you're ok with this, I can set up a PoC with the help of @superherointj (I don't have a Win machine at hand) that adds this useWSL flag and includes the uris translation, so we can try to make it work with bsb. wdyt?

[superherointj]

The path conversion that must happen is from unix (WSL) to windows (VsCode). (This solves reverse problem). As a test, the function I provided works, and then once it works and actually fixes problems, optimize with an official solution to problem.

[jchavarri]

One thing that @superherointj and I realized today is that merlin running on WSL mode handles properly the file URIs. So that's one thing less to care about to support WSL mode, I guess.

@freebroccolo Do you know the reason why in this case it's working without any special handling? Are the file paths resolved downstream? (instead of merlin providing them)

[superherointj]

There is still the problem of correcting paths for BSB and for opening files on VSCode.

Unable to open 'City.re': File not found (file:///mnt/d/dev/meta/provedor/webapp/server/phoenix/assets/src/City.re).

Reminder: Path translation must be done from Unix to Windows.

[yminsky]

Is there any news on this feature? I'm thinking about setting up OCaml on Windows, and wondering if this is a viable option (or if I should just install VSCode within WSL, or just try a VM...)

[superherointj]

This feature isn't required, VSCode extension works with some minor glitches in Windows. (With this issue open still.)

Merlin is what doesn't work properly in Windows native environment. But when Merlin is executed in WSL environment it works. But VSCode is executed in Windows environment (not WSL). Merlin & Ocaml Toolchain have to be executed in WSL. Such way Merlin file has unix path (not Windows path), so it matches.

OCaml's Windows native support is really VERY VERY painful. While the Ocaml compiler works in Windows native. Packages don't. Unless some Unix environment is used for building packages. You are probably better off going with WSL (if you can)... things there just work.

There is a guide for Ocaml/Reason in WSL here:
reasonml/reasonml.github.io#195

[ghost]

Is there any news on this feature? I'm thinking about setting up OCaml on Windows, and wondering if this is a viable option (or if I should just install VSCode within WSL, or just try a VM...)

@yminsky Proper WSL support is the main feature I'm currently working on in the development branch of the language server. It has taken longer than expected due to some fairly significant but necessary refactoring. It's coming along but not ready yet and probably won't be available until next week at the earliest.

If you try to use OCaml on WSL with VS Code installed natively under Windows (not WSL, which currently isn't possible afaik) a lot of the Merlin features won't work properly due to the paths reported by Merlin being in unix style and the paths the extension is working with (on Windows) being win32 style.

This is a high priority feature though so it is actively being worked on.

What I am currently using on Windows is an ubuntu VM running under hyper-V. Then I set up an Xpra server in the VM to launch the Ubuntu version of VS Code on connect. With everything properly configured I get a mostly seamless looking UI (losslessly encoded with LZ4) and the latency is very good at ~5-10ms.

It's a little bit of a hassle to set that up but not so bad if you know the steps. I would be willing to write up a little guide if people think it would be useful.

[superherointj]

@freebroccolo Speculation: I think Merlin 2.5.x works fine in Windows native environment. I did some tests with a Node.JS app attached to Merlin and it answered commands correctly. See here and here. (But I didn't test anything too fancy) So I was wondering if there could be a path problem in ocaml-language-server or extension. Been thinking of this theory, VSCode requires double slash, and Merlin file is generated with only a single slash. What do you think? You are waaay more experienced than me on this. Maybe, just maybe, there is something to it?

[ghost]

It's possible that Merlin 2.5.x works under Windows natively. I don't think I've ever tried it.

The server itself has mostly been developed against Merlin 3.0.x.

To be honest, even if it is possible to get an older version of Merlin to work under Windows native, I'm not very keen on recommending people try to use it with the extension. If Windows native someday becomes more of a 1st tier target for the language and surrounding tooling then it might be worth looking at but for now it just seems like too much trouble.

[yminsky]

Out of curiosity, does launching code from within wsi (which is easy enough) help the problem in any way?

[ghost]

Out of curiosity, does launching code from within wsi (which is easy enough) help the problem in any way?

I don't think so. Theoretically the VS code developers could add some logic to detect that scenario and change the way paths are handled in the editor but as far as I know there are no plans to do that.

[ghost]

@yminsky It looks like it is actually possible now to run the Linux version of vscode from within WSL. See here and scroll up to the first comment for instructions. Only thing I would add is that MobaXterm seems to be the best X11 server on Windows in my experience so I would try that over the other one listed.

If you install both OCaml and the Linux version of vscode in WSL this way, it should work fine. Still perhaps not ideal since you will need to have an X11 server installed and configured in Windows, but it should probably be better than a VM solution. (I haven't tried so I'm not certain about the performance; the hyper-v approach performs well though).

EDIT: You could probably use Xpra for this instead of X11 and it'd likely be more responsive.

[yminsky]

I got vscode set up and working on an ubuntu VM for now. I think I'll just wait until there's support for using a WSL-based OCaml via a WIndows-native vscode.

Thanks!

[robinei]

Why was this closed without comment? Will this feature not be supported?

[ghost]

The plan is still to support WSL. I started working on this awhile ago. In order to support WSL properly, the server action queue needs to be modified to support pre/post-processing of merlin commands. This needs to be handled at the level of the action queue otherwise every part of the server that deals with paths and command invocations will need to include special logic for the WSL case. I implemented a prototype for this but it's difficult to deal with nicely due to limitations of typescript (notably a lack of support for existential types).

Given the complexity of the backend rewrite (including some subtle bugs I haven't had time to fix) and the fact that the plan was to do another native rewrite anyway almost as soon as it was completed, I've decided to just focus on the native rewrite and put the WSL support on hold. It's not ideal, but it will be a better use of my time and lead to a much more maintainable implementation in the long run.

[jchavarri]

I've decided to just focus on the native rewrite

@freebroccolo Just curious, what are you referring to with "native rewrite"? Is it a reimplementation of the lang server in OCaml? Or something else?

[ghost]

@jchavarri yeah, in OCaml.

I'll just add btw that if someone wants to create a PR for the current implementation that gets WSL working in a robust way, I'm happy to merge that as a temporary measure. I'm just spread a bit thin with other projects at the moment and it's kind of hard to justify putting a lot more energy into a rewrite of the current server core in typescript if the intention is to move on to something better soon after anyway.