Getting neanderthal to run in the Windows Subsystem for Linux 2

[BorisVSchmid]

This is a little bit a duplicate from the issues #119 and #127, but as I don't seem to run into any trouble getting neanderthal to run on the WSL2, with support of CUDA, it seems worth posting it as a separate issue with a clear title.

Using sublime text as an IDA and the Terminus / Clojure Sublimed packages, I can start a WSL as a terminal, and connect Clojure Sublimed to a clj -M:repl nrepl I start in that terminal. After that, everything is smooth sailing.

These are the steps I took to get neanderthal to run on the WSL. Nothing exotic.

Neanderthal on the WSL2

install WSL & Ubuntu

1. Install WSL2 in windows.
2. run wsl.exe --update in a powershell
3. Install Ubuntu 22.04.1 LTS from the microsoft store

Go to /users/ and make a file called .wslconfig, and add (and adapt to your system) the following text into the file

[wsl2]
memory=120GB # Limits VM memory in WSL 2 to 128 GB

Else your WSL only gets access to 6GB of memory or so.

install MKL and clojure in Ubuntu

1. sudo apt-get update

2. sudo apt-get install intel-mkl
   choose the default in the install screen (libmkl_rt is not the default alternative to BLAS/LAPACK)

3. sudo apt-get install clojure
   As installing clojure in this way doesn't get you the clojure tools ('clj'), I am also installing clojure through homebrew, just minus the 1 million dependencies.

4. sudo apt-get install build-essential

5. execute /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

6. execute eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"

7. nano ~/.bashrc and paste eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)" as the top line

8. run brew install --ignore-dependencies clojure
   brew will install clojure for you with the clojure tools (clj). And the ignore dependencies saves you from a ton of weird stuff being installed

9. sudo apt-get install leiningen

Testing in hello_world of neanderthal

1. git clone https://github.com/uncomplicate/neanderthal.git
2. navigate to examples/hello_world.
3. edit the project.clj so that [org.bytedeco/mkl-platform-redist "2020.3-1.5.4"] and "-XX:+UseLargePages" are disabled / removed.
4. do a lein repl in the examples/hello_world directory (in WSL!, not in powershell)
5. (load-file "src/hello_world/native.clj")
   and see that neanderthal native.clj works.

Getting the GPU to work.

GPU won't work yet in the above setting (native will). There is a large block of text available here explaining some of the below steps:
https://docs.nvidia.com/cuda/wsl-user-guide/index.html

From the article, and within WSL

1. sudo apt-key del 7fa2af80
   The article uses CUDA 12-0 as an example. Neanderthal uses 11.7.0
2. wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
3. sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
4. wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda-repo-ubuntu2204-11-7-local_11.7.0-515.43.04-1_amd64.deb
5. sudo dpkg -i cuda-repo-ubuntu2204-11-7-local_11.7.0-515.43.04-1_amd64.deb
6. sudo cp /var/cuda-repo-ubuntu2204-11-7-local/cuda-*-keyring.gpg /usr/share/keyrings/
7. sudo apt-get update

Note, that if you look up the syntax to install other packages than 11.7.0 on https://developer.nvidia.com/cuda-downloads?, it will almost give you the correct syntax, just not the last line - the article on how to install cude and wsl specifically warns against that, but it is easy to miss.

The installation instructions for the CUDA Toolkit can be found in the CUDA Toolkit download page for each installer. But DO NOT choose the “cuda”, “cuda-12-0”, or “cuda-drivers”
meta-packages under WSL 2 as these packages will result in an attempt to install the Linux NVIDIA driver under WSL 2. Install the cuda-toolkit-12-x metapackage only.

So in our case , make sure to do:
8. sudo apt-get install cuda-toolkit-11-7
That is enough to get hello_world running on the wsl (both native and cuda.)
9. navigate to examples/hello_world.
10. do a lein repl in the examples/hello_world directory (in WSL!, not in powershell)
11. (load-file "src/hello_world/native.clj")
12. (load-file "src/hello_world/cuda.clj")
and see that neanderthal native.clj and cuda.clj works :-)

[blueberry]

Thanks!
Do I understand well that this works, and the issue can be closed?

[BorisVSchmid]

Issue can be closed. Neanderthal on WSL seems to work for native and cuda, but not opencl. And that seems to be what to expect from WSL2 at the moment (microsoft/WSL#6951).

Matches what I see when I run lein test on neanderthal's repository, there are three test files that fail (clblast_test.clj, clblast_cpu_test.clj, and tutorial_opencl_test.clj.fail).

If I disable those tests, lein test completes. One exception, that might be worth looking in: I have this error popping up a dozen of times:

FAIL div at (mkl_test.clj:130)
Actual result did not agree with the checking function.
Actual result:
clojure.lang.ExceptionInfo: LAPACK error. {:bad-argument 5, :error-code -5}
  uncomplicate.neanderthal.internal.host.mkl.DoubleTREngine.copy(mkl.clj:1511)
  uncomplicate.neanderthal.internal.host.buffer_block.RealUploMatrix.host(buffer_block.clj:1243)
  uncomplicate.neanderthal.core$transfer.invokeStatic(core.clj:135)
  uncomplicate.neanderthal.math_test$diff_square_2.invokeStatic(math_test.clj:106)
  uncomplicate.neanderthal.math_test$diff_square_2.invokeStatic(math_test.clj:102)
  uncomplicate.neanderthal.math_test$diff_square_2.invoke(math_test.clj:102)
  uncomplicate.neanderthal.math_test$test_math$fn__55902$fn__55903$fn__55904.invoke(math_test.clj:130)
  uncomplicate.neanderthal.math_test$test_math$fn__55902$fn__55903.invoke(math_test.clj:130)
  uncomplicate.neanderthal.math_test$test_math$fn__55902.invoke(math_test.clj:130)
  uncomplicate.neanderthal.math_test$test_math.invokeStatic(math_test.clj:127)
  uncomplicate.neanderthal.math_test$test_all_host.invokeStatic(math_test.clj:233)
  uncomplicate.neanderthal.mkl_test$eval56456.invokeStatic(mkl_test.clj:103)
  uncomplicate.neanderthal.mkl_test$eval56456.invoke(mkl_test.clj:103)
Checking function: (zero)

For those without a linux environment, a plus from running neanderthal in WSL is that it is easy to install a system-wide MKL, and that [org.bytedeco/mkl-platform-redist "2022.0-1.5.7"] works. Both of those currently are non-trivial to get to work when staying in windows.