Support OCaml Multicore

[pklehre]

Describe the bug

When compiling code using the OCaml Multicore compiler, I get the following error message:

dune build jsoo js_of_ocaml .js/stdlib/std_exit.cmo.js (exit 1) (cd _build/default/.js/stdlib && /home/pkl/.opam/4.10.0+multicore+no-effect-syntax/bin/js_of_ocaml --pretty --source-map-inline -o std_exit.cmo.js /home/pkl/.opam/4.10.0+multicore+no-effect-syntax/lib/ocaml/std_exit.cmo) /home/pkl/.opam/4.10.0+multicore+no-effect-syntax/bin/js_of_ocaml: You found a bug. Please report it at https://github.com/ocsigen/js_of_ocaml/issues : Error: File "compiler/lib/parse_bytecode.ml", line 693, characters 2-8: Assertion failed make: *** [Makefile:4: all] Error 1

Expected behavior

Code compiles without problems.

Versions

4.10.0+multicore+no-effect-syntax opam switch

[hhugo]

Js_of_ocaml doesn't support ocaml multicore

• ocaml multicore uses a different set of bytecode instructions, it also shuffles instruction codes

-  GETFIELD0, GETFIELD1, GETFIELD2, GETFIELD3, GETFIELD, GETFLOATFIELD,
+  GETFIELD0, GETFIELD1, GETFIELD2, GETFIELD3, GETFIELD,
+  GETMUTABLEFIELD0, GETMUTABLEFIELD1, GETMUTABLEFIELD2, GETMUTABLEFIELD3,
+  GETMUTABLEFIELD, GETFLOATFIELD,

• It's unclear how multicore would work in javascript

[hhugo]

IIRC @Armael and @kayceesrk worked on supporting multicore in jsoo. I dont know what they ended up with

[pklehre]

I thought it might be possible to compile plain OCaml code which does not use any of the multicore features to JS. However given that the bytecode instructions have changed, it is a more substantial task than I expected. I believe multicore will be part of OCaml 5, so this issue might become relevant again soon.

[hhugo]

I've a branch that propagate changes for some of the instructions. But it fail to compile the stdlib due to instruction related to effects.
https://github.com/ocsigen/js_of_ocaml/tree/multicore-410 837566a

[hhugo]

I've opened an issue regarding the instructions/opcodes reuse. ocaml-multicore/ocaml-multicore#475

[Lupus]

It's unclear how multicore would work in javascript

There is relevant prior work done on supporting Golang's goroutines (which have M:N mapping to real threads with smart runtime scheduler on the native Golang compilation target) in Gopherjs project [1]. What they seem to be doing is unwinding stack when blocking call is encountered and reconstruct it later when it is to be resumed. It works well in practice - when playing around with it I was just copy pasting some Go code using threads and watched it somehow work in JS, that felt like magic. May be something similar could be done for js_of_ocaml to support multicore and effects?

[1] https://github.com/gopherjs/gopherjs#goroutines

[patricoferris]

From an effects perspective I thought I would just flag that I managed to port some of https://github.com/Armael/js_of_ocaml to the 4.12+domains+effects and js_of_ocaml.3.9.0 which is here: https://github.com/patricoferris/js_of_ocaml/tree/4.12.0+domains%2Beffects

From the root of the project (with a 4.12+domains+effects switch) you should be able to do some simple effects programs:

$ ocamlc -version
4.12+domains+effects
$ cat hello.ml
effect Print : string -> unit

let print s = perform (Print s)

let main () = 
  print "world";
  print "hello"

let () = 
  match main () with 
   | () -> ()
   | effect (Print s) k -> continue k (); print_endline s
$ ocamlc -o hello.bc hello.ml
$ dune exec -- js_of_ocaml -o hello.js hello.bc
$ node hello.js
hello
world

I tried a lot to get this to work with examples using js_of_ocaml libraries but the CPS transform wasn't happy (or I ported it incorrectly). I think this might have something to do with Printexc and how the entry/exit nodes of exceptions are impacted by this... I'm not really a jsoo or multicore/effects expert but for visibility I thought I would share this :))

[Armael]

I think the reason is that the CPS translation I implemented back then is not correct! :D
So yeah, I probably wouldn't advocate to use it, unfortunately.

[hhugo]

I think the reason is that the CPS translation I implemented back then is not correct! :D
So yeah, I probably wouldn't advocate to use it, unfortunately.

Do you have more information about the incorrect implementation worth keeping track of ?

[patricoferris]

Just to clarify what I meant by "...something to do with Printexc...", when compiling programs that tried to use effects and say used js_of_ocaml as a library, the assertions in the CPS transform about the number of things in the exit nodes would not hold (i.e. at places like https://github.com/patricoferris/js_of_ocaml/blob/4.12.0%2Bdomains%2Beffects/compiler/lib/effects.ml#L712). I could slowly get past some of these by removing code like the printexc registering printers or printing backtraces but I wasn't really understanding the issue just looking to see what parts of the code were causing issues and removing them to make some progress.

[hhugo]

I've updated my branch for multicore.4.12+domains -> https://github.com/ocsigen/js_of_ocaml/tree/multicore-412

[hhugo]

We now have #1265

[hhugo]

wip in https://github.com/ocsigen/js_of_ocaml/tree/effects

[hhugo]

#1340 has been merged.