Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support bucklescript #140

Open
jaredly opened this issue Jun 11, 2017 · 46 comments

Comments

@jaredly
Copy link

commented Jun 11, 2017

I tried just aliasing ocamlc to bsc.exe on my PATH, but got an error about cmdliner looking for a config.h file.
It would be awesome to be able to use jbuilder with my web projects!

@diml

This comment has been minimized.

Copy link
Member

commented Jun 12, 2017

I thought bucklescript already came with a recommended ninja-based build system? /cc @bobzhang

@jaredly

This comment has been minimized.

Copy link
Author

commented Jun 12, 2017

Bucklescript has a build system, but it's much less flexible than jbuilder. If it's not too much work on your part, I'd really prefer to be able to use jbuilder to target web + native.

@diml

This comment has been minimized.

Copy link
Member

commented Jun 12, 2017

I see. I don't have the time to look at bucklescript myself, and can't promise either that rules for it would be integrated into jbuilder, as it is additional maintenance work.

What I would recommend is start by forking jbuilder and change it to use bucklescript rather than the OCaml compiler, then we could see what can be done from there. For instance what could be abstracted in jbuilder to either make it possible to use jbuilder to build bucklescript code, or make it easy to produce a specialized jbuilder for buckescript, etc...

@ELLIOTTCABLE

This comment has been minimized.

Copy link

commented Jul 14, 2018

👍 on this. bsb, their build-system, is very young and yet horribly archaic in some ways.

If nothing else, I'm desperate for (inline_tests); but way more usefully, I want to build hybrid projects which only have small Node-facing and opt-facing modules, most of which are platform-agnostic, with only one build-system — not a Frankensteinian mish-mash of multiple build-systems cobbled together by hand for each project!

I have to admit, I'm a little confused: The Introduction says,

Dune is a build system for OCaml and Reason. It is not intended as a completely generic build system that is able to build any given project in any language.

Was the addition of Reason there a mistake? I can find no other mention of Reason or Bucklescript in the docs; only js_of_ocaml

Anyway. I understand concerns about the maintenance burden, but I strongly feel like the OCaml/Reason community needs some collaboration / cohesiveness. (I mean, there's oasis and dune of course, whatever's going on with bsb itself, esy, bsb-native …) I really think giving such a first-class build tool first-class support for BS could really bridge that gap!

@Khady

This comment has been minimized.

Copy link
Contributor

commented Jul 14, 2018

Reason the syntax is supported. Nothing special to do, it just works. Bucklescript is not reason. Bucklescript is not supported.

@diml

This comment has been minimized.

Copy link
Member

commented Jul 16, 2018

Personally, I'm not against supporting bucklescript in dune, but I don't want it to be some kind of competition between dune and bsb, because that would be a big waste of everybody's time.

@ELLIOTTCABLE

This comment has been minimized.

Copy link

commented Jul 16, 2018

Seems like a bit off-topic, but I'd posit that a healthy balance of collaboration and competition between open-source projects for mindshare could only benefit the end-user ¯\_(ツ)_/¯

Anyway: coming back to this, it looks like I mis-read the response to the OP: you're perhaps not willing to support a BuckleScript extension itself, but you're interested in seeing what is required over here to support such an extension, and implementing/maintaining the subsequent extension-or-whatever-system?

Because forking the entire project, and them maintaining an entire fork, porting forward upstream changes, is hella untenable obviously — but a P.O.C. to motivate otherwise-weclome upstream changes to support a small, maintainable extension, is much more palatable! Seems in-line with the Dune 1.0 spirit, too. Do I understand your suggestion correctly?

@diml

This comment has been minimized.

Copy link
Member

commented Jul 16, 2018

Yes, basically at this point none of us has any idea what is required to support bucklescript, so we can't really make promises. If someones spends some time and produce a working prototype, even if it is not a generic solution, then we can look at the diff and assess the complexity to decide whether it is something we are happy to support or not in the long run or not.

In any case, just the initial support is not enought: if we support bucklescript, we need at least one person who uses bucklescript regularly in the dune team, so that we can assign bucklescript issues to this person. So someone has to be happy to take this role.

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Jul 17, 2018

One issue that I'd like to see resolved in bucklescript itself before supporting it is the 4.02.3 issue. If supporting bucklescript means that we must support 4.02.3 forever, then I'm very much against the idea. In my opinion, we should wait and see that bucklescript is able to keep up with the compiler before spending time on this.

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Sep 21, 2018

@diml can we assign this to @zploskey somehow?

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Sep 21, 2018

I plan to start working on this over the weekend with a test project.

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Sep 21, 2018

Don't hesitate to post your progress here :)

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Sep 22, 2018

I'll be trying things out here: https://github.com/zploskey/hello-esy-dune-bsb

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Sep 24, 2018

The build.ninja file generated by BuckleScript for a simple "hello world" program may be informative. I haven't used ninja directly before but it reads somewhat similarly to a makefile.

bs_package_flags = -bs-package-name hello-esy-dune-bsb
src_root_dir = /home/zach/src/hello-esy-dune-bsb
bsc = /home/zach/src/hello-esy-dune-bsb/node_modules/bs-platform/lib/bsc.exe
bsdep = /home/zach/src/hello-esy-dune-bsb/node_modules/bs-platform/lib/bsb_helper.exe
warnings = -w -30-40+6+7+27+32..39+44+45+101+a -warn-error +a
bsc_flags = -bs-suffix -nostdlib -I '/home/zach/src/hello-esy-dune-bsb/node_modules/bs-platform/lib/ocaml' -bs-g -g -color always
ppx_flags = 
bs_package_includes = 
bs_package_dev_includes = 
namespace = 
bsb_dir_group = 0
bsc_lib_includes = -I bin
rule build_ast_and_module_sets
  command = ${bsc}  ${pp_flags} ${ppx_flags} ${warnings} ${bsc_flags} -c -o ${out} -bs-syntax-only -bs-binary-ast ${in}
  description = Building ${out}
build bin/hello.mlast : build_ast_and_module_sets $src_root_dir/bin/hello.ml
rule build_deps
  command = ${bsdep} ${namespace} -g ${bsb_dir_group} -MD ${in}
  description = Building ${out}
build bin/hello.mlast.d : build_deps bin/hello.mlast
rule build_cmj_cmi
  command = ${bsc} ${bs_package_flags} -bs-assume-no-mli -bs-no-builtin-ppx-ml -bs-no-implicit-include ${bs_package_includes} ${bsc_lib_includes} ${bsc_extra_includes} ${warnings} ${bsc_flags} -o ${out} -c  ${in} $postbuild
  depfile = ${in}.d
  description = Building ${out}
build bin/hello.cmj |  $src_root_dir/bin/hello.bs.js bin/hello.cmi : build_cmj_cmi bin/hello.mlast
  bs_package_flags = $bs_package_flags  -bs-package-output commonjs:bin
build build.ninja : phony  ||  bin/hello.mlast.d

For our purposes we will likely require using bsc in place of ocamlc and bsb-helper.exe (aliased to bsdep here) in place of ocamldep, or something like that. I'll be looking more closely at what differences there may be.

Thanks to @jaredly for explaining what is going on with the .mlast files. As we thought, these are the binary AST but with some extra data at the beginning expressing the number of dependency files to follow, then those file paths. The code in bsb that reads the AST is here:

https://github.com/BuckleScript/bucklescript/blob/3cf3133a4978df05e0b5fb351a1d2d18dce16c32/jscomp/depends/binary_ast.ml#L33-L43

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Oct 6, 2018

I'll have some time to work on this over the weekend today and tomorrow. If anyone more familiar with how Dune is designed could sketch out how you might go about adding the a different back-end you might save me some time. I'm still not clear whether dune will need to learn how to deal with bsconfig.json files (possibly by calling bsb as a library) or if we want to add equivalent configuration to dune files as what BuckleScript adds. The latter seems nicer but a harder transition for people migrating from bsb configs.

We'll probably need to read bsconfigs no matter what so that bsb-only dependencies are supported.

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Oct 6, 2018

I'm still not clear whether dune will need to learn how to deal with bsconfig.json files (possibly by calling bsb as a library)

I think having dune deal with bsconfig.json would be great. Building existing bucklescript projects with dune will immediately open up a large test suite for us, and dramatically reduce the entry barrier for this feature.

However, depending on bsb as a library doesn't seem so useful however. I'd expect that we'd convert the specification to something that is very similar to dune's stanzas, so it doesn't seem like reusing bsb would work.

or if we want to add equivalent configuration to dune files as what BuckleScript adds

If dune supports bucklescript, I'd expect it to be able to build existing libraries and executables using bucklescript. Of course, it's likely that we'll need a field for bucklescript specific configuration.

@jaredly

This comment has been minimized.

Copy link
Author

commented Oct 7, 2018

Hrmmm bsb as currently constituted has some differences from dune that will probably just overcomplicate things -- and honestly the reason I'd want dune w/ bucklescript is to get dune's package semantics instead of bsb's.
So let's get it working with bsc directly, and then maybe add on bsconfig.json support later.

@jaredly

This comment has been minimized.

Copy link
Author

commented Oct 7, 2018

Like, what would even be the benefit of having dune call bsb? bsb is a build tool, dune is a build tool... it would be like having dune call jenga. I don't think it makes sense.

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Oct 7, 2018

I don't think we should be calling bsb, but we probably need to read bsconfigs eventually. We just need to support the same feature set when calling bsc and be able to build dependencies that use bsb and not dune as a build system. I created a bucklescript branch here and started working on adding a way to set bsc flags in a dune file. Input is definitely appreciated.

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Oct 7, 2018

dune calling out bsb is definitely ruled out. This would be impractical and wrong on many levels. However, I do think that dune would need a full understanding of bsconfig.json files.

I created a bucklescript branch here and started working on adding a way to set bsc flags in a dune file

Just curious, do we have a good understanding of the various bucklescript flags yet? Understanding these will be useful in making sure that there are no nasty surprises down the road.

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Oct 7, 2018

Here's the help output from bsc from latest BS (4.0.6):

$ bsc --help
Usage: bsc <options> <files>
Options are:
  -bs-super-errors  Better error message combined with other tools 
  -bs-re-out  Print compiler output in Reason syntax
  -bs-suffix  Set suffix to .bs.js
  -bs-no-implicit-include  Don't include current dir implicitly
  -bs-assume-has-mli  (internal) Assume mli always exist 
  -bs-assume-no-mli  (internal) Don't lookup whether mli exist or not
  -bs-D  Define conditional variable e.g, -D DEBUG=true
  -bs-list-conditionals  List existing conditional variables
  -bs-binary-ast  Generate binary .mli_ast and ml_ast
  -bs-syntax-only  only check syntax
  -bs-no-bin-annot  disable binary annotations (by default on)
  -bs-eval  (experimental) Set the string to be evaluated, note this flag will be conflicted with -bs-main
  -bs-g  debug mode
  -bs-sort-imports  Sort the imports by lexical order so the output will be more stable (default false)
  -bs-no-sort-imports  No sort (see -bs-sort-imports)
  -bs-package-name  set package name, useful when you want to produce npm packages
  -bs-package-map  set package map, not only set package name but also use it as a namespace
  -bs-no-version-header  Don't print version header
  -bs-package-output  set npm-output-path: [opt_module]:path, for example: 'lib/cjs', 'amdjs:lib/amdjs', 'es6:lib/es6' 
  -bs-no-warn-unimplemented-external  disable warnings on unimplmented c externals
  -bs-no-builtin-ppx-ml disable built-in ppx for ml files (internal use)
  -bs-no-builtin-ppx-mli disable built-in ppx for mli files (internal use)
  -bs-cross-module-opt enable cross module inlining(experimental), default(false)
  -bs-diagnose  More verbose output
  -bs-no-check-div-by-zero  unsafe mode, don't check div by zero and mod by zero
  -bs-noassertfalse  no code for assert false
  -bs-main  set the Main entry module in script mode, for example -bs-main Main
  -bs-I  add source dir search path in script mode
  -bs-files  Provide batch of files, the compiler will sort it before compiling
  -impl <file>  Compile <file> as a .ml file
  -intf <file>  Compile <file> as a .mli file
  - <file>  Treat <file> as a file name (even if it starts with `-')
  -absname  Show absolute filenames in error messages
  -annot  Save information in <filename>.annot
  -bin-annot  Save typedtree in <filename>.cmt
  -c  Compile only (do not link)
  -config  Print configuration values and exit
  -g  Save debugging information
  -i  Print inferred interface
  -I <dir>  Add <dir> to the list of include directories
  -color {auto|always|never}  Enable or disable colors in compiler messages
    The following settings are supported:
      auto    use heuristics to enable colors only if supported
      always  enable colors
      never   disable colors
    The default setting is 'auto', and the current heuristic
    checks that the TERM environment variable exists and is
    not empty or "dumb", and that isatty(stderr) holds.
  -intf-suffix <string>  Suffix for interface files (default: .mli)
  -keep-docs  Keep documentation strings in .cmi files
  -keep-locs  Keep locations in .cmi files
  -labels  Use commuting label mode
  -no-alias-deps  Do not record dependencies for module aliases
  -no-app-funct  Deactivate applicative functors
  -noassert  Do not compile assertion checks
  -nolabels  Ignore non-optional labels in types
  -nostdlib  Do not add default directory to the list of include directories
  -o <file>  Set output file name to <file>
  -open <module>  Opens the module <module> before typing
  -pp <command>  Pipe sources through preprocessor <command>
  -ppx <command>  Pipe abstract syntax trees through preprocessor <command>
  -principal  Check principality of type inference
  -rectypes  Allow arbitrary recursive types
  -safe-string  Make strings immutable
  -short-paths  Shorten paths in types
  -strict-sequence  Left-hand part of a sequence must have type unit
  -strict-formats  Reject invalid formats accepted by legacy implementations
     (Warning: Invalid formats may behave differently from
      previous OCaml versions, and will become always-rejected
      in future OCaml versions. You should use this flag
      to detect and fix invalid formats.)
  -unsafe  Do not compile bounds checking on array and string access
  -v  Print compiler version and location of standard library and exit
  -verbose  Print calls to external commands
  -version  Print version and exit
  -vnum  Print version number and exit
  -w <list>  Enable or disable warnings according to <list>:
        +<spec>   enable warnings in <spec>
        -<spec>   disable warnings in <spec>
        @<spec>   enable warnings in <spec> and treat them as errors
     <spec> can be:
        <num>             a single warning number
        <num1>..<num2>    a range of consecutive warning numbers
        <letter>          a predefined set
     default setting is "+a-4-6-7-9-27-29-32..39-41..42-44-45-48-50-102"
  -warn-error <list>  Enable or disable error status for warnings according
     to <list>.  See option -w for the syntax of <list>.
     Default setting is "-a"
  -warn-help  Show description of warning numbers
  -where  Print location of standard library and exit
  -color {auto|always|never}  Enable or disable colors in compiler messages
    The following settings are supported:
      auto    use heuristics to enable colors only if supported
      always  enable colors
      never   disable colors
    The default setting is 'auto', and the current heuristic
    checks that the TERM environment variable exists and is
    not empty or "dumb", and that isatty(stderr) holds.
  -nopervasives  (undocumented)
  -dsource  (undocumented)
  -dparsetree  (undocumented)
  -dtypedtree  (undocumented)
  -drawlambda  (undocumented)
  -dlambda  (undocumented)
  -help  Display this list of options
  --help  Display this list of options
@andreypopp

This comment has been minimized.

Copy link
Contributor

commented Oct 7, 2018

Last time I tried using bsc directly it required bsconfig.json but that could be empty (I didn't look into why it requires though).

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Oct 7, 2018

It doesn't seem to need a bsconfig.json to build a simple hello world in ocaml syntax with bsc hello.ml. With Reason syntax I think there is currently not a way to pass the information about it being reason syntax directly to bsc as flags. Running bsc hello.re gives don't know what to do with hello.re. Maybe this is something we can coordinate with BuckleScript on?

@zploskey

This comment has been minimized.

Copy link
Collaborator

commented Oct 7, 2018

Oh I see. For Reason it applies the extra preprocessors. Well, we can do that too I suppose.

bs_package_flags = -bs-package-name hello-esy-dune-bsb
src_root_dir = /home/zach/src/hello-esy-dune-bsb
bsc = /home/zach/src/hello-esy-dune-bsb/node_modules/bs-platform/lib/bsc.exe
bsdep = /home/zach/src/hello-esy-dune-bsb/node_modules/bs-platform/lib/bsb_helper.exe
warnings = -w -30-40+6+7+27+32..39+44+45+101+a -warn-error +a
bsc_flags = -bs-suffix -nostdlib -I '/home/zach/src/hello-esy-dune-bsb/node_modules/bs-platform/lib/ocaml' -bs-g -g -color always
ppx_flags = 
bs_package_includes = 
bs_package_dev_includes = 
namespace = 
bsb_dir_group = 0
refmt = /home/zach/src/hello-esy-dune-bsb/node_modules/bs-platform/lib/refmt.exe
reason_react_jsx = 
refmt_flags = --print binary
bsc_lib_includes = -I bin
rule build_ast_and_module_sets_from_re
  command = ${bsc} -pp "${refmt} ${refmt_flags}" ${reason_react_jsx}  ${ppx_flags} ${warnings} ${bsc_flags} -c -o ${out} -bs-syntax-only -bs-binary-ast -impl ${in}
  description = Building ${out}
build bin/rehello.mlast : build_ast_and_module_sets_from_re $src_root_dir/bin/rehello.re
rule build_deps
  command = ${bsdep} ${namespace} -g ${bsb_dir_group} -MD ${in}
  description = Building ${out}
build bin/rehello.mlast.d : build_deps bin/rehello.mlast
rule build_cmj_cmi
  command = ${bsc} ${bs_package_flags} -bs-assume-no-mli -bs-no-builtin-ppx-ml -bs-no-implicit-include ${bs_package_includes} ${bsc_lib_includes} ${bsc_extra_includes} ${warnings} ${bsc_flags} -o ${out} -c  ${in} $postbuild
  depfile = ${in}.d
  description = Building ${out}
build bin/rehello.cmj |  $src_root_dir/bin/rehello.bs.js bin/rehello.cmi : build_cmj_cmi bin/rehello.mlast
  bsc_flags = $bsc_flags -bs-re-out -bs-super-errors
  bs_package_flags = $bs_package_flags  -bs-package-output commonjs:bin
build build.ninja : phony  ||  bin/rehello.mlast.d
@andreypopp

This comment has been minimized.

Copy link
Contributor

commented Oct 8, 2018

Running bsc hello.re gives don't know what to do with hello.re. Maybe this is something we can coordinate with BuckleScript on?

Dune's preprocessing pipeline should deal with that as it does now for Reason compiling with ocamlopt/ocamlc.

@jordwalke

This comment has been minimized.

Copy link
Contributor

commented Oct 10, 2018

This is a cool idea, and I'd be curious to see how this works. One question I have is how / if Dune with bsc would compile dependencies? All bs-platform packages currently use bsb. Bsb sort of requires that everything use bsb in order for one bsb package to work. Building with Dune+bsc seems to put us in the same place. I really do like how right now Dune+jsoo can use arbitrary dependencies regardless of how they were compiled and regardless of which build systems they use. I wonder if it would be possible to use jsoo for dependencies when you can't control which build system they use, and then use Dune+bsc for the top level project.

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Oct 10, 2018

Full support for bucklescript would include support for interpreting bsconfig.json. To make external packages work with this setup, I think we should simply install external dependencies with the full source and build them on demand. This will only make dune dependencies available through the bucklescript target, but it is acceptable to provide new features that are dune only.

I wonder if it would be possible to use jsoo for dependencies when you can't control which build system they use, and then use Dune+bsc for the top level project.

Which cmi's would the bucklescript project use? I think you still need bucklescript to generate the cmi for w/e library it plans to use.

@jaredly

This comment has been minimized.

Copy link
Author

commented Oct 10, 2018

@jordwalke jsoo & bsc have incompatible runtime representations, so there would have to be a serialization/interop layer between them if they are to be used together :/

I think "making dune able to build all bucklescript projects (e.g. support bsconfig.json)" is a way bigger task than just "make dune able to use bsc as the compiler instead of jsoo or ocamlc". So maybe it makes sense to tackle the simpler one first?

@diml

This comment has been minimized.

Copy link
Member

commented Oct 10, 2018

I think "making dune able to build all bucklescript projects (e.g. support bsconfig.json)" is a way bigger task than just "make dune able to use bsc as the compiler instead of jsoo or ocamlc". So maybe it makes sense to tackle the simpler one first?

That's my impression as well. Being able to build a project using dune files with bsc seems like a very good first goal.

@abate

This comment has been minimized.

Copy link

commented Apr 13, 2019

any new about this ? the ocaml/dune#bucklescript bucklescript branch seems a bit outdated. How can I help ?

@ELLIOTTCABLE

This comment has been minimized.

Copy link

commented Apr 14, 2019

👍 here — I have some free-time at the moment to dedicate to something BuckleScript-related. @abate, if you figure out what's necessary, and want to tag somebody in, catch me in Discord?

@diml

This comment has been minimized.

Copy link
Member

commented Apr 15, 2019

I don't really know how the bucklescript compilation pipeline works, but I'm assuming that it is fairly similar to OCaml.

As a result, I suggest to proceed as follow:

  • setup an environment where you have both ocamlc and bsc in your path
  • do: cp $(which bsc) $(which ocaml). i.e. overwrite ocamlc by bsc. It's important to overwrite the ocamlc binary in place as dune expect to find all the ocaml* tools in the same directory
  • try to build a simple hello world program in bytecode (i.e. build prog.bc rather than prog.exe) without preprocessing and without using libraries
  • do whatever changes necessary in dune to make it work
  • try to define a simple library and use it in the executable
  • do whatever changes necessary in dune to make it work

By looking at the diff, we should understand better how all this work and how to properly integrate this feature into dune.

@abate

This comment has been minimized.

Copy link

commented Apr 21, 2019

I played a bit with this idea and followed your plan :
First I had to copy the node_modules/lib directory in my _opam directory and add copy the Makefile.config in _opam/lib/ocaml/Makefile.config

Then I linked the bsc file pointing to _opam/lib/node_modules/bs-platform/lib/bsc and on top of my ocamlc. In the end I created a hello_world.ml and dune file and run dune build hello_world.bc --verbose

Running[3]: (cd _build/default && _opam/bin/ocamlc.opt -w @a-4-29-40-41-42-44-45-48-58-59-60-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -bin-annot -I .hello_world.eobjs/byte -no-alias-deps -o .hello_world.eobjs/byte/hello_world.cmo -c -impl hello_world.ml)
Rule failed to generate the following targets:
- .hello_world.eobjs/byte/hello_world.cmo

The hello_world.js file is there, but I get an error as the bsc compiler doesn't know how to generate a cmo file and dune complains. Adding a library stanza to dune yield a similar result with the command :
dune build hello.cma --verbose , and the js artifact is correctly generated. I tried to add another module to test the dependency resolution, but this didn't work out of the box.

Judging from these simple experiments, I guess we need to add a new Js mode to dune to make it select the correct compiler and to stop throwing errors for cm? files that bsc is not going to generate.

But I'm not sure what should be changed. Ideally, calling dune with dune build hello.bsc.js I would expect dune to call bsc instead of ocamlc and picking up the correct makefile. I'm not sure how to use ocamldep or bsb .

Something else that bugs me, is that I didn't find a way to simply install bsc using opam. i tried creating an opam switch using the BS ocaml-variants, but then I'm not sure how to proceed to get bsc compiled and installed inside an opam switch. It would be great if bsc could be installed as an opam package.

I'll keep looking at this, if anyone has any comments or idea this would be of great help.

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Apr 21, 2019

Good work @abate and your initiative is appreciated.

To proceed with the experiment, perhaps we should try to convince dune that .cmo and .cmj files serve the same purpose. One way we can do that is by replacing cmo with cmj in dune. Another way is to add a .cmj -> .cmo copying rule when invoking the compiler from dune.

@diml

This comment has been minimized.

Copy link
Member

commented Apr 23, 2019

One way we can do that is by replacing cmo with cmj in dune

That's indeed what I had in mind for the experiment.

@abate

This comment has been minimized.

Copy link

commented Apr 23, 2019

I started experimenting here : abate@47ff78a

To set everything up you should :

yarn add bs-platform --modules-folder $OPAM_SWITCH_PREFIX/lib/node_modules
ln -s $OPAM_SWITCH_PREFIX/lib/node_modules/bs-platform/lib/bsc $OPAM_SWITCH_PREFIX/bin/bsc
cp $OPAM_SWITCH_PREFIX/lib/ocaml/Makefile.config $OPAM_SWITCH_PREFIX/lib/node_modules/bs-platform/lib/ocaml/

I've set this up in a brand new switch ocaml-variants.4.02.3+buckle-master just to be on the safe side.

This way setting up the env variable DUNE_BSC=1, dune will use bsc instead of ocamlc. Now I've tried to attack this cmj -> cmo problem, but it's not that easy.

Maybe somebody can have a look at what I've done and tell me if there is an easier way of doing it ?

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Apr 24, 2019

Seems like what you're trying to do is to add proper support for bucklescript in dune. That's definitely the right way to go eventually, but I think that initially what we had in mind was far simpler, just sed s/cmo/cmj/g and then see what breaks.

@abate

This comment has been minimized.

Copy link

commented Apr 24, 2019

I tried, but the very nature of dune, doesn't allow for this kind of monkey-patching. There are many hidden dependencies, and in the end it was just easier for me to hack in the code and to try to figure out how to untangle this bundle. But it seems I'm missing something in particular regarding extensions (I can't make bs.js to work), and I'm not sure if adding a new kind and just say ocamlc == bsc is the right way to go . Maybe poeple with a bit more expertise than me in the dune code base can have a quick look and point me in the right direction ?

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Apr 24, 2019

There's also a bit of a complication with preprocessing. I believe that we'll need some sort of cross compilation here as I'm not sure if we can make our preprocessor work in bucklescript. To hack around this, we might need to build bucklescript dune with the installed version. This will make sure the native pp is used

@diml

This comment has been minimized.

Copy link
Member

commented Apr 24, 2019

@rgrinberg since bucklescript seems to produce a different set of artifacts, I was thinking that we would store them in a separate directory, for instance in .<lib>.objs/bs. Although, that would only work well if the cmi files are compatible between ocamlc and bsc.

@jaredly

This comment has been minimized.

Copy link
Author

commented Apr 25, 2019

the .cmi files are compatible, so that should work :)

@diml

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

Great :)

@Lupus

This comment has been minimized.

Copy link

commented May 30, 2019

As of bsconfig.json, @andreypopp isn't the game plan to migrate that to esy's package.json and have esy manage the dependencies, leaving only actual building phase to dune?

@Lupus

This comment has been minimized.

Copy link

commented Jul 10, 2019

Has there been any progress on this? We're thinking of building some sort of abstraction library (in ReasonML) that is supposed to work in both BuckleScript and OCaml native(esy,dune) and so far no clues on how to build that from same codebase.

Interestingly, BuckleScript has an issue to support limited subset of dune files.

@bobzhang @chenglou do you guys have any vision to share on what should be the best way to build cross-platform OCaml/Reason projects targeting BuckleScript on the JS side?

@rgrinberg

This comment has been minimized.

Copy link
Member

commented Jul 11, 2019

No progress so far. Nobody in the current dune team is qualified to maintain such a project and nobody from the community has stepped up.

@diml

This comment has been minimized.

Copy link
Member

commented Jul 11, 2019

Supporting dune files in BuckleScript is a really interesting idea!

FTR, the original goal of Jbuilder was to provide a second implementation of a tool that read jbuild files and interpret them to build OCaml projects. The first implementation was Jenga which wasn't suitable for the opensource world, and in particular the opam world. It succeeded because the jbuild language, which has now evolved into the dune language sits just at the right place: it's simple and high-level enough to make it doable to have several implementations of it, but it is complete enough to describe the majority of projects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
10 participants
You can’t perform that action at this time.