snaplet-ghcjs: any advice on packaging and maybe AMD?

[johncant]

Hello!

I have forked snaplet-haste to use GHCJS, and was wondering if you could give me some advice on how best to make GHCJS find dependencies, or how I might use some kind of AMD to avoid having to serve up all the compiled js at once.

Here's my repo, and at the moment it works fine but is a bit of a hack:
https://github.com/johncant/snaplet-haste/tree/ghcjs

I shell out to cabal exec ghcjs-pkg list and parse the package DBs from the output, using them in ghcjs -package-db foo, but if the user's repo is a backend web app, they won't want to set ghcjs as their compiler in cabal.config, so I write that option to a new config file cabal.snaplet-ghcjs.config and use cabal --config-file=cabal.snaplet-ghcjs.config exec ghcjs-pkg list. Works, but is there a better way of invoking GHCJS to compile hs to js, detecting the package DBs?

Then, I just extract the Foo.jsexe/all.js and serve it up to the user, but this seems a bit wasteful. What would be a good way of dealing with the dependencies between different compiled JS files from different hs?

[luite]

Note that there was a bug in cabal exec that prevents it from detecting the configured compiler for the sandbox properly, that was only fixed very recently, but the pull request hasn't been merged yet: haskell/cabal#2859 (if you use a cabal config file with a line compiler: ghcjs you won't run into this problem)

As for your question: GHCJS does split-objects style linking for everything, and typically does not serve all code for a package (usually only a small fraction of the code is needed, especially when there are many type-specialized code paths). Loading whole packages is unlikely to be close to optimal. The easiest way to get code reuse, is by manually writing something that represents a typical use case for some set of packages, and loading the dependencies of that code from a shared static location with the generate-base / use-base functionality.

For example the GHCJS test framework makes use of this by linking all dependencies of the file https://github.com/ghcjs/ghcjs/blob/master/test/TestLinkBase.hs by running:

$   ghcjs -generate-base TestLinkBase -o base TestLinkMain.hs

You get a base bundle consisting of the JS files base.jsexe/out.base.js, base.jsexe/lib.base.js and base.jsexe/rts.js. Metadata of their contents is stored in base.jsexe/base.symbs. You include these files in your page, and link the full program with:

$ ghcjs -use-base SomeDir/base.jsexe/base.symbs myProgram.hs

The files myProgram.jsexe/out.js and myProgram.jsexe/lib.js then contain the generated code for your program, excluding anything that was already loaded through the base.jsexe bundle. You can build these things incrementally, building another base bundle on top of an existing one.

Currently the only way to select the code that ends up in a base bundle is by writing a module that uses the code, GHCJS includes all dependencies of the module (but not the module itself) in the generated bundle. If you need another method, let me know. It may not be hard to modify the linker for this.

Codeworld.info also makes use of this approach:

https://github.com/google/codeworld/blob/99e2d42a123dd4abfbc6c398651cba1790fc238f/codeworld-server/src/Build.hs#L39-L65

[johncant]

Thanks for answering, @luite ! snaplet-ghcjs works around this bug by spitting out and using a new config file. I'll keep an eye on the PR and change the snaplet when it's merged.

You've answered my second question directly and extremely well! I think it might be a good idea to wait until someone actually needs this functionality in snaplet-ghcjs before implementing it, especially before asking you to modify the GHCJS linker. Personally, I'm waiting until the marshalling saga #413 onwards is resolved before using the snaplet myself. I'd like to express enormous appreciation for GHCJS and its contributors, in proportion to their efforts.