Add task pkg to bundle lumo project with lumo build

[hlolli]

First of all, I don't expect this to get merged right away, this works mostly but I need to do further testing on Mac and Windows.

How to use:
As the boot task describes

p ppath        PROJECTPATH str  "String with absolute path to project root-dir."
m main         MAIN        str  "String with the name of the main namespace to load."
s source-paths SOURCEPATHS edn  "Vector of relative source paths or jar paths." 
a asset-paths  ASSETPATHS  edn  "Vector of resource paths to bundle."
r with-repl                bool "If passed, lumo repl promt will be started."

the users will need to provide absolute path to the root folder of the project they want to bundle with -p or --ppath.
This task assumes a project with -main entry point, so the namespace name for that main namespace needs to be provided, this also, as far as I've tested, enables lumo to compile all aot's that the project needs at runtime, by actually running the project, similar what we expect when compiling jars with Clojure uberjar task.
Source paths and asset-paths are seperated, the idea being that clj/s/c files need to be resolved into its namespace dir-paths and asset-paths need to maintain it's relative path. Therefore all resources folders that don't contain clojure code should be listed there and it will be implicitly bundled into lumo.internal (which lumo.io/resource can open).
Also with-repl is an option if the user wants an repl based application.

Using my dummy test project https://github.com/hlolli/lumo-quiz-game I compile it with

boot pkg --ppath /home/hlolli/tmp/lumo-quiz-game/ --main lumo-quiz-game.main --source-paths '["src" "~/.m2/repository/macchiato/fs/0.1.0/fs-0.1.0.jar" "~/.m2/repository/andare/andare/0.7.0/andare-0.7.0.jar"]' --asset-paths '["resources"]'

The linux_64 binary of that is here: https://s3.amazonaws.com/hlolli/lumo_quiz_game (note 53 mb json file bundled with this binary which is the reason for it's huge size).

The only problem compiling this project is due to my readline promt that causes the second binary stage to come to a hault on the task pkg-aot. To solve this I'd suggest adding deployment flag to lumo where after generations of aot/cache data, lumo quits. But waiting a bit and running the last two tasks of pkg succeeds in generating the final binary.

This took me some time, for example the decision to monkeypatch the require call is very risky and maybe stupid thing to do. But in comparison with bundling the node_modules which the users need to run their project is even riskier I think (would we use rollup, do we need plugins for their code? etc..), and there's always the option that the user bundles this themselves and add it as resource, escaping require-ing bundled node_modules. But as far as I've tested, the monkeypatch works.

Unrelated changes:
Node bumped to version 9.2.0
added to js Environment: setTimeout, setInterval and setImmediate
accedental indentation due to my emacs clojure-mode (this I will fix)
added entries to gitignore, important here is src/js/pkg.js which is a generated entry point into lumo.

[arichiardi]

Ok this is huge, both in size and importance of what you are trying to achieve. Nice job.

[arichiardi]

Just skimmed the code, one thing I was wondering is whether we could move the two task definitions to another source file and use the packager as a library from other build.boot.

So if I have a project with some lumo code in it, I could just have a build.boot:

(require 'lumo.boot.package :refer [pkg pkg-aot]

(deftask package-binary
  (comp
     (...)
     (pkg-aot)
     (pkg)))

I think having it here but as a separate library makes a lot of sense. I will try to understand the moving parts in order to see if I can help with it.

[arichiardi]

It does not seem to be possible actually.

[hlolli]

Yes I thought of that, it would be bit cleaner. If you know how to make boot recognize tasks in external files, it would be great I also rely on tasks in build.boot, so it's bit cyclical.

…

On 2 December 2017 at 17:17, Andrea Richiardi ***@***.***> wrote: It does not seem to be possible actually. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#315 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFyxgtiZWJQERsD3uaBfuW3Aw22q0wqCks5s8XgwgaJpZM4Qnyug> .

[arichiardi]

No well the thing is you would need to release the whole build.boot + necessary js bundler files as a library.

[arichiardi]

To be clearer, your build boot functions now would be contained in:

lumo-package/src/lumo/boot/package.clj
lumo-package/scripts/embed.js
....

It wouldn't be that complicated to do by copying things in a separate project...in this one probably too many things would change. Let me think about it a bit more.

[hlolli]

I just had the idea of actually makeing a seperate project out of this. That would have the added benefit of targeting specific lumo binary and skip 2 out of 3 nexe tasks needed. The source code is already available with the releases, so I have everything I need. lumo-pkg command line task or something, would write it in javascript probably. So for now, no need to look at this, I close this PR soon probably.

[arichiardi]

Awesome, I think that sounds like a better idea.

[hlolli]

Ok so I've published my very first experimentalbeta*1000 of this PR in this npm package
https://www.npmjs.com/package/pkg-lumo
the project is here:
https://github.com/hlolli/pkg-lumo

It's just "it works on my computer" mode right now, but if someone stumbles upon this closed PR in the future, I hope I will have polished this project better.

[arichiardi]

Awesome! I will have a look.