New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mitigating the impact of "lein X auto" antipattern #1752

Closed
si14 opened this Issue Nov 2, 2014 · 7 comments

Comments

7 participants
@si14
Copy link

si14 commented Nov 2, 2014

lein X auto is handy at first glance, but actually it is an antipattern, as noted by Brandon Bloom. One of problems with this approach is an exploding amount of memory consumed by 2-3 processes needed just to compile/reload things (think lein figwheel + lein garden auto + lein hiccup watch, and if Clojure server is needed, one more Clojure process).

Is is possible to implement something akin to lein do for blocking, auto-like lein tasks? I believe it will help a bit with the memory issue.

@technomancy

This comment has been minimized.

Copy link
Owner

technomancy commented Nov 3, 2014

Dmitry Groshev notifications@github.com writes:

Is is possible to implement something akin to lein do for blocking,
auto-like lein tasks? I believe it will help a bit with the memory
issue.

My understanding is that one or two general-purpose higher-order auto
tasks exist; however, I have not taken the time to evaluate them. I
would be open to the idea of including one in Leiningen itself, provided
someone were to evaluate the alternatives and provide details about
which they believe to be the best choice.

-Phil

@si14

This comment has been minimized.

Copy link

si14 commented Nov 3, 2014

Another way I can think of is, indeed, auto implementation in Leiningen itself with callbacks into plugin code (similar to lein ring's root handler or main). It can give an incentive to implement auto-like plugins as a functions that can be called on change. It can also help with REPL workflow issues mentioned by @brandonbloom, because there will be a clear entry point into plugin that can be called manually.

@brandonbloom

This comment has been minimized.

Copy link
Contributor

brandonbloom commented Nov 3, 2014

I strongly advise against lein endorsing an auto-anything-on-file-save workflow. Besides my distain for the approach, I suspect that no general purpose mechanism will be sufficient. The problem is quite complex and a general purpose solution really would require Makefile-style dependency graphs. CLJS compilation, for example, would require recursive make on .d files, like C programs, since the set of depended-on files is dynamic and language specific. A lot of this specific logic is either buried in the Closure compiler or re-implemented in closure.clj as well as the various downstream cljs build tools. This is a hard problem and lein is extremely likely to run in to a XKCD 927 situation.

@weavejester

This comment has been minimized.

Copy link
Contributor

weavejester commented Nov 7, 2014

I've written a third-party plugin, lein-auto, that does just this.

@Deraen

This comment has been minimized.

Copy link

Deraen commented Nov 7, 2014

This is something that's been bothering me for some weeks and I hacked together a plugin which runs several tasks in one JVM: lein-simuflow, but the implementation is a mess and I'm not using it even myself...
Since I heard about Boot 2 few days ago I have pretty much abandoned the plugin as it seems that Boot provides everything (and more) I was aiming for.

@arrdem

This comment has been minimized.

Copy link
Contributor

arrdem commented Feb 18, 2015

While I personally find lein auto cloverage to be a huge part of my personal workflow these days, I agree with @brandonbloom. Supporting this pattern fully is going to require a dependency graph between tasks to be run and potentially time delays. Consider the case of wanting to run API client tests which depend on booting a server instance and waiting for it to come online before continuing and then tearing down the server instance when done.

{:user
 {:tasks
  {"integration"
   [["serve" "&"] ;; creates logical child "1" as in a *nix shell
    ["sleep" 5]   ;; stop, wait for boot
    ["cloverage"] ;; run cloverage alias/task
    ["kill"  1]   ;; kill the serve "job"
    ]}}}

Which would make lein auto integration "just work", but now we have near-full BASH implementation in lein just to escape booting some extra JVMs.

@technomancy

This comment has been minimized.

Copy link
Owner

technomancy commented May 22, 2017

I don't think this will ever be handled in Leiningen itself; let's leave it to 3rd-party plugins.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment