High memory consumption

[honza]

I created the default project with lein new chestnut <xyz>, started the repl with lein repl followed by (run) and (browser-repl). Before doing this, I removed all of my plugins from ~/.lein/profiles.clj.

I am now left with 3 java processes. Together, these processes use almost 2GB of memory.

This is on OSX 10.10 and here is the repl session if it helps.

~/Code/front on master  $ lein repl
nREPL server started on port 53644 on host 127.0.0.1 - nrepl://127.0.0.1:53644
REPL-y 0.3.5, nREPL 0.2.6
Clojure 1.6.0
Java HotSpot(TM) 64-Bit Server VM 1.7.0_60-b19
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

front.server=> (run)
2014-11-23 10:25:39.806:INFO:oejs.Server:jetty-7.6.13.v20130916
2014-11-23 10:25:39.856:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:10555
Starting web server on portStarting figwheel.
 10555 .
#<Server org.eclipse.jetty.server.Server@6f7714c8>
front.server=> Compiling ClojureScript.
Figwheel: Starting server at http://localhost:3449
Figwheel: Serving files from '(dev-resources|resources)/public'
front.server=> (browser-repl)
<< started Weasel server on ws://0.0.0.0:9001 >>
Type `:cljs/quit` to stop the ClojureScript REPL
nil

Is this normal? Can something be done about this?

[plexus]

Hi, thanks for reporting!

The three processes are leiningen, the repl, and figwheel. You can bring it down to two by using lein trampoline repl. This will replace the leiningen process with the repl once it starts.

Unfortunately Figwheel can't be used as a library, which means we have to invoke it through leiningen, so that creates an extra JVM. See also #47.

Apart from that this behavior is indeed "normal", although 2GB seems like a lot. For me it's more around 500MB.

There are various flags to control or limit the amount of memory the JVM uses, you could experiment with those.

[honza]

Would you be willing to share some details about your setup? I'm having a hard time understanding why someone people have such low memory numbers...

[plexus]

This is what I use

$  java -version
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

$  lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.1 LTS
Release:        14.04
Codename:       trusty

$  lein --version
Leiningen 2.5.0 on Java 1.8.0_25 Java HotSpot(TM) 64-Bit Server VM

Not much special for the rest. ~/lein/profiles.clj only contains the cider-nrepl plugin. I'm not sure what else could be relevant here.

Is it only with chestnut projects that you have this? Maybe compare with some other configs to see what part is so memory hungry. So do a lein trampoline repl in an empty directory, or in a new project, compare memory before/after (run), do lein figwheel separately and see how that behaves, etc.

[honza]

Thank you for your help!

I have upgraded to JDK 1.8 and now my setup matches what you described (apart from the OS, of course).

A couple of notes:

1. When using lein trampoline repl, I can't successfully execute the (browser-repl) call.
2. When using the cider-nrepl middleware in my profile, I only get 2 java processes instead of 3 and save about 500mb of memory.

Yes, I have noticed this with other projects as well but this case was unfortunately especially bad.

Thanks again.

[plexus]

Glad it's better now. It's good to know this is a pain point. I hope we can make it all work seemlessly in a single process eventually.

[bhauman]

I've been watching this and been thinking a lot about making a library thats executable from the repl.

[martinklepsch]

I think lively could be embedded in the process easily. You'd need to handle Clojurescript compilation on your own then though.

[plexus]

@bhauman that would be absolutely fabulous!

@martinklepsch how does lively relate to figwheel? Are they completely separate implementations?

[martinklepsch]

@plexus They're separate. Lively only implements the pushing of new code to the the browser. And does not care about compiling Clojurescript.

[bhauman]

Just want to let you know that I just released lein-figwheel 0.2.0-SNAPSHOT and you can now launch the autobuilder/change server from the new figwheel-sidecar library.

https://github.com/bhauman/lein-figwheel/blob/master/sidecar/src/figwheel_sidecar/auto_builder.clj#L38

Unfortunately config options are not yet validated at this level. The config validation is all still up in the leiningen plugin.

[plexus]

Thank you @bhauman , that's great news!