Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Make javac run in a subprocess. Fixes #809. #886

Merged
merged 2 commits into from

3 participants

Jean Niklas L'orange Phil Hagelberg Anthony Grimes
Jean Niklas L'orange
Collaborator

If we're not making javac run in a subprocess, libraries the project
depends on which leiningen has another version of will clash. This is
because leiningen append its standalone to java's bootclasspath, which
makes the ToolProvider add these classpaths automatically into the java
compiler it returns. By starting a subprocess without leiningen added to
the bootclasspath, we avoid this library clashing.

UI changes as a result of this patch: Whenever javac fails, the task one
wanted to run will be responsible for aborting Leiningen, whereas javac
did this itself when it discovered that the compilation failed. As we're
running in a subprocess, an abort message such as

Uberjar aborting because jar/compilation failed: Subprocess failed

will appear, though this will only be appended to the javac error
message and the "Compilation of Java sources (lein javac) failed."
message.

This commit also adds the overhead of starting a subprocess when one
have to compile java sources, but it will not start one if no
.class-files are outdated/not existing.

Not tested on Windows, though unless leiningen.core.eval does some unknown magic with subprocesses, this shouldn't be an issue.

Jean Niklas L'orange hyPiRion Make javac run in a subprocess. Fixes #809.
If we're not making javac run in a subprocess, libraries the project
depends on which leiningen has another version of will clash. This is
because leiningen append its standalone to java's bootclasspath, which
makes the ToolProvider add these classpaths automatically into the java
compiler it returns. By starting a subprocess without leiningen added to
the bootclasspath, we avoid this library clashing.

UI changes as a result of this patch: Whenever javac fails, the task one
wanted to run will be responsible for aborting Leiningen, whereas javac
did this itself when it discovered that the compilation failed. As we're
running in a subprocess, an abort message such as

    Uberjar aborting because jar/compilation failed: Subprocess failed

will appear, though this will only be appended to the javac error
message and the "Compilation of Java sources (lein javac) failed."
message.

This commit also adds the overhead of starting a subprocess when one
have to compile java sources, but it will not start one if no
.class-files are outdated/not existing.

Not tested on Windows.
377a98f
Phil Hagelberg
Owner

Thanks. I agree that using a subprocess is the right thing to do here; I didn't understand the interaction with the bootclasspath when this was first implemented.

A couple notes:

0) On line 102 it checks to make sure we have the Java compiler in Leiningen's process. This isn't quite right since JAVA_CMD and LEIN_JAVA_CMD can point to different executables, meaning Leiningen can be run with a JRE and the project run with a JDK and things should still work. So this check should be moved inside the eval-in-project call.

1) The codebase currently doesn't use any prefix-style :require clauses; for consistency could you replace those with full namespace names? This makes it easier to use grep to see where a namespace is being consumed.

2) If you add ^:displace metadata to the Clojure dependency inside the :dependencies vector it will ensure that the project's specified version of Clojure isn't overridden by this profile.

Thanks for submitting this change; would be happy to merge it once the above considerations are addressed.

Phil Hagelberg
Owner

Though it occurs to me that since I don't actually use the javac task it might be helpful to get a review from @Raynes or @michaelklishin as well to see if they have thoughts.

Anthony Grimes
Collaborator

I think this is fine, but I hate having to do the subprocess stuff. Might want to look into just shelling out at some point and see how hard/reliable it would be to do it with conch instead. It'd certainly be faster.

Phil Hagelberg
Owner

You mean it would be faster to shell out to javac in a way that didn't require loading Clojure code? That's probably true. Still, this approach is much better than what we currently have.

If we do decide to hit javac directly we can probably do it with the same subprocess functions that eval-in-project is built upon; we already roll our own sh function.

Jean Niklas L'orange hyPiRion Check compiler existence within javac subprocess.
Also added ^:displace to avoid adding multiple clojure-versions onto the
classpath, and fully expanded non-`clojure.core` namespaces.
d38ecd3
Jean Niklas L'orange
Collaborator

Re: 1) - not entirely sure how thorough you wanted me to be there. I've not expanded clojure.core functions and macros into clojure.core/foo, but every other namespace is now fully expanded.

The other issues have been resolved.

Phil Hagelberg technomancy merged commit 3a1f84f into from
Phil Hagelberg
Owner

Never mind; easier for me to fix it than explain what I'm talking about. =)

Thanks.

Phil Hagelberg
Owner

Prefix list removal in :require: 9cceba8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 13, 2012
  1. Jean Niklas L'orange

    Make javac run in a subprocess. Fixes #809.

    hyPiRion authored
    If we're not making javac run in a subprocess, libraries the project
    depends on which leiningen has another version of will clash. This is
    because leiningen append its standalone to java's bootclasspath, which
    makes the ToolProvider add these classpaths automatically into the java
    compiler it returns. By starting a subprocess without leiningen added to
    the bootclasspath, we avoid this library clashing.
    
    UI changes as a result of this patch: Whenever javac fails, the task one
    wanted to run will be responsible for aborting Leiningen, whereas javac
    did this itself when it discovered that the compilation failed. As we're
    running in a subprocess, an abort message such as
    
        Uberjar aborting because jar/compilation failed: Subprocess failed
    
    will appear, though this will only be appended to the javac error
    message and the "Compilation of Java sources (lein javac) failed."
    message.
    
    This commit also adds the overhead of starting a subprocess when one
    have to compile java sources, but it will not start one if no
    .class-files are outdated/not existing.
    
    Not tested on Windows.
Commits on Dec 14, 2012
  1. Jean Niklas L'orange

    Check compiler existence within javac subprocess.

    hyPiRion authored
    Also added ^:displace to avoid adding multiple clojure-versions onto the
    classpath, and fully expanded non-`clojure.core` namespaces.
This page is out of date. Refresh to see the latest.
Showing with 41 additions and 15 deletions.
  1. +41 −15 src/leiningen/javac.clj
56 src/leiningen/javac.clj
View
@@ -1,7 +1,10 @@
(ns leiningen.javac
"Compile Java source files."
(:require [leiningen.classpath :as classpath]
- [leiningen.core.main :as main]
+ [leiningen.core
+ [main :as main]
+ [eval :as eval]
+ [project :as project]]
[clojure.java.io :as io])
(:import java.io.File
javax.tools.ToolProvider))
@@ -66,27 +69,50 @@
"-d" (:compile-path project)]
files)))
+;; Pure java projects will not have Clojure on the classpath. As such, we need
+;; to add it if it's not already there.
+(def subprocess-profile
+ {:dependencies [^:displace
+ ['org.clojure/clojure (clojure-version)]]
+ :eval-in :subprocess})
+
+(defn- subprocess-form
+ "Creates a form for running javac in a subprocess."
+ [compile-path files javac-opts]
+ `(let [abort# (fn [& msg#]
+ (.println java.lang.System/err (apply str msg#))
+ (java.lang.System/exit 1))]
+ (if-let [compiler# (javax.tools.ToolProvider/getSystemJavaCompiler)]
+ (do
+ (println "Compiling" ~(count files) "source files to" ~compile-path)
+ (.mkdirs (clojure.java.io/file ~compile-path))
+ (when-not (zero?
+ (.run compiler# nil nil nil
+ (into-array java.lang.String ~javac-opts)))
+ (abort# "Compilation of Java sources(lein javac) failed.")))
+ (abort# "lein-javac: system java compiler not found; "
+ "Be sure to use java from a JDK\nrather than a JRE by"
+ " either modifying PATH or setting JAVA_CMD."))))
+
;; We can't really control what is printed here. We're just going to
;; allow `.run` to attach in, out, and err to the standard streams. This
;; should have the effect of compile errors being printed. javac doesn't
;; actually output any compilation info unless it has to (for an error)
;; or you make it do so with `-verbose`.
-(defn- run-javac-task
- "Run javac to compile all source files in the project."
+(defn- run-javac-subprocess
+ "Run javac to compile all source files in the project. The compilation is run
+ in a subprocess to avoid it from adding the leiningen standalone to the
+ classpath, as leiningen adds itself to the classpath through the
+ bootclasspath."
[project args]
(let [compile-path (:compile-path project)
- files (stale-java-sources (:java-source-paths project) compile-path)]
+ files (stale-java-sources (:java-source-paths project) compile-path)
+ javac-opts (vec (javac-options project files args))
+ form (subprocess-form compile-path files javac-opts)]
(when (seq files)
- (if-let [compiler (ToolProvider/getSystemJavaCompiler)]
- (do
- (main/info "Compiling" (count files) "source files to" compile-path)
- (.mkdirs (io/file compile-path))
- (when-not (zero? (.run compiler nil nil nil
- (javac-options project files args)))
- (main/abort "Compilation of Java sources (lein javac) failed.")))
- (main/abort "lein-javac: system java compiler not found;"
- "Be sure to use java from a JDK\nrather than a JRE by"
- "either modifying PATH or setting JAVA_CMD.")))))
+ (eval/eval-in
+ (project/merge-profiles project [subprocess-profile])
+ form))))
(defn javac
"Compile Java source files.
@@ -99,4 +125,4 @@ Like the compile and deps tasks, this should be invoked automatically when
needed and shouldn't ever need to be run by hand. By default it is called before
compilation of Clojure source; change :prep-tasks to alter this."
[project & args]
- (run-javac-task project args))
+ (run-javac-subprocess project args))
Something went wrong with that request. Please try again.