NPE on connect with embedded nREPL server

[DerGuteMoritz]

Expected behavior

Embedding an nREPL server as described in https://github.com/clojure-emacs/cider-nrepl#via-embedding-nrepl-in-your-app works.

Actual behavior

With version 0.15.x this still worked but with 0.16.0-SNAPSHOT (specifically version bd7b2bc4c780549594bd7f03c772465216c1e9ca), the server runs into a NullPointerException when a client connects, like this:

18:26:38.661 ERROR [clojure.tools.nrepl.server] (clojure-agent-send-off-pool-3) Unhandled REPL handler exception processing message {:op describe, :session 2a63e04c-29af-4e9b-acbb-5247dcc140f2, :id 61}
java.lang.NullPointerException: null
	at clojure.core$deref_future.invokeStatic(core.clj:2297) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.core$deref.invokeStatic(core.clj:2317) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.core$deref.invoke(core.clj:2303) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at cider.nrepl$wrap_pprint_fn$fn__24216.invoke(nrepl.clj:85) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.middleware$wrap_conj_descriptor$fn__23554.invoke(middleware.clj:22) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at cider.nrepl$wrap_tracker$fn__24336.invoke(nrepl.clj:410) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.middleware$wrap_conj_descriptor$fn__23554.invoke(middleware.clj:22) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.middleware.session$session$fn__23873.invoke(session.clj:192) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.middleware$wrap_conj_descriptor$fn__23554.invoke(middleware.clj:22) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at cider.nrepl$wrap_apropos$fn__24230.invoke(nrepl.clj:124) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.middleware$wrap_conj_descriptor$fn__23554.invoke(middleware.clj:22) ~[foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.server$handle_STAR_.invokeStatic(server.clj:19) [foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.server$handle_STAR_.invoke(server.clj:16) [foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.tools.nrepl.server$handle$fn__23930.invoke(server.clj:28) [foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.core$binding_conveyor_fn$fn__5478.invoke(core.clj:2027) [foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at clojure.lang.AFn.call(AFn.java:18) [foo-0.1.0-SNAPSHOT-standalone.jar:na]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_144]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_144]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_144]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_144]

Steps to reproduce the problem

1. Embed an nREPL server as described in https://github.com/clojure-emacs/cider-nrepl#via-embedding-nrepl-in-your-app
2. Connect to the server using the Emacs CIDER client

Environment & Version information

cider-nrepl version

0.16.0-SNAPSHOT (bd7b2bc4c780549594bd7f03c772465216c1e9ca)

Java version

1.8

Operating system

Linux

[DerGuteMoritz]

Oh right, forgot to mention that you need to run it as a standalone app (e.g. from an uberjar) rather than from lein repl or something (with that it works for me). My guess is that it is related to the dynamic middleware loading refactoring.

[orb]

I'm having this problem too. I don't know which commit specifically broke this, but reverting to [cider/cider-nrepl "0.16.0-20170916.095133-1"] fixes it. (there's nothing special about that version except it's a known good version from a previous release)

And, as with the original reporter, we are running without lein. It's not an uberjar, but effectively the same thing. (https://github.com/orb/lein-metajar)

[orb]

A little more investigation.

Given thisproject.clj

(defproject nrepl-bug "0.1.0-SNAPSHOT"
  :description "CIDER nrepl test"
  :url "https://github.com/clojure-emacs/cider-nrepl/issues/447"
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [org.clojure/tools.nrepl "0.2.12"]
                 [cider/cider-nrepl "0.16.0-SNAPSHOT"]]
  :profiles {:uberjar {:aot [nrepl-bug.core]}}
  :main nrepl-bug.core)

And this source:

(ns nrepl-bug.core
  (:gen-class)
  (:require [clojure.tools.nrepl.server :as nrepl-server]
            [cider.nrepl :as cider]))

(defn -main []
  (println (format "Starting with classpath: %s" (System/getProperty "java.class.path")))
  (nrepl-server/start-server
   :port 6666
   :handler cider/cider-nrepl-handler))

I get a failure with with cider-jack-in to localhost 6666, running the server with:

$ lein uberjar
$ java -jar target/nrepl-bug-0.1.0-SNAPSHOT-standalone.jar

Changing the cider link to be more resolves the problem:

(ns nrepl-bug.core
  (:gen-class)
  (:require [clojure.tools.nrepl.server :as nrepl-server]))

(defn nrepl-handler []
  (require 'cider.nrepl)
  (ns-resolve 'cider.nrepl 'cider-nrepl-handler))

(defn -main []
  (println (format "Starting with classpath: %s" (System/getProperty "java.class.path")))
  (nrepl-server/start-server
   :port 6666
   :handler (nrepl-handler)))

I haven't looked at the code to see why this might cause the problem, but this is a good workaround for now, at the very least.

[bbatsov]

That's likely related to the deferred middleware changes introduced by @vspinu in 0.16. His PR should give you a good idea what exactly caused this I guess.

[bbatsov]

Ops I forgot to post it - #438

[vspinu]

This problem is related but different from what I have reported in the pull - the parallel loading of middleware, which is now fixed. The fact that splitting cider.nrepl out of main (ns ... (:require ...)) suggests (again) that something is amis with the require itself. I wish I could replicate this outside of cider somehow.

[daveliepmann]

This issue bit me as well. @orb's workaround helped.

Should I make a PR mentioning this issue in the embedding-nrepl-in-your-app instructions, at least until it's resolved?

[bbatsov]

@daveliepmann It'd be best to solve this #464 I guess, but anything's better than nothing until a proper fix.

[otwieracz]

Any progress here? With 0.18.0-SNAPSHOT I still had to use workaround

(defn cider-nrepl-handler []
  (require 'cider.nrepl)
  (ns-resolve 'cider.nrepl 'cider-nrepl-handler))

otherwise I am getting NPE.

[timvisher]

I am seeing an NPE even after applying this workaround when launched from an uberjar.

[timvisher]

I've been able to track it down (the failure of the workaround) to some kind of bad interaction with another library being on the classpath.

[timvisher]

It wound up being a transitive dependency that was :refer [cider-nrepl-handler]ing. That was fun to track down. (－‸ლ)

[bbatsov]

That's weird. I'm glad you managed to work this out!

[jiacai2050]

I'm using CIDER 0.23.0 (Lima) and meet this NPE in my cljs project, it turns out that my project have explict [com.cemerick/piggieback "0.2.1"] dependency, which seems cause this NPE, after remove this, everything is fine.

[bbatsov]

@jiacai2050 A while ago the coordinates of com.cemerick/piggieback were changed to cider/piggieback.

[agriffis]

I'm working in a project that uses @orb 's workaround from this issue to start a REPL in a Tomcat server:

(defn nrepl-handler []
  (require 'cider.nrepl)
  (ns-resolve 'cider.nrepl 'cider-nrepl-handler))

I'd like to add more middleware to the default list, specifically refactor-nrepl and iced-nrepl. This seems to involve re-implementing cider-nrepl-handler and also resolve-or-fail in my project just to add the extra middleware. Am I on the right track?

[agriffis]

Here's what I ended up with. Open to suggestions if there's a better approach.

(def development (Boolean/getBoolean "development"))

(defn cider-middleware
  "Roundabout way of getting the list of cider middleware, see
  https://github.com/clojure-emacs/cider-nrepl/issues/447"
  []
  (require 'cider.nrepl)
  (map resolve (var-get (ns-resolve 'cider.nrepl 'cider-middleware))))

(defn dev-middleware
  "Extra nrepl middleware for development, especially for vim-iced.
  https://liquidz.github.io/vim-iced/vim-iced.html#vim-iced-install-manually"
  []
  (when development
    (mapcat (fn [[ns syms]] (require ns) (map (partial ns-resolve ns) syms))
      [['refactor-nrepl.middleware ['wrap-refactor]] ['iced.nrepl ['wrap-iced]]])))

(defn nrepl-handler
  "Re-implement cider-nrepl-handler so we can add middleware to the default list."
  []
  (apply nrepl-server/default-handler (concat (cider-middleware) (dev-middleware))))

[bbatsov]

Looks like a good approach to me. Still, at some point we'll have to tackle the underlying issue with the deferred middleware loading.

[v3nd3774]

Here's what I ended up with. Open to suggestions if there's a better approach.

(def development (Boolean/getBoolean "development"))

(defn cider-middleware
  "Roundabout way of getting the list of cider middleware, see
  https://github.com/clojure-emacs/cider-nrepl/issues/447"
  []
  (require 'cider.nrepl)
  (map resolve (var-get (ns-resolve 'cider.nrepl 'cider-middleware))))

(defn dev-middleware
  "Extra nrepl middleware for development, especially for vim-iced.
  https://liquidz.github.io/vim-iced/vim-iced.html#vim-iced-install-manually"
  []
  (when development
    (mapcat (fn [[ns syms]] (require ns) (map (partial ns-resolve ns) syms))
      [['refactor-nrepl.middleware ['wrap-refactor]] ['iced.nrepl ['wrap-iced]]])))

(defn nrepl-handler
  "Re-implement cider-nrepl-handler so we can add middleware to the default list."
  []
  (apply nrepl-server/default-handler (concat (cider-middleware) (dev-middleware))))

Ty @agriffis , this workaround is still working now.

[AL4AL]

Today the workaround for me, for java.lang.ClassNotFoundException: com.sun.tools.javac.util.List error was to upgrade cider version to 0.26.0
to do so in vs code you should go in calva setting in vscode preferences