Haystack Interaction

An reproduction of the odd haystack interaction that can happen when using custom executors.

Note that you need OpenJDK 19 to test the issue.

Final Conclusion

So, after a day of debugging it looks like this is most likely an issue with Loom more than anything. Calling set-agent-send-off-executor! will cause pmap to use that executor. Deeply nested spawning of threads seems to lock up loom easily (ie. a few level of pmap nesting). Below is a code snippet that can be used to see it:

(defn ppmap
  [f coll]
  (->> coll
       (mapv
         (fn [x]
           (let [cf (java.util.concurrent.CompletableFuture.)]
             (Thread/startVirtualThread
               (bound-fn []
                 (try
                   (.complete cf (f x))
                   (catch Exception e
                     (.completeExceptionally cf e)))))
             cf)))
       (map deref)))

(let [n 4]
   (->> (range n)
        (ppmap
          (fn [_]
            (->> (range n)
                 (ppmap (fn [_]
                          (->> (range n)
                               (ppmap (fn [_]
                                        (ppmap identity (range n))))
                               (apply concat)
                               (into #{}))))
                 (apply concat)
                 (into #{}))))
        (apply concat)
        (into #{})))

Instructions

Start the nrepl server using the following command:

/usr/bin/clojure -Sdeps '{:deps {nrepl/nrepl {:mvn/version "1.0.0"} cider/cider-nrepl {:mvn/version "0.30.0"}} :aliases {:cider/nrepl {:main-opts ["-m" "nrepl.cmdline" "--middleware" "[cider.nrepl/cider-middleware]"]}}}' -M:repro:cider/nrepl

Open src/repro.clj and evaluate the buffer.

Evaluate the content of the comment block to trigger a compilation error and bask in the glory of a a hung nrepl session.

My Best Guess

After a lot of debugging, I have narrowed it down to an interaction between delay, pmap and changing the send-off executor (which is used by pmap) to the (Executors/newVirtualThreadPerTaskExecutor) (a JDK 19 preview feature w/ Loom).

This interaction can be seen with the use of pmap inside analyze-stacktrace-data and the delay used by ns-common-prefix in haystack.

I suspect it might be some weird interaction with haystack being an inlined dependency. Edit: I tried switching haystack to not be an inlined dependency and the issue was still hit, so it doesn't matter whether it's inlined or not.