Closed
Description
Initial debugging steps
Before creating a report, especially around exceptions being thrown when running Leiningen, please check if the error still occurs after:
- [x] Updating to using the latest released version of Leiningen (`lein upgrade`).
- [x] Moving your `~/.lein/profiles.clj` (if present) out of the way. This contains third-party dependencies and plugins that can cause problems inside Leiningen.
- [x] Updating any old versions of plugins in your `project.clj`, especially if the problem is with a plugin not working. Old versions of plugins like nREPL and CIDER (as well as others) can cause problems with newer versions of Leiningen.
- [x] (If you are using Java 9 or newer), updating your dependencies to their most recent versions. Recent JDK's have introduced changes which can break some Clojure libraries.
Describe the bug
lein test
returns a 0
exit code for multiples of 256 test failures+errors.
To Reproduce
(defproject lein-test-exit-code "0.1.0-SNAPSHOT"
:dependencies [[org.clojure/clojure "1.10.1"]])
(ns lein-test-exit-code.core-test
(:require [clojure.test :refer [deftest is]]))
(defmacro gen-failing-deftests [n]
`(do
~@(for [i (range n)]
`(deftest ~(symbol (str "failing-" i))
(is false)))))
(gen-failing-deftests 256)
Actual behavior
$ lein test
...
...
Ran 256 tests containing 256 assertions.
256 failures, 0 errors.
$ echo $?
0
Expected behavior
A non-zero exit code should be returned for test failures.
Environment
- Leiningen Version:
Leiningen 2.9.5 on Java 1.8.0_275 OpenJDK 64-Bit Server VM
- Leiningen installation method: manual
- JDK Version:
openjdk version "1.8.0_275"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_275-b01)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.275-b01, mixed mode)
- OS: macOS 10.15
Metadata
Metadata
Assignees
Labels
No labels