Update to ncurses causes sbt to throw an exception on startup

[amuttsch]

After upgrading ncurses, starting sbt shows the following error:

amu ~ >sbt
[ERROR] Failed to construct terminal; falling back to unsupported
java.lang.NumberFormatException: For input string: "0x100"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.valueOf(Integer.java:766)
at jline.internal.InfoCmp.parseInfoCmp(InfoCmp.java:59)
at jline.UnixTerminal.parseInfoCmp(UnixTerminal.java:233)
...

This is because of a recent update to ncurses: http://invisible-island.net/ncurses/NEWS.html#index-t20170506

    + modify tic/infocmp display of numeric values to use hexadecimal when
          they are "close" to a power of two, making the result more readable.

steps

I'm using Arch Linux and have ncurses updated from 6.0+20170429-1 to 6.0+20170527-1. Just run sbt after installing the newest version and the error will appear.

problem

sbt uses jline for some terminal stuff. jline itself uses the infocmp utility provided by ncurses to determine terminal capabilities. It didn't expect a hex value and only parsed decimal values. This behaviour was already fixed: jline/jline2@c1b1676

sbt should update its jline dependency to include the fix.

expectation

sbt should start without issues.

notes

Using sbt version 0.13.15.

Is it possible to force sbt to use a locally compiled version of jline?

Stackoverflow issue: https://stackoverflow.com/q/44317384/2014080
jline issue: jline/jline2#281

[eed3si9n]

You could try putting the following in project/plugins.sbt:

dependencyOverrides += "jline" % "jline" % "2.15-SNAPSHOT"

but I'm not sure if it would fix because the metabuild still needs to load.

[amuttsch]

Thanks for the fast reply. Unfortunately, it still throws the exception. I will stay on ncurses version 6.0+20170429-1 for now until this issue is resolved :)

Edit: I forgot to mention the version: I'm using 0.13.15.

[eatobin]

Thank you @amuttsch for posting this and your reply on StackOverflow. I will watch for developments. Appreciated. :-)

[BardurArantsson]

Yeah, I think we'll need a new release of jline. I took the liberty of submitting jline/jline3#127

[BardurArantsson]

Another potential workaround would be putting

-Djline.terminal=jline.UnsupportedTerminal

in the _JAVA_OPTIONS env var. That has some unpleasant side effects, though: You lose editing + up/down arrow + color + every single "Resolving ..." line will be printed to console. So only really usable if you're using sbt non-interactively.

[eed3si9n]

• http://lists.gnu.org/archive/html/bug-ncurses/2017-06/msg00000.html
• http://lists.gnu.org/archive/html/bug-ncurses/2017-06/msg00001.html

[tmccombs]

I also run into this problem with the scala REPL.

[grscheller]

Until the jline fixes get push out, you can temporarily "fix" the problem by downgrading ncurses after a Pacman `update:

    $ sudo pacman -Syu
    $ cd /var/cache/pacman/pkg
    $ sudo pacman -U ncurses-6.0+20170429-1-x86_64.pkg.tar.xz

[captainju]

I found using export TERM=xterm-color better than downgrading ncurses

[wsxarcher]

jline-2.14.4 is out with the fix

[amuttsch]

As well as jline-3.3.1

[Rogach]

Are there any plans to release updated version with the fix any time soon?

export TERM=xterm-color fixes exceptions, but breaks some other stuff on my system.

(I can make the PR if needed)

[dwijnand]

@Rogach A PR would be very helpful. Thank you.

[Rogach]

@dwijnand Should I make 2 PRs - one for 0.13 branch, one for 1.0.x?

[dwijnand]

That would be even more lovely! Thank you.

[gaydenko]

Is it possible to force old SBT projects (when SBT version is out of control) use new jline version on user's system base? (that is some magic in ~/.sbt)

[vn971]

It would be great to release a bug-fix version of SBT!

[dwijnand]

I think well be releasing a 0.13.16-M2 or 0.13.16 shortly, which thanks to @Rogach would have fixed this issue.

[avdv]

@liquidarmour I cannot reproduce your problem. Are you seeing any errors from JLine or anything else? Do you have any global plugins enabled?

[liquidarmour]

@avdv I just checked. Intellij had put plugins in both 0.13 and 1.0 directories. I removed them and tried again. Still no luck.

Whatever I type does not appear.

There are no errors in the build and no jline errors visible.

akka-http-docker> sbtVersion
[info] 1.0.1
akka-http-docker> version
[info] 0.1.0-SNAPSHOT
akka-http-docker> sbtBinaryVersion
[info] 1.0
akka-http-docker>

[liquidarmour]

Sorry, should be more precise. The idea.sbt plugin was the only global plugin installed. With or without it the same behaviour ensues.

[djmcgill]

Confirmed that I get the same issue with @liquidarmour 's template. After running sbt test the shell won't echo any more. No JLine errors.

[avdv]

@liquidarmour, @djmcgill do you have any environment variables set? TERM maybe? which terminal emulator are you using? And which ncurses version?

I assume that this does not happen for other SBT tasks like compile, but only for test?

[liquidarmour]

@avdv I have done some more testing and it appears the problem only occurs when the console exits without user interaction.

I do not have TERM set on any of my machines. 3 so far that have seen this problem.

The problem exists in all the terminals I have used on multiple OS. It happens in Linux in the standard Mint terminal and in Terminator. On MacOS it happens in the standard terminal and in iterm.

If I type sbt test or sbt compile the sbt process ends once the task is complete. The terminal is then not echoing.
If I enter the sbt repl, run any tasks I want including, test and compile, and then hit ctrl-c the terminal works properly. sbt run work as well, in that ctrl-c restores a good terminal.

So there is definitely a problem when the sbt process exits automatically, and not when the user forces an exit.

[avdv]

Which sbt script are you using? The original or the one from sbt-extras ?

Could you run the sbt script with bash in verbose mode:

bash -v sbt

and post the output?

[liquidarmour]

~/liquidarmour/akka-http-docker $ bash -v sbt
#!/bin/sh
if [ -f "$HOME/.sbtconfig" ]; then
  echo "Use of ~/.sbtconfig is deprecated, please migrate global settings to /usr/local/etc/sbtopts" >&2
  . "$HOME/.sbtconfig"
fi
exec "/usr/local/Cellar/sbt/1.0.1/libexec/bin/sbt" "$@"
[info] Loading global plugins from /Users/jameswilliams/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /Users/jameswilliams/liquidarmour/akka-http-docker/project
[info] Loading settings from build.sbt,version.sbt ...
[info] Set current project to akka-http-docker (in build file:/Users/jameswilliams/liquidarmour/akka-http-docker/)
[info] sbt server started at 127.0.0.1:5350
akka-http-docker> c~/liquidarmour/akka-http-docker $
~/liquidarmour/akka-http-docker $
~/liquidarmour/akka-http-docker $ bash -v sbt test
#!/bin/sh
if [ -f "$HOME/.sbtconfig" ]; then
  echo "Use of ~/.sbtconfig is deprecated, please migrate global settings to /usr/local/etc/sbtopts" >&2
  . "$HOME/.sbtconfig"
fi
exec "/usr/local/Cellar/sbt/1.0.1/libexec/bin/sbt" "$@"
[info] Loading global plugins from /Users/jameswilliams/.sbt/1.0/plugins
[info] Loading settings from plugins.sbt ...
[info] Loading project definition from /Users/jameswilliams/liquidarmour/akka-http-docker/project
[info] Loading settings from build.sbt,version.sbt ...
[info] Set current project to akka-http-docker (in build file:/Users/jameswilliams/liquidarmour/akka-http-docker/)
[info] Formatting 2 Scala sources {file:/Users/jameswilliams/liquidarmour/akka-http-docker/}root(test) ...
[info] Formatting 2 Scala sources {file:/Users/jameswilliams/liquidarmour/akka-http-docker/}root(compile) ...
[info] Reformatted 2 Scala sources {file:/Users/jameswilliams/liquidarmour/akka-http-docker/}root(compile).
[info] Reformatted 2 Scala sources {file:/Users/jameswilliams/liquidarmour/akka-http-docker/}root(test).
[info] Compiling 1 Scala source to /Users/jameswilliams/liquidarmour/akka-http-docker/target/scala-2.12/classes ...
[info] Done compiling.
[info] Run completed in 49 milliseconds.
[info] Total number of tests run: 0
[info] Suites: completed 0, aborted 0
[info] Tests: succeeded 0, failed 0, canceled 0, ignored 0, pending 0
[info] No tests were executed.
[success] Total time: 7 s, completed 20-Sep-2017 11:16:55

Note the first run release the terminal properly. The second does not.

[avdv]

Oh, your sbt script is not a symlink to the sbt launcher script, so not much to see here. Could you do the same for the real script?

bash -v "/usr/local/Cellar/sbt/1.0.1/libexec/bin/sbt" test

I suspect that there will be a stty call which messes up the terminal after SBT has run...

[avdv]

@liquidarmour @djmcgill SBT 1.0.2 is out, which seems to fix the bug you are seeing: #3507

[djmcgill]

I'll update and report back @avdv

[liquidarmour]

OK. Looks like we have made progress.

Updating the sbt version in the project fixes the problem. The installed version of 1.0.1 works properly as long as the project has version 1.0.2

:-)

@avdv Thanks for your help.

[dwijnand]

Nicely done @avdv.