Can't use Kotter without virtual terminal

[sgammon]

Describe the bug
Hello Kotter authors,

First of all I really love the library! And I feel like your post about Compose dependencies was spot on. I arrived at Kotter the same way you arrived at building it, incidentally.

To Reproduce

1. Download Kotter as part of a Gradle build
2. Hook it up to a Picocli terminal app
3. From zsh on a mac, try spawning a terminal (forced to be system terminal, non-virtual)
4. Observe failures

I have also observed some strange compiler errors, and errors in IDEA, when the app breaks in this manner. Other dependencies aren't found, for some reason, even though they were found in the previous compile run. This could simply be an error inside IDEA, but I'm sharing here in case it helps diagnose.

Expected behavior
I was expecting my case to be rather vanilla for the library, so I was surprised that it didn't work.

Screenshots
I am happy to provide a screenshot, but, in essence, a virtual terminal opens when the system terminal should be used instead. Since I'm on zsh on a Unix-like machine, ANSI color support should work and it should be able to access the system terminal. I'm not supporting Windows at the moment, so there is no code related to Windows/Jansi that might be interfering here.

Desktop (please complete the following information):

• OS: macOS Ventura
• Browser: N/A, it's CLI
• Version Latest

Additional context
I am going to try to capture more error information so I can make this ticket more useful, but I'm filing ahead of that because it has halted our ability to use Kotter :(

As a library author myself I'd definitely want to know about any such issues. Thank you again for all of your hard work on this tool!

[sgammon]

@bitspittle any ideas? friendly poke :) we really want to use kotter!

[bitspittle]

Thank you for the ping! And thank you for checking out Kotter!

I missed your original message for some reason, my apologies about that. It's been kind of crazy here.

I'm not sure how you're running your program. Can you be more specific about how you're starting your app?

Just in case, did you see the README about using installDist to run your program? https://github.com/varabyte/kotter#-running-examples (I'm guessing you did since it sounds like you made it all the way to the advanced section)

If there's anything you can share, I might be able to try running over here. I have an old sleeping MacBook I can spin up after a long slumber (I'm sure I'll need to let it upgrade several versions at this point...) if you have clear, specific repro steps, or maybe a project I can clone.

P.S. Sorry for asking about the Browser in the bug template. That's a bad copy/paste from another project. I'll remove it shortly.

[bitspittle]

Ah, here's a way we can verify if Kotter is working or busted in your environment.

This is the compiled text example from Kotter:
text-1.0-SNAPSHOT.zip

which I built using

# In Kotter project root
$ cd examples/text
$ ./gradlew distZip
# build is in build/distributions/

Try this:

• Download and unzip that file
• $ cd text-1.0-SNAPSHOT/bin
• Run ./text

If that renders text inside you're terminal, then yeah, you just need to use installDist or distZip actions in your own project (these come from the application Gradle plugin). If you're getting the virtual terminal to pop up instead, then we have a serious bug here and I'll want to try to reproduce it on my side :)

[sgammon]

@bitspittle unfortunately my power is out due to the storms on the west coast of the US, so a repro will have to wait :)

it's possible that I was just seeing a virtual terminal during a test run and got confused.

Re/how I'm running the app, there are Java entrypoints for testing, and ultimately it is targeted as a GraalVM native binary.

so it should have access directly to the system terminal, but maybe I'm hitting reflection issues etc. or something otherwise having to do with the native build. just offering this explanation as context until I can get to my computer again.

[bitspittle]

No rush! Good luck with the storms and the power outages.

About two months back, I did some random testing with GraalVM and ended up backing out due to various errors I was getting, but I am a GraalVM newb and was distracted by other priorities. So if you make progress with your GraalVM binary, I'd be very curious to hear more.

Note that you can configure your session to avoid ever even trying to open a virtual terminal. This might help as a way to avoid GraalVM worrying about Swing at all:

session(terminal =SystemTerminal()) { // Don't even let VirtualTerminal be an option
  /* ... */
}

Good luck, I'm hoping you're able to get things working! Feel free to stay in touch as you navigate this.

[bitspittle]

Just thought I would pop in and check if you've made any progress on diagnosing this issue

[sgammon]

@bitspittle not yet (i've moved to some server stuff temporarily), although with more time to think, i bet this is just caused by running Kotter in a test and Gradle redirecting the streams.

in fact, is there a way to gracefully degrade? i suppose i could try and wire together the structure to have two code paths: one which is animated in the CLI, and one which isn't, and simply logs. but Kotter is beautifully used by wrapping a session {}, etc, and that complicates a full copy of the hot path in my application.

does that make sense? if Kotter could fall back to doing nothing fancy, or I could otherwise render it inert, then there is no problem there. otherwise, any advice you may have about running Kotter in a test would be welcome (i haven't confirmed that to be the issue, i'm just strongly suspecting that's why i saw a virtual terminal unexpectedly on zsh/macOS).

[bitspittle]

(Sorry, been crazy here! Meant to reply earlier)

Did you ever confirm that you were using ./gradlew installDist and NOT ./gradlew run?

BTW, I'm probably a week away 🤞 from releasing a Kotlin/Native version of Kotter. Maybe that approach will work for you! Please follow #63 if you haven't given up and moved on yet :).

BTW, I have two code paths in my project that uses Kotter, one for when I expect the user to interact with it and one where I expect the user to run it on some server: https://github.com/varabyte/kobweb/blob/a8910bf5168a3e27be88ab49fce8b0a86322caac/cli/kobweb/src/main/kotlin/com/varabyte/kobweb/cli/run/Run.kt#L72 (the whole first part is the interactive experience using Kotter, while the tiny else is what it looks like without it).

The idea of "degrading" kotter is tricky, because it's not really doing anything that complicated. It's just trying to own the terminal's capabilities so that it can render and rerender text. The only way I've seen that fail is if some other process (like Gradle!) is already in control

[bitspittle]

Hey, I guess you have probably moved on, but I just released some dev snapshots up now that support Kotlin/Native.

Please see #63 (comment) if you're interested in more details.

[bitspittle]

I'm going to close this since...

• I could not reproduce
• I think you moved on
• and I just released 1.1.0-rc1 (the first version with Native support).

I'm sorry we never got to the bottom of what was happening in your situation. Hopefully native targets would have been a feature that would have been a useful solution for you.

Feel free to respond to this if you ever want to dig further, but either way, I hope you found a way forward that worked for you!

[NoUsernameProvided]

I would like to reopen this. For this reason it's impossible to use kotter as a dependency to kscript (or any kotlin script) because there's no gradle instDist step involved.

running "kscript ./test.kts" on linux throws either AWT error or if forced to use SystemTerminal then

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:79)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:484)
        at Main_Launcher$Companion.main(Main_Launcher.kt:6)
        at Main_Launcher.main(Main_Launcher.kt)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at org.jetbrains.kotlin.runner.AbstractRunner.run(runners.kt:70)
        at org.jetbrains.kotlin.runner.Main.run(Main.kt:188)
        at org.jetbrains.kotlin.runner.Main.main(Main.kt:198)
Caused by: java.lang.IllegalStateException: Unable to create a system terminal
        at org.jline.terminal.TerminalBuilder.doBuild(TerminalBuilder.java:329)
        at org.jline.terminal.TerminalBuilder.build(TerminalBuilder.java:271)
        at com.varabyte.kotter.terminal.system.SystemTerminal.<init>(SystemTerminal.kt:32)
        at kscript.scriplet.Launcher$EnvInstaller.run(Launcher.kts:32)
        at kscript.scriplet.Launcher$EnvChecker.run(Launcher.kts:26)
        at kscript.scriplet.Launcher.<init>(Launcher.kts:15)
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
        ... 9 more

[bitspittle]

I'm not familiar with Kotter interacts with kscript. Can you provide clear repro steps?

Depending on what kscript does itself, it may be impossible for Kotter to run at the same time, just a fundamental technical limitation with how terminals work. (I'd have to investigate but I don't have much time for it at the moment.) Basically, Kotter wants to control stdin and stdout, but kscript may be doing it first.

What's your usecase btw? I'm curious, why do you need to use kscript instead of creating a Kotter application directly?

You may also want to look at mordant, which is another Kotlin terminal API that is a bit lower level than Kotter while still feeling very user friendly. It may be another option if you're otherwise totally stuck.

[NoUsernameProvided]

mordant is just for prettifying output i think, not for interactivity.

kscript allows to use kotlin instead of bash, the whole purpose is not to have "applications". So use-case is substitute for bash, so it makes sense to support Kotter.

To reproduce

1. install kotlin
2. install kscript (https://github.com/kscripting/kscript#installation), depending on your distro it's just single command
3. create file test.kts :

#!/usr/bin/env kscript

@file:DependsOn("com.varabyte.kotter:kotter-jvm:1.1.0")

import com.varabyte.kotter.foundation.session
import com.varabyte.kotter.foundation.text.*
import com.varabyte.kotter.terminal.system.SystemTerminal

println("Hello from Kotlin!")
for (arg in args) {
    println("arg: $arg")
}

session(terminal = SystemTerminal()) {
  section { textLine("Hello, World") }.run()
}

4. Run "chmod +x ./test.kts" and then "./test.kts" to run the script from console.

It will throw an exception/

[bitspittle]

Thanks for the repro steps. I'll give that a try later but just keep in mind I'm not disagreeing with your goals. More like there's a chance it will be technically impossible (if kscript takes ownership of stdin or stdout, essentially).

I'm not sure I follow your logic about not using Kotter as an application though. Kotter exists to make it easy to write CLI applications, which is what it seems like exactly you want?

[NoUsernameProvided]

Well, use case for kscript is basically not to have to deal with jars.

It is used just for general scripting like you would use bash, but rather write scripts in kotlin. So you would put these scripts in plain text on your server, without building or compiling prior.

Kotter is obviously usable in applications, but it would also be nice if it was usable in shell scripting environment :) I'm not saying I'm not using Kotter in applications, it's just I also have to write some shell scripts to automate small things... It's just completely "parallel " use-cases, but both are console/terminal heavy.

[bitspittle]

Sure, I'll reopen this to look into it later. But no promises that I'll be able to do anything about it! My guess is that it's not possible (for the same reason you have to use installDist instead of gradle run, because gradle owns the terminal when it runs first)

[bitspittle]

Sorry for the long delay getting back to you.

I was not able to get kscript to work, and I'm pretty sure it's impossible. One thing that worries me: https://github.com/kscripting/kscript/blob/1c97c2197ddd08382834c014096ffe8b26727a4f/src/main/kotlin/io/github/kscripting/kscript/util/Executor.kt#L57

In other words, kscript launches a shell process that owns the input (which it has to, because it's trying to pull information out of its subprocess which would otherwise get lost in the background). This most likely prevents Kotter from taking ownership of the console itself.

For this reason, I'm pretty sure Kotter just can't ever work as a subprocess (although there's still gaps in my understanding here so I'm not 100% sure).

[bitspittle]

I'm going to go ahead and reclose this for now. If someone could show me a repro project where

1. they use Jline to create a terminal with a TerminalBuider that sets system to true and dumb to false
2. they call that jar from kscript and it doesn't crash

I would love to look and reconsider this decision.

(See also: TerminalBuilder code in Kotter)