What's a CLI?

[killercup]

In the first meeting, we discussed how to define what a CLI, so we can say what kind of projects we want to focus on.

@TeXitoi proposed this definition:

A program that is launch in a terminal with parameter, no user interaction during the execution, and that do work on files, network and/or output to terminal

Which also means: not an ncurses TUI, and not a daemon

(To settle this issue, write a PR that adds a definition to the Readme, and have it merged)

[Eijebong]

I missed that but sometimes CLI need to have user interaction, think package managers for example

[spacekookie]

I think the definition was later changed to

A program that is launched in a terminal with parameter, minimal user interaction during the execution, and that does work on files, network and/or output to terminal

But I don't know who is to credit with this change

[kbknapp]

I think daemons are well within the CLI space, as many CLIs have a "daemon mode" which is simply a long running background version.

I'd also say a program that prompts for input is still a CLI...however I do agree that a ncurses/TUI is out of scope

[Eijebong]

@spacekookie https://github.com/rust-lang-nursery/cli-wg/blob/b02504a38f0d3ea4ab1d049e89cfaa51c39cb66f/logs/2018-02-20-cli-wg-initial-meeting.log#L131

[Dylan-DPC-zz]

I think let's keep it independent of the user interaction since on the other side of the spectrum you can have CLI apps that require a lot of user inputs.

[kbknapp]

I'd maybe amend @vitiral's comment to:

A CLI is any program that

• Launches in a terminal
• Accepts configuration from various sources, such as command line arguments, ENV variables, or configuration files
• Runs to completion with minimal/no user interaction
• Accepts input from stdin, files, or network
• Performs processing on some input (files, network, stdin) based on the configuration specified
• Communicates via standard outputs (files, network, std{out,err})

It's long, but I think it accurately describes the scope of a CLI, while also accurately ignoring areas out of scope.

[Eijebong]

Runs to completion with minimal/no user interaction

What about, let say rustfix, you update bitflags to 1.0 in servo, the year is 2025, rustc is even more awesome and can find that those constants moved somewhere else but you don't want to --yolo it. You'd have to press Y like 1000 times, would rustfix still be considered a CLI ?

[XAMPPRocky]

I'm not sure how I feel about the exclusion of TUI & daemons based on that definition, especially as they are the most lacking areas of what could be considered CLI. If no one is interested in working on those areas that is fine, they are difficult problem areas. I think they should be included however due to their lack of attention and this a group about bringing notice to areas we find lacking in rust space. TUI's are very useful for going through multi stage input, allowing a user to go back and adjust their settings in way readline based input is not.

[kbknapp]

What about, let say rustfix, you update bitflags to 1.0 in servo, the year is 2025, rustc is even more awesome and can find that those constants moved somewhere else but you don't want to --yolo it. You'd have to press Y like 1000 times, would rustfix still be considered a CLI ?

@Eijebong in my mind, yes. Typing y 1,000 times is still minimal user input. rm -ri / would fit that, and I'd bet you'd be hard pressed to find someone who says rm isn't a CLI.

[kbknapp]

@Aaronepower I agree about the daemon comment, however from my optic about the TUI is that it's out of scope simply due to time/resources. Plus, if you're following best practices and crates that apply to our strict definition of CLIs, they will almost certainly get you to the 90% solution for a TUI. The only thing lacking at that point is UI design, which is a whole new topic in and of itself. A TUI (IMO) has more in common with GUI design than the version of CLI we're talking about.

[spacekookie]

While being a TUI developer myself I have to agree with @kbknapp. There is the excellent gyscos/Cursive crate to create terminal interfaces with different backends. And the rest of what makes up a TUI can be covered by things that any other CLI has to do too.

[XAMPPRocky]

@kbknapp As I said I'm fine with TUI's being marked as out of scope, and they should be mentioned as that. With like "TUI's are being excluded for time and resource restrictions". I don't want them to be excluded because we created a definition that conveniently leaves them out.

[Dylan-DPC-zz]

@Eijebong in my mind, yes. Typing y 1,000 times is still minimal user input. rm -ri / would fit that, and I'd bet you'd be hard pressed to find someone who says rm isn't a CLI.

@kbknapp what about CLI apps which give you input choices (like radio buttons) or users have to select from a list or type in a string?

[kbknapp]

Only my opinion, but typing in a string is fine

$ myprog
Enter your name: Kevin

Hello Kevin!

$

But a list/radio/check box/etc button is well into TUI land and thus out if scope.

Basically, I'm OK with a simple prompt, but nothing further.

[XAMPPRocky]

@kbknapp Wouldn't that definition exclude how stuff like npm init works?

[kbknapp]

I'm not familiar with npm init so possibly? When I get back to a computer I'll test it. Even without seeing it though, I'm sure there are plenty of examples of programs which straddle the line, but we have to draw the line somewhere 😉

[kbknapp]

Ok, no npm init is exactly what I mean to include from my comments above. It's just a bunch of prompts, but it's still not a TUI.

[Dylan-DPC-zz]

This is from vue cli:

[killercup]

I'm good with https://github.com/rust-lang-nursery/cli-wg/issues/4#issuecomment-367083405 plus an explicit "not TUIs for now" point as pointed out in https://github.com/rust-lang-nursery/cli-wg/issues/4#issuecomment-367091556 as well as a link to this issue :)

[vitiral]

I'm 👍 on pretty much everything here

[TeXitoi]

I'm globally OK with the consensus (i.e. small interaction as Y/N, type your name and the like).

Note that when I say "no daemon", I mean we will not work on double fork, tcp server and the like, but a lot of CLI contributions will be useful for daemons. Same for TUI.

[soc]

Sorry for jumping into this debate uninvited, I just saw this because @killercup was helping me with a library for managing config/cache/... locations.

Would it be easier to tackle the definition from a slightly different angle, where instead of defining which application are in scope or out of scope, the focus is placed on improving specific tasks that developers want to accomplish?

For instance, almost all applications need to sort out how they handle CLI arguments, some may use environment variables, and many will leverage config files, and need a way to figure out where to place cache and runtime files — whether they use a GUI or a CLI, whether they are intended to be run unattended or interactively is relatively unrelated to that in my opinion.

[Dylan-DPC-zz]

It is okay. All are invited here 😄

The reason we are discussing a scope here is so that we can focus on a small subset rather than trying to fix everything that is wrong.

[Lokathor]

A lib that works like readline (but better) is the most important part for the group to try and tackle because it's the hardest part to do. It's not something that you can obviously build on your own in a cross-platform way using other primitives already available.

Ideally it would also allow for colorized output, as well as async output.

[uazu]

Okay, so you're excluding curses-alikes. (Sounds like the original 'more' pager would fit your definition, but something more featureful like 'less' wouldn't.)

But what about bottom line status messages? i.e. writing " Status message ...\r" (flush) repeatedly. If the status message is always the same length, then there's no problem with a simplistic approach. If it gets shorter you need to either pad with spaces to the previous length or use an escape sequence to clear to EOL (but that doesn't work on Windows). Also if it overflows the terminal width things gets ugly. If there isn't a crate for that, maybe that would be useful.

[uazu]

I think one way to make a distinction of CLI versus TUI is that a CLI works like a teletype, i.e. stuff is printed and scrolls up. Only the bottom line may be rewritten (using \r or \b) for the purpose of showing progress or status, or for receiving user input (e.g. readline). Ideally it should be possible to disable any bottom line rewriting (e.g. status messages) with an option when the output will be recorded to a log file.

[Lokathor]

So I guess where I'd draw the line is something like "does it let you implement Colossal Cave Adventure, a MUD client, or similar?" This is about where Haskell's haskeline library sits. I'm deliberately including both a single-threaded program (Colossal Cave) and a multi-threaded program that needs to be able to output even while waiting for an input to complete (a MUD client).

So you have an input buffer, arrow keys let you go "up" or "down" through the input history like a shell, they also go "left" and "right" to move the insertion point in the current line, enter submits a line into the program, tab triggers a possible completion of the current word based on what's written so far and other context, and so forth.

This sort of deal can be done within ANSI terminals using escape sequences as you mentioned, but on windows we can support at least cmd.exe and PowerShell via winapi calls that do the same sorts of effects. What we do is abstract over the effect you want and then either spit some escape sequence into a terminal's stream or call a windows call depending on the current OS.

In the specific case of a bottom line status message (which might be used as a progress bar, or it might be displaying the current partial input to the program) you have to keep track of what you think the current bottom line is, and then do all of your outputs through the lib only, and then the lib will display pending outputs by deleting the current bottom line, doing the output, then putting the bottom line back in place. It's a little heavier weight than rust might normally be to ask people to use our event loop, but that's what the winit crate does for example. And if they use our lib for some stuff and then also just stick normal println all over the place, then that'll look bad maybe but it won't actually be memory-unsafe or anything, so it's ultimately fine.

[uazu]

@Lokathor, at work I wrote a CLI radio modem simulation tool in Java that does exactly that, i.e. keeps a note of the currently displayed status line, and erases it before outputting new lines ending with \n, i.e. lines that are intended to scroll up. For portability to Windows CMD, it's necessary to erase the line using only \r and spaces. I guess there's a case for replacing the status line only after the next status update, or after a timeout has expired (e.g. 100ms) to avoid too much overhead when outputting several \n lines. Obviously the code should have cleared the status if they don't want the status to be displayed any more. The result is pretty neat: Modem status indicators along the bottom, and events scrolling up, portable to Windows and compatible with redirecting to a log-file by disabling the status output with a flag. Another good example of bottom-line status includes apt-get on Debian, which uses knowledge of the terminal width to pack a lot in without overflowing (which is when it gets ugly).

[killercup]

I think we've pretty much come a consensus here. I just opened a PR to add https://github.com/rust-lang-nursery/cli-wg/issues/4#issuecomment-367083405 (and a link to this issue) to the readme: #17. Please review :)

[Lokathor]

So it looks like the whole concept of "a readline-like library" and all that is just off the table?

[uazu]

@killercup It's a bit vague. I'm still not clear whether you're going to permit readline-like functionality, or bottom-line status. Those both ideally need to get hold of terminal width somehow (env-var, ioctl, or who knows what on Windows). Readline needs to put the tty into raw mode and have some kind of a cleanup handler. But both of these are used commonly in simple CLI-style tools.

Also the ability to redirect output to a log file and get something readable and sane is a specific property of CLI-like tools that TUIs don't have. So "output readable with less -R" could be used as a requirement. (less -R option passes through only ANSI colour escapes, treats everything else as invalid binary data)

[killercup]

@Lokathor @uazu It's not a matter of "permitting" or "off the table", it's meant to help answer this question: What should this working group focus on?

If it were down to "let's talk about generating man pages" and "let's talk a good API for readline crate", I'd choose man pages. I think not focussing on improving advanced interaction modes is the right approach. These are huge topics whose design discussion alone might take months to settle. This is not meant to say you need to stop working on/thinking about this, of course! I might misjudge this, and you might be ready to rustyline 2.0 tomorrow :)

Also the ability to redirect output to a log file and get something readable and sane is a specific property of CLI-like tools that TUIs don't have.

That's a good metric!

[Lokathor]

Well, Windows doesn't even use man pages. In other words, I think that you should probably target things that will have benefit for people who aren't necessarily in a unix-like situation. Particularly, please please don't rely on windows users being on Windows 10 for whatever you make to work. It's only ~25% of the windows base.

[killercup]

@Lokathor Sorry, I meant "packaged documentation" not "troff files" when I wrote "man pages" :)

[uazu]

@killercup Looking at crates.io, there is rustyline and indicatif, so probably this doesn't need much work. (I was considering writing some crates to cover this if it wasn't done already.) So even if the CLI-WG doesn't want to directly link in one of these, or organize/motivate integrating it well with the rest of the CLI libraries, all it has to do is give a list of known well-maintained portable crates for people to look at. So there's no reason to exclude this, as it may only be half a page in some docs, or a couple of lines on a webpage. But fine if you want to concentrate on other stuff first.

[killercup]

@uazu oh yes, my stance is to not discuss the design and implementation of these crates here, making them discoverable by users is very much wanted! (See #11)

[jocutajar]

Hi Rusties, I think the debate on user interface to CLI is a bit inconclusive. Programs may have both a CLI and UI. Aptitude for instance is great in both. I think what makes a program a CLI is not the absence of UI, but rather the possibility to run unattended without an interactive terminal. For a program to run unattended, it's configuration/setup/initialization input must be predictably defined. Be it args, env vars, conf files, or stdin... If the program also has a UI as an alternate option, that is not relevant.

Similarly for daemons. CLI program should complete a given finite task and finish. If the program can also optionally run as a daemon, listening, waiting for something... that is irrelevant. Netcat for instance has an extensive CLI and can also run as a daemon.

As someone else said, TUI, GUI and daemon programs may clearly benefit from and build upon CLI features. As such I see the scope rather narrowly. Termcap/terminfo and related features are out imho. The question should be whether a program has CLI, rather than if it is CLI.

Best regards

[Dylan-DPC-zz]

hi @jocutajar

Thanks for the feedback. Yes we agree with what you are saying. It is a bit tricky for a newcomer who hasn't been there when we discussed this in the first meeting.

We agree that UI also belongs to a CLI but what we are addressing in this issue, is the scope that we are aiming to solve throughout the "first iteration" of the cli-wg. We came to a consensus that it is better to focus on CLIs with no or limited UI interaction so that we can fix up the ecosystem around such apps and after that we can look to solving any problems with UIs. Also a lot of satisfactory work has been done on TUIs so we can say it is in safe hands.