[RFC] CLI front-end restructuring

[tony]

I'm playing around with cleaning up the front end for end-users. This PR introduces the concept of using kak in a "mode" to compromise what was formerly -d (daemon), -ui ncurses, -ui dummy, -ui json, -p (stdin) and -f (filter). I believe that its important to give more clarity the many ways kak's editing functionality can be started.

While I did this I observed what other applications did (tmux, vim, etc). I had a few other choices. VIM has modes, while not used often, that still have their own options. I don't think the approach works best long term, as it takes up valuable real estate of options each time a mode is added (this freed up -d, -p, n, -f and -m). Also, I also believe that in the future users may like to have mode-specific option and help dialogs / docs, so options don't clutter up the main space.

I considered making this larger than an end-user PR (and doing more changes inside of parameters_parser) but felt I could have the effect I wanted with a smaller commit

Replace the options of -f, -ui, -d and -p with -m (mode).

While stdin, filter and daemon don't represent a UI, they do relate
to how the app is process is executed to the user.

To start application in various modes:

$ kak  # defaults to ncurses
$ kak -m daemon
$ kak -m json
$ kak -m filter
$ kak -m stdin

-f (required for -mode filter) has been renamed to -ks

-n (no kakrc files) has been renamed -nc

-s is now required for -c

-clear has been replaced with -prune

add -h (help) to print option docs

[lenormf]

What was wrong with the old options naming scheme ?

[tony]

@lenormf

I'm not using my technical hat, but my UX hat, and putting myself in the shoes of a new user with some vim experience. In that its important to separate our own familiarity / habit from what info we give to the user. Its not possible meet the standard of convincing someone its wrong if they're already familiar with it and have engrained as habit. If I could do that - I would change my career to a salesman or politician.

First, I think its a missed opportunity for the user not to see the many ways it can kak can used at a glance. Instead that have three options for -ui (which has 3 arguments it could accept, ncurses [default], json, dummy), -f (which I offered some clarification to in the docs, since "keys" is ambiguous), -d and -p. In the current form, it wasn't obvious to me what these options really done until I moved around the code for a few hours. I think even vim proper should follow the lead

Second, I think the way makes it easier to handle mode-specific options and help dialogs / docs in the future. In the initial form of this PR, I was doing a much bigger overhaul at first, but realized there are some options that change that how kak is ran that were mixed with options that specify a value or setting.

Third, there are some doc/english things that could be improved to reduce ambiguity (such as keys thing). A lot of things weren't obvious to me until I dug around the code for a bit (!)

Forth, I think its best not to use up too many options, since down the road there may be better usage for them. Its always a good thing to free those up if they're not necessary.

[alexherbo2]

I have no a strong opinion but I like how clear it is.

[lenormf]

Grouping UI frontend names with the filter option doesn't make sense to me, ditto regarding the daemon mode. Those three things are very distinct from each other, and don't all reflect a global mode which kak enters in - e.g. -f simply executes keys/commands on all the open buffers, and that's it AFAIK. I see where you're coming from, but grouping different flags under a single one requires the flag to have a common behavior.

Connecting to a session shouldn't require renaming the session itself, I don't understand why -s and -c are hardly bound now.

Using two letters for a short option name looks weird as well, we want to keep that as a plan B in case there are collisions. Plus renaming the -f flag into -ks is a bit confusing, since the option itself is filter (what's the ks stand for?).

[alexherbo2]

How about real words for options?

[tony]

How about real words for options?

I'm a real big fan of that.

I experimented with that using a header file called cxxopts.h and had great results, but not so sure how well it'd be taken seeing as we already have parameters_parser.h.

One of the things I like about cxxopts was defaults and retrieving a value in the correct type. Anything larger than that was obviously off the table to me. But the preference would to see if our current parameters_parser could do it.

@mawww : was parameters_parser meant to handle situations gnu type flags like --help also? I'd like to be able to declare options that'd work with both -h and --help.

[tony]

Using two letters for a short option name looks weird as well, ... (what's the ks stand for?).

ks is key strokes. The original documentation mentioned entering "keys". Which has a very ambigious meaning to programmers.

I do not have a strong opinion on filter's arguments names.

I do have an opinion on how filtering is implemented in general. That is a sed / awk type tool. I think it not being separated from normal-editing in some way will cause less people to recognize how it can be useful to them. What if filtering was in kakf and it could have its own manpage with examples and its own arguments / --help?

For now, I'm going to revert -ks back to -f. Also, using -f will imply -m filter.

Connecting to a session shouldn't require renaming the session itself, I don't understand why -s and -c are hardly bound now.

I'm going to revert that.

[tony]

Connecting to a session shouldn't require renaming the session itself,

Question on -s's behavior: What is -s supposed to do. The description says "set session name", for new sessions I assume. It doesn't rename old ones, right?

[mawww]

Hello,

as a general thing, we don't use gnu style --long-options in kakoune, just -long-option, there is no need for the -- prefix as we don't support compacting short options anyway.

I understand the rational about this change, however that naming "mode" seems quite broad, not sure what would be a better name though. I'd suggest we keep the -ui option to control the user interface as it is used in at least two modes: default (server + local client) and client only. daemon mode is server only, filter mode does not use any ui, pipe mode neither.

I was thinking at first that using -m <mode> with mode being one of "default, client, daemon, pipe, filter, list, prune" could be nice, but client always want a session name, so we can just pass it as argument (and kak -c <session> is quicker to write than kak -m client -s <session>), filter mode always want a list of keys, pipe always wants a session...

In the end, I am not convinced we will gain much, what could be done would be to remove the -d -s <session> and just use -d <session> but we still cannot get rid of -s in the default case.

[tony]

(I'm fine if the approach in the approach PR isn't the way, I still believe there are some usage/workflow things that could be conveyed better. I could change this PR to just be documentation changes)

a lot of the do-one-thing unix utilities are notorious for using short arguments. the thing that makes this different is, kak, as an executable does way way more than one thing.

it starts an editor in ncurses on a new server, but can also connect to a running server, start a server and send it to the bg, send to stdin. it could also open a json rpc thingy. and a tool that can run kak scriptably across a lot of files.

It took me quite a bit of bouncing around the codebase to realize the functionality. If they're fundamental to how the app should be used, I think the potential of it isn't conveyed well enough. *

can you come up with a reference points for a client/server/etc. app you use for inspiration on how you handle kak's args?

• things like tmux use subcommands tmux new-session|new-window|list-sessions, the problem is, we want positional-type args to be files
• applications like git used to do separate commands git-pull, git-merge but since moved to do subcommands git pull/git merge.
• other applications will have a urxvtc/ urxvtd/ urxvtcd, psql/ postgres (postgres having an unfathomable amount of client/server applications)
• As an example when you said "client-only mode" just then, judging by the arguments and main.cc, I thought a server is always started one way or another. If I type kak hi.txt, no server is created? Its just a local client unless there's a SIGTSTP (ctrl-z)?

[mawww]

Hello,
Here is a quick recap of various modes:

• kak hi.txt: runs single process acting as both server and client, using the process pid as the session name. if you suspend it the server part will be forked in a separate process.
• kak -c <session>: runs only the client part connected to the pre-existing server identified by <session>
• kak -d -s <sessions>: runs only the server part and identify it with <sessions>
• kak -p <session>: reads stdin, then connect to the given server and makes it run the contents of stdin as commands. no server or client is created there (well, in practice there is kind of a transient client created, but no UI is associated with it).
• kak -f <keys> <files>...: apply the given keystrokes as-if we were typing them for each file, starting from normal mode, and write the resulting state back to the file, effectively acting as a filter.

as we can see, there is 2 modes using a client, the default one and the client only one. both of these can use various UI implementations, like dummy, ncurses, or json.

• ncurses is the default UI, providing interactive edition from the terminal.
• dummy was used for testing, but is not required anymore, we could remove it eventually.
• json is now used for testing as we can compare the json output of test cases to see if the display commands are the expected ones. it can also be used to implement alternative user interfaces in separate programs: I'd expect a alternative UI to run a kak -c <session> -ui json process in the background, connect to its stdin/stdout and execute the json-rpc commands from its stdout, writing json-rpc commands for keystroke/mouse/resize events on stdin.

As ui can be used in different modes, I think having a separate option for it makes sense.

Regarding modes, as you said, we want positional arguments to be files, so subcommands is not an option, we could build multiple binaries, but that is a lot of added complexity, so I guess the two remaining directions are:

• mode selection through various switches (-f <keys>, -c <session>...): a little disorganized, but quite concise to type.
• mode selection through a single switch (-m client -s <session>, -m filter -k <keys>), more organized, but quite verbose, and we sort of have a redundancy between -m filter and the presence of -k <keys>, and -m client requires the presence of the -s <sessions> option.

My personal preference would be for the first option, but I am open to discussing the merits of alternatives, or any tweaks we can make to improve understandabilty.

Cheers,

Maxime.

[tony]

Hi Maxime,

I'm going to abandon this approach for now since its pretty stylistic. I'd like to wrap my brain around the codebase a bit more.

In my own circumstances, and my case when using vim/nvim, I'm rarely ever feeding it arguments beyond file names. I'm generally not using client/server functionality. I wonder if it'd be better if client/server/filter were split off into separate executables completely.

we could build multiple binaries, but that is a lot of added complexity

I think we could end up refactoring the entry point significantly and ultimately making things more organized for both code and for the user.

What if I did it and it was in CMake? I already have a branch make working with it. Added bonus is you get to build with Ninja.

I played around with two proof of concepts:

• https://github.com/tony/kakoune/tree/cmake cmake
• https://github.com/tony/kakoune/tree/rm-server2 kakoune without client/server

[mawww]

Hello,

The Client/Server functionality is much more important in Kakoune than in Vim/Neovim as thats both the way of having multiple windows (there is no support for window management inside kakoune, you just run another client and let tmux/the x11 wm handle the windows, although there is some integration), and the way of supporting async plugins (the plugin will launch a command in the background that would ultimately connect to the server and make it run some commands).

by

we could build multiple binaries, but that is a lot of added complexity

I was mostly talking about the user/installation side, on the code side its pretty simple, as each mode is already its own function (run_server, run_client, ...). Having multiple binaries seems just more complicated to use, for example on my local machine I dont install kakoune, I just have an alias kak=$HOME/prj/kakoune/src/kak.

Regarding CMake, I'd rather avoid it, its a fine piece of software, but Kakoune build is not complex enough to warrant it, when we can just get away with GNU make, which is installed almost everywhere. I dont believe Ninja would gain us anything significant due to the size of the project.

All that said, thanks for your interest in the project and the code base, I dont think I saw you on the IRC channel, dont hesitate to join us there (#kakoune on freenode).

Cheers,

Maxime.

[doppioandante]

For kakoune a single CMakeLists.txt should be enough to build everything, in that case we could put that in the contrib folder if someone else wants to build with cmake.

[tony]

when we can just get away with GNU make, which is installed almost everywhere.

FreeBSD/OpenBSD don't ship with GNU Make. Our make won't work with the current src/Makefile. The user has to install gmake and run gmake.

I dont believe Ninja would gain us anything significant due to the size of the project.

The boost regex thing

I ran a make of 34c8e6a on a i5-3320m:

gmake 1m48.06s real 1m45.26s user 2.68s sys
gmake -j4 54.84s real 3m9.66s user 4.85s sys

CMake + NInja (https://github.com/tony/kakoune/tree/cmake @901f255):

cmake -GNinja .. 1.28s real 0.96s user 0.33s sys
ninja 41.62s real 2m24.83s user 3.30s sys

CMake would organize the process of creating multiple executables.

Since I hop between FreeBSD as a main OS and hopping between Linux/OS X very often. CMake includes some helpful macros for finding libs too, and I only have to write it one language and not worry about GNU Make/BMake incompatibility. They are incompatible even with the most simplest of things.

POSIX Make (the happy medium that would be compatible with both GNU Make and BMake) is hair-raisingly weak. It is 2016, here is a proposal from 2013 to introduce if/else conditionals: http://austingroupbugs.net/view.php?id=805. GNU Make uses if's as you would expect, BMake, for whatever reason, uses dots before conditionals .if.

CMake is very reliable, but its no silver bullet though. I still have to add OS, and on rare rare circumstances, compiler and architecture-specific conditionals. Also, in cases like finding NCurses w/ wide chars, I had to include a custom module, as CMake's included FindCurses module didn't have a way to grab the W version. CMake's included FindBoost module finds the regex library for me.

[lenormf]

FreeBSD/OpenBSD don't ship with GNU Make. Our make won't work with the current src/Makefile. The user has to install gmake and run gmake.

As @doppioandante said, this can go into the contrib directory, the project doesn't need to have cmake's bloat by default.