Replic

Building a readline application is cool, but readline gives you the basics and you must still build a REPL around it: loop and read commands, catch a C-c, a C-d, ask confirmation to quit, print the general help, the help of a command, setup the completion of commands, the completion of their arguments, load an init file, colorize output,… replic does this for you.

You can use replic as a ready-to-use executable or as a library.

Using the executable, you can define functions and variables in ~/.replic.lisp, give them completion candidates, and use them straight away on the replic command line.

With the library you can automatically build a REPL and turn functions you already have into commands in the readline prompt, with the process described below.

There are a few examples below, in src/examples.lisp and expect more to come.

This is an attempt at generalizing what I did several times with cl-readline.

What this is not:

• this is not a Lisp REPL. See cl-repl for that (itself not a replacement for Slime ;) )
• this is not a shell. See shcl or Lish.

Example applications built on replic:

• cl-torrents
• OpenBookStore
• here’s a commit that added replic capabilities to a library. The library was to be used on the Lisp REPL. With the creation of an executable and 5 lines of replic setup, we can use it in the terminal, as a readline-based application with nice autocompletion.

Installation

You can download the executable (a 14MB zipped download, a 60Mo GNU/Linux x64 self-contained binary, instant start-up !), make it executable (chmod +x replic) and run it: ./replic,

or build it yourself.

• the library is on Quicklisp and on Ultralisp:

(ql:quickload "replic")

or clone this repo into ~/quicklisp/local-projects/,

then build the executable with make build.

Run it:

./replic -h

Available options:
  -h, --help               Print this help and exit.
  -q, --quiet              Do not load the init file.
  -l, --load ARG           Load the given file.

./replic

and see the available commands:

replic > help
replic > help help

Now add commands in your lisp init file (see next section) or build an application with it (see the Developer section).

User: the executable and the init file

Given the example ~/.replic.lisp below, you can try hello <name> (completion for hello) and goodbye <name>, where <name> can be completed from what was given to hello.

(in-package :replic.user)

(defparameter *names* '()
  "List of names (string) given to `hello`. Will be autocompleted by `goodbye`.")

(defun hello (name)
  "Takes only one argument. Adds the given name to the global
  `*names*` variable, used to complete arguments of `goodbye`.
  "
  (format t "hello ~a~&" name)
  (push name *names*))

(defun goodbye (name)
  "Says goodbye to name, where `name` should be completed from what was given to `hello`."
  (format t "goodbye ~a~&" name))

(replic.completion:add-completion "goodbye" (lambda () *names*))

(export '(hello goodbye))

Note that only the export‘ed functions and parameters will be taken into account.

See more examples in the src/examples.lisp file of this repository.

Define a default completion function for a command’s arguments

First write a function or a variable and export it. It becomes a command in the command line interface.

You can tell a command to complete its arguments against a given variable or function:

(replic.completion:add-completion "goodbye" (lambda () *names*))
;; or
(replic.completion:add-completion "goodbye" #'my-function)

Now everytime you type goodbye fooTAB, the lambda function is run and you get completion candidates that start with “foo”.

The functions must return a list of strings.

When you have many functions whose arguments should be completed similarly, you can set a default completion function:

(setf replic.completion:*default-command-completion* #'my-function)

A different completion function for each argument

Each parameter of a command can be completed with its own method.

Let’s define a command say that wants first a greeting message, and then a name:

(defun say (verb name)
  (format t "~a, ~a !~&" verb name))

We can provide the completion functions in the same order as the arguments:

(replic.completion:add-completion "say"
                                  (list "greetings" "\"nice to see you\"")
                                  (lambda () *names*))

Now if you type say TAB you get the two greeting choices. After you pick one and press TAB again, you get the names that were given to hello.

Built-in commands

You get a built-in help command that shows the documentation of functions and variables:

replic > help

Available commands
==================
help       ... Print the help of all available commands.
reload     ... NIL
set        ... Change a variable, see its value, or see all variables.
quit       --- Quit the application.

Available variables
===================
*verbose*  ... If true, print debugging information during the program execution.

Write a preamble and a postamble in *help-preamble* and *help-postamble*.

You can read the help of a specific command or variable (with completion):

help help

The general help shows the first paragraph of the functions/parameters docstring, the help <cmd> function is more complete and shows all of it.

Setting and seeing variables

set can be used with zero, one or two arguments:

set

shows all available variables,

set *variable*

this prints the value of this variable (use auto-completion),

set *variable* new-value

and this sets a new value. “yes”, “true” and “t” denote true.

We kept the “earmuffs” to denote variables.

Configuration file

Replic reads an init-like configuration file. It searches a .replic.conf file under ~/.config/ and at the user’s home directory (~/.replic.conf).

These are the default parameters with their default values:

[default]
confirm-exit = true
verbose = false
prompt = > 
history = true
write-history = true

“true”, “True” and “t” are truthy and “false”, “False” and “nil” are falsy.

By default, replic reads and sets the options of the [default] section.

You can have a section per program:

[myprogram]
option = val

Options of config files are overriden by command line arguments.

Developer: using replic as a library with an existing system

replic is in Quicklisp:

(ql:quickload "replic")

Follow the documentation below, and see example applications on the wiki.

Change the prompt

You can change the prompt. It defaults to “> “. It can contain ansi colors.

(setf replic:*prompt* (cl-ansi-text:green "replic > "))

You can add a prefix to it, for example one that changes with the state of the application (current directory,…):

(setf replic:*prompt-prefix* (format t "(~a) " "sthg"))

and concatenate the two with (replic:prompt).

[optional] Load base commands (help, reload, set)

If you want to have the base commands (help, reload, set, quit), import the base package:

(replic.completion:functions-to-commands :replic.base)

Create commands from a package’s exported functions

This is the core of the library.

Create the commands you’ll find at the readline prompt from the exported functions and variables of a given package:

(replic.completion:functions-to-commands :my-package)

To exclude functions, use the :exclude list:

(replic.completion:functions-to-commands :my-package :exclude '("main"))

For more control, you can create a command from one given function:

(replic.completion:add-command :function :package)
;; add a variable:
(replic.completion:add-variable :*variable* :package)

It is generally a good idea to have a package for the lisp functions you’ll use at the repl, and another package for the ones that must be commands at the readline interface.

[optional] Automatically printing the result of functions

A lisp function from a library usually returns some result and doesn’t necessarily print it. If you want replic to automatically print it, ask it like so:

(replic:autoprint-results-from :my-package :exclude '("exclude" "those-functions"))

[optional] Overriding the default printing of results

We export a default print-result (result) function, which is called for functions whose results are printed automatically (see autoprint-results-from and autoprint-results-p).

A user can override this function in his/her lisp init file:

;; ~/.replic.lisp
(in-package :replic)

(defun print-result (result)
  (format t "=== this new result is:~&")
  (format t "~a~&" result))

In doing so, you should see a warning at startup:

WARNING: redefining REPLIC:PRINT-RESULT in DEFUN

Load a config file

replic searches by default for a .replic.conf (see above). The function replic.config:apply-config takes as paramaters:

• (warn: the parameters order was changed on Jan, 2023) an optional section parameter (string), defaults do the “default” section.
• an optional package name, defaults to :replic.

If you do this:

(replic.config:apply-config)

this will read the settings inside the “default” section, and it will apply them to the parameters of the replic package. So, you can change “confirm-exit” and other built-in parameters (see below).

If you have a config file with another section:

[default]
confirm-exit: true

[my-app]
confirm-exit: false

You would read the “my-app” section with:

(replic.config:apply-config "my-app")

this still tries to set replic’s default parameters.

WARN: this is less tested.

If you do:

(replic.config:apply-config "my-app" :my-app-package)

this will try to set the parameters of your own application.

As an optional third parameter, you can give another file name:

(replic.config:apply-config :mypackage ".mysoftware.conf")

Default parameters

The exported variables from the package you give as argument can be overriden in the config file. For example, the :replic package exports:

(:export :main
         :confirm
         :repl
         :help
         :set
         :reload
         ;; settings        ;; <--- exported *parameters* start here.
         :*help-preamble*
         :*help-postamble*
         :*prompt*
         :*prompt-prefix*
         :*confirm-exit*
         :*write-history*
         :*verbose*))

so we can configure:

[default]
write-history = true
verbose = true
prompt = my silly prompt

and so on.

Start the repl

Start the repl:

(replic:repl)

That’s it. You didn’t have to write the REPL.

Settings

Variables that are exported from a package on the lisp side will be automacitally available for the config file and read when the application starts up. The rule is that in the config file, we don’t use earmuffs (*foo* -> foo). Lispers shall use a lispy config file anyway.

The available variables are:

• *verbose* (bool): if true, print debugging information during the program execution.
• *confirm-exit* (bool): if true (the default), ask for confirmation when a user tries to exit the program with a C-d (EOF).
• *prompt* (str): the readline prompt. Defaults to simply > . Can contain ansi colours (use cl-ansi-text:green for example).
• *confirm-exit* (t or nil): if t (the default), ask for confirmation when the user tries to exit the command line with a C-d (EOF).
• *write-history* (t or nil): if t (the default), write the commands to the app’s history. (this needs cl-readline superior to may, 2018)
• *help-preamble*: text to display at the beginning of the help.
• *help-postamble*: text to display last.

Other helpers

• print colored output from markdown or code with pygments: (format-markdown txt :lang "md"). It outputs text for a console display with ansi colours. Needs pygments, or does nothing.

Readline settings

The GNU Readline library provides settings you might take advantage of. We can set the settings in the readline init file (~/.inputrc by default, obeys the INPUTRC environment variable).

For example, you can change the completion behavior. This:

TAB: menu-complete

inserts the first completion candidate, even if there are many, instead of showing the list of choices under the prompt.

If you prefer vi mode:

set editing-mode vi

etc. See readline’s documentation.

Dev

This is a generalization on cl-readline. See also the simple cl-readline example. Once you’ve built two even basic readline apps you’ll want to factorize the common parts.

We want to store a list of commands (functions, “verbs”) and a list of variables (the ones to use with “set”). We want to read them from any Lisp file, hence we need to remember the package they come from. This mechanism is provided through an interface in completion.lisp.

Clone this repo in QL’s local projects (~/quicklisp/local-projects).

Build the executable:

make build

You can build the binary with SBCL core compression (see commented .asd). We passed from a 78 to a 18MB binay, but the startup time increased from 0.04 to 0.26s, which is noticeable. We don’t use compression by default.

Develop and test interactively into the console

By starting a swank server in the (real) Lisp repl we can compile code in our editor and try instantly in the terminal, without re-building the executable. See this cl-charms crash course for now. Some details need fixing.

Simpler and still handy, you can add trace statements into your .replic.lisp, call the reload command and see the effects. Then, (untrace) and reload.

Changelog

dev

• 2023-01: updated reading a section. Fix loading the ini file for another app. To load the ini file, use:

(replic.config:apply-config)

this will read the “default” section and will check the parameters of the replic package. Optionally, you can read another section of the ini file:

[default]
confirm-exit: true

[my-app]
confirm-exit: false

and load it:

(replic.config:apply-config "my-app")

• read an option from a given section.

This:

(replic.config:apply-config :myprogram)

only reads and sets options of the “myprogram” section and set the matching variables found in :myprogram.

• July, 2022: added with-rl-completion
• v0.12, upcoming in Quicklisp of november
  • added: a different completion for each command argument
  • added: completion for sentences (strings in quotes).
• Quicklisp, october 2019
  • fixed 0.11 regression: arguments had to always be surrounded by quotes (sept, 14th). We can now write command arg1 "second arg" as expected.
• v0.11 (end of june, 2019)
  • added a declarative way to automatically print a function’s result. The default function can be overriden by users (in order to, for example, color the output).
  • fixed: a quoted string on the readline prompt is now understood as one single argument.

Resources

• cl-readline
• cl-readline-example

Learning:

• Common Lisp Cookbook
• https://github.com/CodyReichert/awesome-cl#learning-and-tutorials

Getting started:

• Common Lisp editors (Emacs, Portacle, Vim, Lem, Atom, Sublime), notebooks, REPLs
• https://lispcookbook.github.io/cl-cookbook/getting-started.html