Skip to content
Simple lexical analyzer for shell-like syntaxes
Common Lisp
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
t Initial release. Aug 8, 2019
.gitignore Initial release. Aug 8, 2019
LICENSE.txt Initial release. Aug 8, 2019
README.md Add map-tokens and do-tokens. Aug 15, 2019
package.lisp Add map-tokens and do-tokens. Aug 15, 2019
shlex.asd Initial release. Aug 8, 2019
shlex.lisp Add map-tokens and do-tokens. Aug 15, 2019

README.md

A lexer for syntaxes that use shell-like rules for quoting and commenting. It is a port of the shlex module from Python’s standard library.

The point of entry is split:

(shlex:split "a 'b c' \"d e\" \"f ' g\"")
=> '("a" "b c" "d e" "f ' g")

For a parse that is closer to a shell, you can pass the :punctuation-chars keyword:

;; Not what you want.
(shlex:split "a && b; c && d || e; f >'abc'; (def \"ghi\")")
=> '("a" "&&" "b;" "c" "&&" "d" "||" "e;" "f" ">abc;" "(def" "ghi)")

(shlex:split "a && b; c && d || e; f >'abc'; (def \"ghi\")" :punctuation-chars t)
=> ("a" "&&" "b" ";" "c" "&&" "d" "||" "e" ";" "f" ">" "abc" ";" "(" "def" "ghi" ")")

If you would rather iterate over each token in turn, you can use shlex:map-tokens or shlex:do-tokens instead.

;; Print each token in STRING.
(shlex:map-tokens #'print string :punctuation-chars t)

;; Do the same thing with a macro.
(shlex:do-tokens (token (string :punctuation-chars t))
  (print token))
You can’t perform that action at this time.